Skip to content

References and Deep Merging

pytest-httpchain supports JSON references for reusing scenario components across files. References are resolved with deep merging, allowing you to compose scenarios from shared fragments.

$include / $merge vs $ref

Three directives are supported and work identically:

  • $include (recommended): Avoids conflicts with VS Code's JSON Schema validation
  • $merge (recommended): Alias for $include, semantically clearer when merging properties
  • $ref: Standard JSON Reference syntax, but may cause VS Code/IDE validation warnings
// Recommended - no VS Code conflicts
{ "$include": "common.json#/headers" }
{ "$merge": "base.json", "extra": "value" }

// Also works, but may show VS Code warnings
{ "$ref": "common.json#/headers" }

Basic Syntax

Reference another file:

{
    "$ref": "path/to/file.json"
}

Reference a specific key within a file:

{
    "$ref": "path/to/file.json#/key/path"
}

File References

Same Directory

{
    "$ref": "common.json"
}

Relative Paths

{
    "$ref": "../shared/auth.json"
}

Nested Directories

{
    "$ref": "fragments/requests/login.json"
}

JSON Pointer References

Reference specific keys using JSON Pointer syntax:

common.json: 

{
    "headers": {
        "default": {
            "Content-Type": "application/json",
            "Accept": "application/json"
        },
        "auth": {
            "Authorization": "Bearer {{ token }}"
        }
    },
    "requests": {
        "login": {
            "url": "https://api.example.com/login",
            "method": "POST"
        }
    }
}

test_scenario.http.json: 

{
    "stages": [
        {
            "name": "login",
            "request": {
                "$ref": "common.json#/requests/login",
                "headers": {
                    "$ref": "common.json#/headers/default"
                }
            }
        }
    ]
}

Deep Merging

When a $ref is used alongside other properties, the siblings are deep merged into the referenced content — they add to it. A sibling cannot change a value the reference already sets; see Merge Rules.

base.json: 

{
    "request": {
        "url": "https://api.example.com/users",
        "headers": {
            "Content-Type": "application/json"
        },
        "timeout": 30
    }
}

test_scenario.http.json: 

{
    "stages": [
        {
            "name": "custom_request",
            "$ref": "base.json",
            "request": {
                "method": "POST",
                "headers": {
                    "X-Request-Id": "abc-123"
                }
            }
        }
    ]
}

The sibling request adds method and a new header. The nested headers object is merged recursively, so the referenced Content-Type is kept alongside the added X-Request-Id, and url/timeout carry through from base.json untouched.

Resolved result: 

{
    "stages": [
        {
            "name": "custom_request",
            "request": {
                "url": "https://api.example.com/users",
                "method": "POST",
                "headers": {
                    "Content-Type": "application/json",
                    "X-Request-Id": "abc-123"
                },
                "timeout": 30
            }
        }
    ]
}

Merge Rules

A $ref (or $include/$merge) and its sibling properties are combined by additive deep merge: siblings extend the referenced value, they do not override it.

  1. Objects: Recursively merged — sibling keys are added, and keys present in both are merged by these same rules.
  2. Arrays: Concatenated — referenced elements first, then sibling elements. Arrays are not replaced and not merged element-by-element.
  3. Scalars: A sibling must match the referenced value. Any differing scalar raises a merge conflict at load time (Merge conflict at <path>).
  4. Type mismatch: Combining different JSON types at the same path (object vs array, scalar vs object, …) raises a merge conflict.

null is the one exception: a null on either side of a path is always accepted and the sibling wins — so a sibling null can blank out a referenced value of any type.

References add, they don't override. To change a value a fragment already sets, don't merge over it — keep that key out of the shared fragment (so the local scenario is its only writer), or point the $ref at a sub-node that omits it. Trying to replace a referenced scalar with a different one is a load-time error by design, so a shared fragment can never be silently contradicted.

Composing Scenarios

Shared Stage Fragments

fragments/stages.json: 

{
    "login": {
        "name": "login",
        "request": {
            "url": "https://api.example.com/auth/login",
            "method": "POST",
            "body": {
                "json": {
                    "username": "{{ username }}",
                    "password": "{{ password }}"
                }
            }
        },
        "response": [
            {"verify": {"status": 200}},
            {"save": {"jmespath": {"token": "access_token"}}}
        ]
    },
    "logout": {
        "name": "logout",
        "always_run": true,
        "request": {
            "url": "https://api.example.com/auth/logout",
            "method": "POST",
            "headers": {
                "Authorization": "Bearer {{ token }}"
            }
        }
    }
}

test_workflow.http.json: 

{
    "substitutions": [
        {
            "vars": {
                "username": "testuser",
                "password": "testpass"
            }
        }
    ],
    "stages": [
        {
            "$ref": "fragments/stages.json#/login"
        },
        {
            "name": "do_something",
            "request": {
                "url": "https://api.example.com/action",
                "headers": {
                    "Authorization": "Bearer {{ token }}"
                }
            }
        },
        {
            "$ref": "fragments/stages.json#/logout"
        }
    ]
}

A fragment file may carry its own top-level $schema key for editor support — wherever the fragment lands in the referencing scenario, validation discards the key. A fragment that is a JSON Schema (e.g. pulled into a verify step's schema field) keeps its $schema dialect declaration, since that position is data rather than scenario structure.

Shared Configuration

config/defaults.json: 

{
    "ssl": {
        "verify": true
    },
    "auth": "auth_module:get_default_auth",
    "substitutions": [
        {
            "vars": {
                "base_url": "https://api.example.com",
                "timeout": 30
            }
        }
    ]
}

test_with_defaults.http.json: 

{
    "$ref": "config/defaults.json",
    "stages": [
        {
            "name": "test",
            "request": {
                "url": "{{ base_url }}/test",
                "timeout": "{{ timeout }}"
            }
        }
    ]
}

Security: Path Traversal Limits

The ref_parent_traversal_depth configuration limits how many ../ segments are allowed:

# pytest.ini
[pytest]
ref_parent_traversal_depth = 3

With depth 3, these are valid: - ../file.json - ../../file.json - ../../../file.json

This would fail: - ../../../../file.json

Circular Reference Detection

pytest-httpchain detects and prevents circular references:

a.json: 

{
    "$ref": "b.json"
}

b.json: 

{
    "$ref": "a.json"
}

This will raise an error during scenario loading.

Best Practices

  1. Organize by purpose: Group related fragments (auth, common headers, base configs)
  2. Use meaningful paths: fragments/auth/login.json vs f1.json
  3. Keep references shallow: Deeply nested refs are harder to debug
  4. Document shared files: Add comments about expected variables
  5. Version shared fragments: Consider separate directories for breaking changes