References and Deep Merging¶

pytest-httpchain supports JSON references for reusing scenario components across files. References are resolved with deep merging, allowing you to compose scenarios from shared fragments.

`$include` / `$merge` vs `$ref`¶

Three directives are supported and work identically:

$include (recommended): Avoids conflicts with VS Code's JSON Schema validation
$merge (recommended): Alias for $include, semantically clearer when merging properties
$ref: Standard JSON Reference syntax, but may cause VS Code/IDE validation warnings

// Recommended - no VS Code conflicts
{ "$include": "common.json#/headers" }
{ "$merge": "base.json", "extra": "value" }

// Also works, but may show VS Code warnings
{ "$ref": "common.json#/headers" }

Basic Syntax¶

Reference another file:

{
    "$ref": "path/to/file.json"
}

Reference a specific key within a file:

{
    "$ref": "path/to/file.json#/key/path"
}

File References¶

A referenced file may have any name: common.jsonc works as common.json does. Every file is read as JSON with comments, so a shared fragment can explain itself:

{
    // Headers every request to the API sends.
    "headers": {
        "Accept": "application/json",
        "X-Client": "pytest-httpchain", /* identifies the test traffic */
    },
}

Same Directory¶

{
    "$ref": "common.json"
}

Relative Paths¶

{
    "$ref": "../shared/auth.json"
}

Nested Directories¶

{
    "$ref": "fragments/requests/login.json"
}

Lookup Order¶

A relative reference path is looked up against two bases, in order:

the referencing file's directory (the file containing the $ref);
the root path — pytest's rootdir when collecting, and when using the CLI the rootdir pytest would determine for the same paths, or --root-path.

The first base under which the file exists wins. This lets a suite keep fragments next to the scenarios that use them and reference shared fragments by a root-relative path — but it also means the same string can name two different files. When a file exists under both bases, the file-relative one wins and an AmbiguousReferenceWarning is emitted (reported as the HTTPCHAIN026 diagnostic by pytest-httpchain validate), because dropping a file next to a scenario silently changing what its references mean is exactly the kind of surprise you want flagged. Rename one of the files to resolve the ambiguity.

JSON Pointer References¶

Reference specific keys using JSON Pointer syntax:

common.json:

{
    "headers": {
        "default": {
            "Content-Type": "application/json",
            "Accept": "application/json"
        },
        "auth": {
            "Authorization": "Bearer {{ token }}"
        }
    },
    "requests": {
        "login": {
            "url": "https://api.example.com/login",
            "method": "POST"
        }
    }
}

test_scenario.http.json:

{
    "stages": [
        {
            "name": "login",
            "request": {
                "$ref": "common.json#/requests/login",
                "headers": {
                    "$ref": "common.json#/headers/default"
                }
            }
        }
    ]
}

Deep Merging¶

When a $ref is used alongside other properties, the siblings are deep merged into the referenced content — they add to it. A sibling cannot change a value the reference already sets; see Merge Rules.

base.json:

{
    "request": {
        "url": "https://api.example.com/users",
        "headers": {
            "Content-Type": "application/json"
        },
        "timeout": 30
    }
}

test_scenario.http.json:

{
    "stages": [
        {
            "name": "custom_request",
            "$ref": "base.json",
            "request": {
                "method": "POST",
                "headers": {
                    "X-Request-Id": "abc-123"
                }
            }
        }
    ]
}

The sibling request adds method and a new header. The nested headers object is merged recursively, so the referenced Content-Type is kept alongside the added X-Request-Id, and url/timeout carry through from base.json untouched.

Resolved result:

{
    "stages": [
        {
            "name": "custom_request",
            "request": {
                "url": "https://api.example.com/users",
                "method": "POST",
                "headers": {
                    "Content-Type": "application/json",
                    "X-Request-Id": "abc-123"
                },
                "timeout": 30
            }
        }
    ]
}

Merge Rules¶

A $ref (or $include/$merge) and its sibling properties are combined by additive deep merge: siblings extend the referenced value, they do not override it.

Objects: Recursively merged — sibling keys are added, and keys present in both are merged by these same rules.
Arrays: Concatenated — referenced elements first, then sibling elements. Arrays are not replaced and not merged element-by-element. A verify.status list and a stage's retry.on are exceptions, below.
Scalars: A sibling must match the referenced value. Any differing scalar raises a merge conflict at load time (Merge conflict at <path>).
Type mismatch: Combining different JSON types at the same path (object vs array, scalar vs object, …) raises a merge conflict.

null is not an exception: it is a value like any other, not an override or a hole. A null paired with a different value at the same path is a merge conflict; two nulls merge fine.

Equal means equal as JSON, at any depth: true and 1 differ, and so do [true] and [1], while 1 and 1.0 are one value.

References add, they don't override. To change a value a fragment already sets, don't merge over it — keep that key out of the shared fragment (so the local scenario is its only writer), or point the $ref at a sub-node that omits it. Trying to replace a referenced scalar with a different one is a load-time error by design, so a shared fragment can never be silently contradicted.

Status lists merge whole¶

Elsewhere, concatenating arrays only adds — more steps, more expressions that must hold, more contains strings — so a sibling can extend a fragment but never weaken it. A verify.status list is different: its entries are alternatives, any one of which passes, so a longer list accepts more. Concatenating would let a sibling quietly loosen the fragment's check — a negative test's [404] merged onto a shared ["2xx"] would become ["2xx", 404] and pass on a 200. A verify.status list therefore merges like a scalar: an equal list is kept, and a different one is a merge conflict:

common.json:

{
    "ok": {"verify": {"status": ["2xx"]}}
}

{"$merge": "common.json#/ok", "verify": {"status": [404]}}

Merge conflict at verify.status

To accept more codes, list them all in one place. A reference inside status ("status": {"$include": "codes.json#/accepted"}) still resolves as usual.

A stage's retry.on lists alternatives too: any one kind of failure it names makes another attempt. A sibling ["request"] written beside a shared {"on": ["verify"]} to narrow it would retry both, resending a request that timed out, so retry.on merges whole as well (Merge conflict at on).

JMESPath expectations merge whole¶

What one verify.jmespath expression must give is one value, so it merges like a scalar too. Concatenated, a sibling's ["b"] on a fragment's ["a"] would assert ["a", "b"], which neither wrote, and two objects under eq would blend into a third. An equal expectation is kept and a different one, a matcher included, is a merge conflict; different expressions still merge side by side:

common.json:

{
    "checks": {"verify": {"jmespath": {"tags": ["a"], "price": {"gt": 0}, "meta": {"eq": {"page": 1}}}}}
}

{"$merge": "common.json#/checks", "verify": {"jmespath": {"count": 3}}}

adds the count check to the fragment's three, while

{"$merge": "common.json#/checks", "verify": {"jmespath": {"price": {"lt": 100}}}}

fails with Merge conflict at verify.jmespath.price: write the whole matcher in one place, or build it from the shared one by writing the reference at the expectation itself. There its siblings are one value composed on purpose, not a second one written for the same expression, so they merge key by key:

{"verify": {"jmespath": {"price": {"$merge": "common.json#/checks/verify/jmespath/price", "lt": 100}}}}

checks {"gt": 0, "lt": 100}. A key both sides give is one operand, and must still agree whole, an array or an object as much as a number:

{"verify": {"jmespath": {"meta": {"$merge": "common.json#/checks/verify/jmespath/meta", "eq": {"size": 2}}}}}

fails with Merge conflict at eq rather than checking {"page": 1, "size": 2}, which neither side wrote, and two contains arrays conflict rather than concatenate. To build an operand from a shared one, write the reference inside it:

{"verify": {"jmespath": {"meta": {"eq": {"$merge": "common.json#/checks/verify/jmespath/meta/eq", "size": 2}}}}}

checks {"eq": {"page": 1, "size": 2}}.

Composing Scenarios¶

Shared Stage Fragments¶

fragments/stages.json:

href="#__codelineno-21-1">{ "login": { "name": "login", "request": { "url": "https://api.example.com/auth/login", "method": "POST", "body": { "json": { "username": "{{ username }}", "password": "{{ password }}" } } }, "response": [ {"verify": {"status": 200}}, {"save": {"jmespath": {"token": "access_token"}}} ] }, "logout": { "name": "logout", "always_run": true, "request": { "url": "https://api.example.com/auth/logout", "method": "POST", "headers": { "Authorization": "Bearer {{ token }}" } } } }

test_workflow.http.json:

{
    "substitutions": [
        {
            "vars": {
                "username": "testuser",
                "password": "testpass"
            }
        }
    ],
    "stages": [
        {
            "$ref": "fragments/stages.json#/login"
        },
        {
            "name": "do_something",
            "request": {
                "url": "https://api.example.com/action",
                "headers": {
                    "Authorization": "Bearer {{ token }}"
                }
            }
        },
        {
            "$ref": "fragments/stages.json#/logout"
        }
    ]
}

A fragment file may carry its own top-level $schema key for editor support — wherever the fragment lands in the referencing scenario, validation discards the key. Inline verify.body.schema values are the exception to resolution itself: that position holds a standard JSON Schema, so the resolver leaves the whole subtree untouched — its $ref/$defs/$schema belong to the schema validator, which resolves a $ref to a local file itself (relative to the scenario file's directory, unless an $id sets another base), and scenario directives ($include/$merge) are not processed there (the validator warns with HTTPCHAIN028). The opacity extends to sibling merging: two differing schema values arriving at the same position via $merge are a merge conflict, never blended — an opaque subtree merges atomically, like a scalar. To share a schema between scenarios, use the file-path form ("schema": "./schemas/user.json", or "./openapi.json#/components/schemas/User" for one inside a document) or a JSON Schema $ref, rather than a reference directive.

Shared Configuration¶

config/defaults.json:

{
    "ssl": {
        "verify": true
    },
    "auth": {"bearer": "{{ env('API_TOKEN') }}"},
    "client": {
        "base_url": "https://api.example.com",
        "headers": {
            "Accept": "application/json"
        },
        "timeout": 30
    }
}

test_with_defaults.http.json:

{
    "$ref": "config/defaults.json",
    "stages": [
        {
            "name": "test",
            "request": {
                "url": "/test"
            }
        }
    ]
}

The client block gives every stage the base URL, headers and timeout, so the stages spell out only their own path. A value a stage takes from a reference counts as the stage's own: a request fragment $included with "timeout": 30 keeps 30 seconds whatever client.timeout says.

Security: Path Traversal Limits¶

The httpchain_ref_parent_traversal_depth configuration limits how many ../ segments are allowed:

# pytest.ini
[pytest]
httpchain_ref_parent_traversal_depth = 3

With depth 3, these are valid: - ../file.json - ../../file.json - ../../../file.json

This would fail: - ../../../../file.json

The root path is a containment boundary¶

The traversal depth is not the only constraint. Every resolved reference must also stay inside the root path — the directory root_path names above. A reference that resolves outside it is rejected even when the file exists and the ../ count is within the limit; symlinks are resolved before the check, so a link pointing out of the tree is rejected too. Absolute paths are refused outright.

The root is pytest's rootdir during collection. The CLI determines the rootdir pytest would for a run on the paths it is given, from the current directory: the directory of the configuration file pytest would read (pytest.toml, pytest.ini, a pyproject.toml with a [tool.pytest] or [tool.pytest.ini_options] table, ...), else of the nearest setup.py, else the common ancestor of the current directory and the paths. --root-path sets it explicitly, as a run with --rootdir or -c needs:

pytest-httpchain validate --root-path . tests/test_login.http.json

A reference rejected this way says so specifically — resolves outside the reference root <dir> — rather than reporting the file as missing.

The same three rules — relative paths only, the traversal depth, the root — hold for a JSON Schema $ref to a file inside a verify.body.schema (see references between documents).

Circular Reference Detection¶

pytest-httpchain detects and prevents circular references:

a.json:

{
    "$ref": "b.json"
}

b.json:

{
    "$ref": "a.json"
}

This will raise an error during scenario loading.

Best Practices¶

Organize by purpose: Group related fragments (auth, common headers, base configs)
Use meaningful paths: fragments/auth/login.json vs f1.json
Keep references shallow: Deeply nested refs are harder to debug
Document shared files: Add comments about expected variables
Version shared fragments: Consider separate directories for breaking changes

References and Deep Merging¶

$include / $merge vs $ref¶

Basic Syntax¶

File References¶

Same Directory¶

Relative Paths¶

Nested Directories¶

Lookup Order¶

JSON Pointer References¶

Deep Merging¶

Merge Rules¶

Status lists merge whole¶

JMESPath expectations merge whole¶

Composing Scenarios¶

Shared Stage Fragments¶

Shared Configuration¶

Security: Path Traversal Limits¶

The root path is a containment boundary¶

Circular Reference Detection¶

Best Practices¶

`$include` / `$merge` vs `$ref`¶