Validation diagnostics

Every finding from the scenario validator carries a stable diagnostic code (HTTPCHAINxxx), a severity, a human-readable message and (where meaningful) a location, so tooling can filter, sort and route diagnostics deterministically.

You meet these codes in three places:

• pytest-httpchain validate output (and its --format json payload);
• pytest collection: error-severity findings fail collection, warnings are emitted as ScenarioValidationWarning in the form [HTTPCHAINxxx] ...;
• CI gates: validate --strict exits non-zero on warnings too.

Codes are append-only: a code's meaning never changes, and retired checks do not free their numbers for reuse.

Code reference

Code	Severity	Meaning
HTTPCHAIN000	error	Schema validation failed (Pydantic Scenario model)
HTTPCHAIN001	error	Duplicate stage names
HTTPCHAIN002	error	Fixture and variable share the same name
HTTPCHAIN003	warning	Variable referenced but never defined/saved/fixture (typo), or saved only by earlier stages with a skip_if or a skip, skipif or xfail mark of their own, which leave it undefined when they skip (read it with get())
HTTPCHAIN004	warning	Variable referenced before it is saved or defined — saved by a later stage or by a later step of the same stage's response, or defined by a later substitution step (ordering / data-flow)
HTTPCHAIN005	warning	Stage has no verify step (no response validation)
HTTPCHAIN006	warning	Verify step asserts nothing (no-op)
HTTPCHAIN007	error	Body or header matcher contains/not_contains list the same substring, or a verify.jmespath matcher's contains and not_contains are the same JSON value
HTTPCHAIN008	error	Body, header or verify.jmespath matcher matches/not_matches list the same pattern
HTTPCHAIN009	warning	Saved variable is shadowed by a scenario-level fixture
HTTPCHAIN010	error	File not found
HTTPCHAIN011	error	Path is not a file
HTTPCHAIN012	error	$ref resolution failed
HTTPCHAIN013	warning	File extension is neither .json nor .jsonc
HTTPCHAIN014	error	Invalid JSON in the scenario or a file it includes, which the message then names: a syntax error (a /* comment never closed is one, reported at its opening; see Comments and trailing commas), a duplicate object key, bytes that are not UTF-8, or an integer too long to parse
HTTPCHAIN015	error	Failed to parse JSON file (for example, nested too deeply to parse)
HTTPCHAIN016	error	Fixture referenced in a scenario-level template
HTTPCHAIN017	error	Scenario-level template references an undefined name
HTTPCHAIN018	warning	Verify expression is not a template ({{ }}) — cannot evaluate to the required bool
HTTPCHAIN019	error	Invalid pytest marker expression (scenario or stage marks)
HTTPCHAIN020	warning	Referenced file does not exist (deep, opt-in): a file path (ssl, a binary body, a file a files or multipart body uploads), a body schema file, or a local file a body schema's $ref or $dynamicRef names
HTTPCHAIN021	warning	A body schema cannot be used (deep): its file is not JSON, its JSON pointer leads nowhere, an $id on the pointer's way or in the schema cannot be read (not a string, or not a URI), the schema it selects is not valid, or a $ref or $dynamicRef it reaches does not resolve (not a string, remote, an absolute path, more ../ than httpchain_ref_parent_traversal_depth, outside the root, a pointer to nothing, a malformed $id) or points to an invalid schema
HTTPCHAIN022	warning	User function cannot be imported (deep)
HTTPCHAIN023	warning	Unexpected argument passed to a user function (deep)
HTTPCHAIN024	warning	Missing required argument for a user function (deep)
HTTPCHAIN025	info	Template parametrize values force collection-time resolution
HTTPCHAIN026	warning	$ref path matches files under both lookup bases (ambiguous)
HTTPCHAIN027	warning	User-defined name shadowed by the reserved response namespace
HTTPCHAIN028	warning	Scenario directive ($include/$merge) inside an inline JSON Schema — not resolved there; a JSON Schema $ref is, to a local file too
HTTPCHAIN029	warning	Template expression in a dict key — only values are substituted, so the key is sent literally (a verify.jmespath key is evaluated as written; one that is not valid JMESPath fails validation instead). Also raised for an escaped \{{ in a key, which does nothing there: the key keeps its backslash, and its braces need no escape
HTTPCHAIN030	warning	Template expression in a functions substitution's kwargs — kwargs are passed to the function unrendered, so it arrives as literal text. Also raised for an escaped \{{ there, which arrives with its backslash
HTTPCHAIN031	error	xdist_group marker in a stage's marks, naming a group the scenario does not declare — under --dist loadgroup it runs that stage apart from the rest of the scenario; put it in the scenario's marks (see pytest-xdist)
HTTPCHAIN032	error	Stage name contains ::, pytest's node-id separator — the stage cannot be run by its node id, and --dist loadscope runs it apart from the rest of the scenario
HTTPCHAIN033	error	Scenario's xdist_group name has a ] after its last @ — pytest-xdist ignores such a group, so --dist loadgroup does not keep the scenario's stages together
HTTPCHAIN034	error	Stage request.url is relative (/users/1, or /users/{{ id }}), but the scenario's client sets no base_url to resolve it against (see Client configuration)
HTTPCHAIN035	warning	A built-in function that is no use as a value (a time, encoding, URL or hashing helper, uuid4, env, rand or randint) used without calling it ({{ now }} for {{ now() }}, {{ env }}, or str(timestamp)) — the template gets the function itself, not its value, and one that renders to a function fails the stage (a parametrize value, collection; a scenario-level template, scenario initialization, or collection for the substitutions of a scenario whose parametrize values are templates, which resolves them there)
HTTPCHAIN036	warning	A built-in's name the scenario defines too, used as a function where that definition is not in scope (in another stage, in a scenario-level template, in a step before the one defining it): called (timestamp()) where the scenario's is a fixture or function substitution, or handed to a function that takes one: as a key= (sorted(rows, key=len)), or to your own function or a method of your fixture's object (sign(timestamp), helper.ids(uuid4)) where the scenario's is a fixture or function substitution — the built-in runs in its place, silently. A save or variable handed to your function out of scope is a missing name (HTTPCHAIN003/004, 016/017): the function gets the built-in function in its value's place
HTTPCHAIN037	warning	A template in a stage (not a parametrize value: see HTTPCHAIN038) that is no single expression the engine evaluates, as its text alone shows: a syntax error ({{ 1 + }}, or a dict literal whose } runs into the template's closing }}, as in {{ {'a': 1}}}), more than one statement ({{ a; b }}), an assignment ({{ user.active = True }} where == was meant, +=, :=) or another statement, a kind of expression the engine does not evaluate (a lambda, a set comprehension, * unpacking outside a list literal, yield, await), an attribute it does not read (doc._id, for which write doc['_id'], or '{}'.format(x)), or a call of anything but a name or an attribute (fns[0]()), wherever in the template it sits, a branch never taken included — the template fails the stage. This is its only finding: the names in it are not checked. A refusal that depends on a value (a module, a function the engine forbids) comes when the stage runs
HTTPCHAIN038	error	The same in a template resolved before any stage runs, so that nothing runs: a scenario-level one (substitutions, auth, ssl, client), which fails scenario initialization and every stage with it (the scenario's collection, for substitutions a templated parametrize value resolves there), or a stage's parametrize value, which fails the scenario's collection
HTTPCHAIN039	error	A directory given to validate holds no scenario file: none named test_<name>.<suffix>.json or .jsonc for the suffix searched by, outside the directories pytest skips — so a mistyped path, or a suffix that names no file, cannot pass the gate as an empty run
HTTPCHAIN040	warning	A parallel stage's stats_as names a variable its own response saves too — the stats are saved after the response steps' saves and replace that one, which no later stage can read. When the stage runs, a name only a user_functions save returns, which validate cannot see, raises the same warning

Deep (opt-in) checks

HTTPCHAIN020–HTTPCHAIN024 come from deep validation (validate --deep), which imports your user modules and touches the filesystem — so it is opt-in and never runs at pytest collection time. Deep findings are always warnings; pair --deep with --strict to fail CI on them.

Filtering collection warnings

At collection time, warning-severity findings are emitted as pytest_httpchain.ScenarioValidationWarning (error-severity findings fail collection outright, and info findings — HTTPCHAIN025 — never affect validity, are exempt from --strict, and are not warned about at collection). Standard warning filters apply — e.g. to silence one code project-wide:

# pytest.ini
[pytest]
filterwarnings =
    ignore:.*HTTPCHAIN005.*:pytest_httpchain.ScenarioValidationWarning