Skipping a test no longer modifies its baseline file. Previously, running
with --update would overwrite the baseline of a skipped test with an empty
file, and running without --update would fail if the baseline was non-empty.
Skipped tests now leave existing baselines untouched, so toggling a skip
condition no longer causes unrelated baseline churn.
Parallel suites are now scheduled more reliably when running with --jobs > 1, so interdependent tests (for example publisher/subscriber pairs) start together sooner instead of being delayed behind large backlogs of unrelated tests.
This update also adds an explicit correctness guard for parallel suites: you can set suite.min_jobs in test.yaml to declare the minimum job count required for valid execution. The run now fails fast if --jobs is lower than suite.min_jobs, and it also fails hard when a parallel suite cannot reserve at least min_jobs workers at runtime (for example under slot contention), instead of proceeding under-provisioned.
In addition, the harness now warns when a parallel suite has more tests than available jobs, making it easier to spot cases where effective suite parallelism is capped by worker count.
]]>Feb 25, 2026 · @mavam, @codex · #32
Python fixture tests could fail with serialization errors when the test
configuration included enum values like mode: sequential in test.yaml.
These values are now properly converted to strings before being passed to
test scripts.
Feb 24, 2026 · @mavam, @codex · #29
Test suites can now execute their members in parallel by specifying mode: parallel in the suite configuration. By default, suites continue to run tests sequentially for stability and predictability.
To enable parallel execution, set mode: parallel in the test.yaml file alongside the suite configuration:
suite:
name: my-suite
mode: parallel
fixtures:
- node
Parallel suite execution is useful when tests within a suite are independent and can safely run concurrently. All suite members share the same fixtures and execute within the same fixture lifecycle, while test execution itself happens on separate threads. The suite thread pool is bounded by --jobs, so suite-level concurrency stays within the configured worker budget and does not scale unbounded with suite size.
Suite-level constraints like timeouts, fixture requirements, and capability checks still apply uniformly across all members, whether running sequentially or in parallel.
Feb 25, 2026 · @mavam, @codex · #31
Interrupting test runs with Ctrl+C now exits cleanly without leaking Python tracebacks from subprocess or fixture startup/teardown paths. Interrupt-shaped errors (including nested causes with signal-style return codes such as 130) are treated as graceful cancellation so the CLI reports interrupted tests instead of crashing.
Feb 24, 2026 · @mavam, @codex · #30
Shell test files (.sh) now always default to the "shell" runner, even when a directory-level test.yaml file specifies a different runner (for example, runner: tenzir). This makes shell scripts work reliably in mixed-runner directories without requiring explicit runner: frontmatter in each file. Explicit runner: declarations in test file frontmatter still take precedence and can override this behavior if needed.
Manual fixture control now behaves consistently for fixtures that declare dataclass options.
Previously, starting a fixture with acquire_fixture(...) could fail when no
explicit options were provided, even if the fixture defined defaults.
With this fix, current_options(...) returns default typed options in the
manual fixture-control path, so fixtures started manually and fixtures started
through normal test activation behave the same way.
Feb 22, 2026 · @mavam, @codex · #28
Test suites can now declare operator dependencies with a requires configuration key in test.yaml. When a required operator is unavailable in the target Tenzir build, you can gracefully skip the suite by pairing requires with skip: {on: capability-unavailable}.
The skip.on key now accepts either a single condition as a string or a list of conditions, letting you skip on either fixture-unavailable or capability-unavailable (or both). When a capability is unavailable and the suite doesn't opt into skipping, the test run fails with a clear error message listing the missing operators.
Example configuration in test.yaml:
requires:
operators: [from_gcs, to_s3]
skip:
on: capability-unavailable
reason: requires from_gcs and to_s3 operators
This ensures test suites targeting specific capabilities only run when those capabilities are present, improving test reliability in heterogeneous build environments.
]]>Feb 18, 2026 · @mavam, @codex · #27
Fixtures can now define assertion hooks that run after test execution completes while fixtures remain active. Define assertions using the assertions parameter with a frozen dataclass type, then pass fixture-specific assertion payloads under assertions.fixtures.<name> in test frontmatter. The framework automatically invokes the assert_test hook with assertion data before tearing down fixtures, letting you validate side effects like HTTP requests or log output.
Example usage with an HTTP fixture:
fixtures: [http]
assertions:
fixtures:
http:
count: 1
method: POST
path: /api/endpoint
body: '{"key":"value"}'
The HTTP fixture receives the typed assertion payload and validates that the expected request was received. Payload structure is fixture-defined, so fixtures can choose flat fields or nested schemas as needed.
Assertion checks are tracked separately from test counts in the run summary as pass/fail check metrics, so fixture-level validations are visible without changing total test counts.
]]>Feb 16, 2026 · @mavam, @codex · #26
The test harness no longer initializes fixtures for suites where all tests are statically skipped. Previously, fixtures were activated even when no tests would run, causing unnecessary startup overhead and potential errors.
Feb 16, 2026 · @mavam, @codex · #25
When a fixture becomes unavailable during test execution, the test harness now provides a clean error message instead of a Python traceback.
]]>Feb 15, 2026 · @mavam, @codex · #23, #24
tenzir-test now supports both coarse and fine-grained controls for running skipped tests.
Use --run-skipped as a sledgehammer, or --run-skipped-reason with the same matching semantics as --match (bare substring or glob):
tenzir-test --run-skipped
tenzir-test --run-skipped-reason 'maintenance'
tenzir-test --run-skipped-reason '*docker*'
If both options are provided, --run-skipped takes precedence.
Feb 15, 2026 · @mavam, @codex · #22
Writing container-based fixtures no longer requires duplicating boilerplate for runtime detection, process management, and readiness polling.
The new container_runtime module provides reusable building blocks that handle
the common plumbing: detecting whether Docker or Podman is available, launching
containers in detached mode, polling for service readiness, and tearing down
cleanly. Custom fixtures can import these helpers and focus on their
service-specific logic instead.
The built-in docker-compose fixture and the example project now use these
shared helpers internally.
Adds a built-in docker-compose fixture that starts and tears down Docker Compose services from structured fixture options.
The fixture validates Docker Compose availability, waits for service readiness (health-check first, running-state fallback), and exports deterministic service environment variables (including host-published ports).
Feb 14, 2026 · @mavam, @claude · #20
You can now start fixtures in foreground mode without running any tests by using the --fixture option. This lets you provision services (like a database or message broker) and use them in your workflow.
Specify fixture names or YAML-style configuration:
uvx tenzir-test --fixture mysql
uvx tenzir-test --fixture 'kafka: {port: 9092}' --debug
The harness prints fixture-provided environment variables like HOST, PORT, and DATABASE, then keeps services running until you press Ctrl+C. You can repeat --fixture to activate multiple fixtures. The option is mutually exclusive with positional TEST arguments.
Feb 13, 2026 · @mavam, @claude · #19
Configuration override log messages now use relative paths and human-friendly formatting instead of absolute paths and raw Python repr output, making debug output easier to read.
]]>Feb 11, 2026 · @mavam, @claude · #18
Fixture options now support nested dataclass hierarchies, allowing you to structure complex configurations with multiple levels of organization.
Previously, fixture options were limited to flat fields. Now you can declare nested dataclasses as field types, and tests can pass deeply structured options through frontmatter:
fixtures:
- node:
server:
message:
greeting: world
Type safety is preserved across all nesting levels. Optional nested fields (declared with Optional[T] or T | None) are supported, and omitted nested records use their default values. The example server fixture demonstrates this capability with a message.greeting field that flows through to environment variables.
Feb 10, 2026 · @mavam, @claude · #17
Fixtures can now receive typed configuration options passed from test YAML files.
Declare options on a fixture with @fixture(options=MyDataclass), where MyDataclass is a frozen dataclass. Tests can then provide structured options in their frontmatter using either compact syntax (for simple cases) or expanded syntax (for clarity):
fixtures:
- node:
tls: true
port: 8443
- http:
port: 9090
The @fixture decorator validates that option keys match the dataclass fields, and the fixture retrieves its typed instance using current_options(name). When options aren't provided, fixtures receive a default-constructed instance of their options class.
The feature maintains backward compatibility with the existing fixtures: [node, http] syntax for fixtures without options.
skip
configurations in test.yaml failed with a JSON serialization error.
Python fixture tests that use skip in their test.yaml no longer fail with
Object of type SkipConfig is not JSON serializable.
-m/--match flag with automatic substring matching and centralizes skip handling in the engine for more consistent test reporting.
Feb 6, 2026 · @mavam, @claude · #16
The -m/--match flag now treats bare strings as substring matches automatically.
Previously, you had to use glob syntax like tenzir-test -m '*mysql*' to match tests by name. Now you can write tenzir-test -m mysql and it automatically matches any test path containing "mysql". This makes the common case of substring matching simpler and more intuitive.
Patterns that contain glob metacharacters (*, ?, [) still use fnmatch syntax as before, so existing patterns with wildcards continue to work exactly as they did. This change is fully backwards-compatible.
Feb 6, 2026 · @mavam, @claude · #15
Skip handling now lives in the engine instead of individual runners. Skipped tests are recorded consistently in summary stats regardless of which runner executes them, so skip counts and paths in the test report are always accurate.
Previously, each runner duplicated the same skip-config parsing, which could
lead to inconsistent skip tracking. Custom runners no longer need to implement
skip logic because the engine evaluates the skip configuration before
dispatching to any runner.
-m/--match CLI option.
Feb 6, 2026 · @mavam, @claude · #13
Select tests by relative path using the new -m/--match option with fnmatch glob patterns. You can repeat the option to match multiple patterns, and tests matching any pattern are selected. When you combine TEST paths with -m patterns, the framework runs only tests matching both (intersection). If a matched test belongs to a suite configured via test.yaml, all tests in that suite are included automatically. Empty or whitespace-only patterns are silently ignored.
Example: tenzir-test -m '*context*' -m '*create*' runs all tests whose paths contain "context" or "create" anywhere in the name.
Feb 2, 2026 · @mavam, @claude · #12
The help text for the tenzir-test command now provides more context and guidance.
The command description now explains the baseline comparison behavior and shows how to regenerate baselines with the --update flag.
Jan 31, 2026 · @mavam, @claude
The test harness now supports .stdin files for piping data directly to test processes.
Place a .stdin file next to any test to have its contents automatically piped to the subprocess stdin. The harness also sets the TENZIR_STDIN environment variable with the file path. This is particularly useful for TQL tests where you can start a pipeline with a parser directly:
read_csv
where count > 10
Instead of using .input files with from_file env("TENZIR_INPUT"). Both approaches remain valid—choose whichever fits your test better.
Shell and Python tests can also read from stdin or access TENZIR_STDIN when needed. Custom runners can use get_stdin_content(env) to read the file contents and pass them via the stdin_data parameter to run_subprocess().
Jan 27, 2026 · @mavam, @claude
When listing projects in the execution plan, satellite projects now display their path relative to the root project instead of just their directory name. This makes satellite projects with identical directory names distinguishable. Additionally, project markers have been refined: root projects use a filled marker (● for packages, ■ for regular projects), while satellite projects use an empty marker (○ for packages, □ for regular projects).
]]>--debug flag automatically enable verbose output, so you see all test results when diagnosing failures.
Jan 26, 2026 · @mavam, @claude · #9
The --debug flag (or TENZIR_TEST_DEBUG=1 environment variable) now automatically enables verbose output. When debugging test failures, you now see all test results (pass/skip/fail) instead of only failures, making it easier to diagnose issues. This provides more context without requiring users to pass both --debug and --verbose flags separately.
Jan 23, 2026 · @mavam, @claude · #8
Fixed an error that occurred when running diff tests with multi-word Tenzir commands like uvx tenzir.