May 13, 2026 · @mavam, @codex · #47
Project fixtures can now declare Python package dependencies inline with PEP 723 metadata:
# /// script
# dependencies = ["boto3"]
# ///
tenzir-test installs these dependencies with uv before importing fixture
modules, so fixtures used during regular test runs and tenzir-test --fixture
can manage their own Python package requirements.
Select tests by requested fixture name with the new --fixture-name option:
tenzir-test --fixture-name node
tenzir-test tests/alerts --match kafka --fixture-name docker-compose
--fixture-name can be repeated and combines with --fixture-tag using OR
semantics before intersecting with positional test paths and --match.
Fixture selectors are long-only; the previous -F alias for --fixture-tag
has been removed before the CLI shape settles.
May 12, 2026 · @mavam, @codex · #44
tenzir-test now requires Python 3.13 or newer.
Users on Python 3.12 need to upgrade their interpreter before installing or running the CLI:
uvx --python 3.13 tenzir-test --help
May 11, 2026 · @mavam, @codex · #42
Select tests by fixture tag with the new --fixture-tag option:
tenzir-test --fixture-tag container
tenzir-test --fixture-tag docker-compose
Fixture tags are cumulative and can be repeated. The selector intersects with positional test paths and --match patterns, so you can narrow a directory to container-backed tests or run only tests that request the built-in Docker Compose fixture.
Fixtures that use the shared container runtime helpers inherit the container tag automatically. Custom fixtures can pass explicit tags at registration time when they use their own abstraction.
May 9, 2026 · @mavam, @codex · #41
The test harness can detect the installed Tenzir version across current and upcoming Tenzir releases. This keeps startup/version checks working during the transition to the next Tenzir release series.
]]>May 5, 2026 · @mavam, @codex · #40
Hook diagnostics emitted with --debug now use the same formatting as the rest of the harness debug trace. Previously, hook invocation messages used ad-hoc debug: lines, which made debug output inconsistent.
Apr 29, 2026 · @mavam, @codex · #39
Suite-level requires.operators checks now apply consistently to every test runner. Mixed suites that combine TQL, shell, Python, or custom tests no longer depend on the first runner type to decide whether required Tenzir operators are available.
Apr 29, 2026 · @mavam, @codex · #38
Suite-scoped fixture setup and teardown failures now appear as regular test failures instead of aborting the entire run with a Python traceback.
This lets the harness continue with independent queued tests after a fixture assertion or cleanup error.
]]>Apr 27, 2026 · @mavam, @codex · #36
Projects can now register Python hooks that run at stable tenzir-test lifecycle points, including before settings discovery:
from tenzir_test import hooks
@hooks.startup
def use_local_build(ctx):
ctx.path.insert(0, str(ctx.root / "build" / "bin"))
ctx.env["TENZIR_BINARY"] = str(ctx.root / "build" / "bin" / "tenzir")
ctx.env["TENZIR_NODE_BINARY"] = str(ctx.root / "build" / "bin" / "tenzir-node")
This makes it possible to select local Tenzir binaries, prepare project-scoped environment variables, and collect diagnostics for failed tests without wrapping the test command in custom shell scripts. Use --no-hooks or TENZIR_TEST_DISABLE_HOOKS=1 to bypass hooks when debugging.
Apr 22, 2026 · @mavam, @codex · #35
The test harness now honors skip: {on: fixture-unavailable} for fixtures that are selected by individual tests:
skip:
on: fixture-unavailable
fixtures:
- optional-service
This lets parameterized per-test fixtures skip only the tests that need the unavailable service. Suite fixtures still require the opt-in in directory-level test.yaml, so one test's frontmatter cannot control the whole suite.
100% passed and 0% failed.
The final aggregate summary now reports non-perfect pass and fail percentages whenever at least one executed test fails. Previously, rounding could show 100% passed and 0% failed for runs with a small number of failures, even though the overall result was not a full success.
For example, a run like 586 passed / 1 failed / 152 skipped now renders the executed-test percentages as 99% passed and 1% failed. This makes mixed outcomes easier to spot at a glance in the CLI output.
Skipping a test no longer modifies its baseline file. Previously, running
with --update would overwrite the baseline of a skipped test with an empty
file, and running without --update would fail if the baseline was non-empty.
Skipped tests now leave existing baselines untouched, so toggling a skip
condition no longer causes unrelated baseline churn.
Parallel suites are now scheduled more reliably when running with --jobs > 1, so interdependent tests (for example publisher/subscriber pairs) start together sooner instead of being delayed behind large backlogs of unrelated tests.
This update also adds an explicit correctness guard for parallel suites: you can set suite.min_jobs in test.yaml to declare the minimum job count required for valid execution. The run now fails fast if --jobs is lower than suite.min_jobs, and it also fails hard when a parallel suite cannot reserve at least min_jobs workers at runtime (for example under slot contention), instead of proceeding under-provisioned.
In addition, the harness now warns when a parallel suite has more tests than available jobs, making it easier to spot cases where effective suite parallelism is capped by worker count.
]]>Feb 25, 2026 · @mavam, @codex · #32
Python fixture tests could fail with serialization errors when the test
configuration included enum values like mode: sequential in test.yaml.
These values are now properly converted to strings before being passed to
test scripts.
Feb 24, 2026 · @mavam, @codex · #29
Test suites can now execute their members in parallel by specifying mode: parallel in the suite configuration. By default, suites continue to run tests sequentially for stability and predictability.
To enable parallel execution, set mode: parallel in the test.yaml file alongside the suite configuration:
suite:
name: my-suite
mode: parallel
fixtures:
- node
Parallel suite execution is useful when tests within a suite are independent and can safely run concurrently. All suite members share the same fixtures and execute within the same fixture lifecycle, while test execution itself happens on separate threads. The suite thread pool is bounded by --jobs, so suite-level concurrency stays within the configured worker budget and does not scale unbounded with suite size.
Suite-level constraints like timeouts, fixture requirements, and capability checks still apply uniformly across all members, whether running sequentially or in parallel.
Feb 25, 2026 · @mavam, @codex · #31
Interrupting test runs with Ctrl+C now exits cleanly without leaking Python tracebacks from subprocess or fixture startup/teardown paths. Interrupt-shaped errors (including nested causes with signal-style return codes such as 130) are treated as graceful cancellation so the CLI reports interrupted tests instead of crashing.
Feb 24, 2026 · @mavam, @codex · #30
Shell test files (.sh) now always default to the "shell" runner, even when a directory-level test.yaml file specifies a different runner (for example, runner: tenzir). This makes shell scripts work reliably in mixed-runner directories without requiring explicit runner: frontmatter in each file. Explicit runner: declarations in test file frontmatter still take precedence and can override this behavior if needed.
Manual fixture control now behaves consistently for fixtures that declare dataclass options.
Previously, starting a fixture with acquire_fixture(...) could fail when no
explicit options were provided, even if the fixture defined defaults.
With this fix, current_options(...) returns default typed options in the
manual fixture-control path, so fixtures started manually and fixtures started
through normal test activation behave the same way.
Feb 22, 2026 · @mavam, @codex · #28
Test suites can now declare operator dependencies with a requires configuration key in test.yaml. When a required operator is unavailable in the target Tenzir build, you can gracefully skip the suite by pairing requires with skip: {on: capability-unavailable}.
The skip.on key now accepts either a single condition as a string or a list of conditions, letting you skip on either fixture-unavailable or capability-unavailable (or both). When a capability is unavailable and the suite doesn't opt into skipping, the test run fails with a clear error message listing the missing operators.
Example configuration in test.yaml:
requires:
operators: [from_gcs, to_s3]
skip:
on: capability-unavailable
reason: requires from_gcs and to_s3 operators
This ensures test suites targeting specific capabilities only run when those capabilities are present, improving test reliability in heterogeneous build environments.
]]>Feb 18, 2026 · @mavam, @codex · #27
Fixtures can now define assertion hooks that run after test execution completes while fixtures remain active. Define assertions using the assertions parameter with a frozen dataclass type, then pass fixture-specific assertion payloads under assertions.fixtures.<name> in test frontmatter. The framework automatically invokes the assert_test hook with assertion data before tearing down fixtures, letting you validate side effects like HTTP requests or log output.
Example usage with an HTTP fixture:
fixtures: [http]
assertions:
fixtures:
http:
count: 1
method: POST
path: /api/endpoint
body: '{"key":"value"}'
The HTTP fixture receives the typed assertion payload and validates that the expected request was received. Payload structure is fixture-defined, so fixtures can choose flat fields or nested schemas as needed.
Assertion checks are tracked separately from test counts in the run summary as pass/fail check metrics, so fixture-level validations are visible without changing total test counts.
]]>Feb 16, 2026 · @mavam, @codex · #26
The test harness no longer initializes fixtures for suites where all tests are statically skipped. Previously, fixtures were activated even when no tests would run, causing unnecessary startup overhead and potential errors.
Feb 16, 2026 · @mavam, @codex · #25
When a fixture becomes unavailable during test execution, the test harness now provides a clean error message instead of a Python traceback.
]]>Feb 15, 2026 · @mavam, @codex · #23, #24
tenzir-test now supports both coarse and fine-grained controls for running skipped tests.
Use --run-skipped as a sledgehammer, or --run-skipped-reason with the same matching semantics as --match (bare substring or glob):
tenzir-test --run-skipped
tenzir-test --run-skipped-reason 'maintenance'
tenzir-test --run-skipped-reason '*docker*'
If both options are provided, --run-skipped takes precedence.
Feb 15, 2026 · @mavam, @codex · #22
Writing container-based fixtures no longer requires duplicating boilerplate for runtime detection, process management, and readiness polling.
The new container_runtime module provides reusable building blocks that handle
the common plumbing: detecting whether Docker or Podman is available, launching
containers in detached mode, polling for service readiness, and tearing down
cleanly. Custom fixtures can import these helpers and focus on their
service-specific logic instead.
The built-in docker-compose fixture and the example project now use these
shared helpers internally.
Adds a built-in docker-compose fixture that starts and tears down Docker Compose services from structured fixture options.
The fixture validates Docker Compose availability, waits for service readiness (health-check first, running-state fallback), and exports deterministic service environment variables (including host-published ports).
Feb 14, 2026 · @mavam, @claude · #20
You can now start fixtures in foreground mode without running any tests by using the --fixture option. This lets you provision services (like a database or message broker) and use them in your workflow.
Specify fixture names or YAML-style configuration:
uvx tenzir-test --fixture mysql
uvx tenzir-test --fixture 'kafka: {port: 9092}' --debug
The harness prints fixture-provided environment variables like HOST, PORT, and DATABASE, then keeps services running until you press Ctrl+C. You can repeat --fixture to activate multiple fixtures. The option is mutually exclusive with positional TEST arguments.