# Tenzir Node v2.4.0

## 🚀 Features

### Make data predicate evaluation column-major

Dec 6, 2022 · [@dominiklohmann](https://github.com/dominiklohmann) · [#2730](https://github.com/tenzir/tenzir/pull/2730)

Queries without acceleration from a dense index run significantly faster, e.g., initial tests show a 2x performance improvement for substring queries.

### Add “-total” metric keys for schema-dependent metrics

Nov 4, 2022 · [@Dakostu](https://github.com/Dakostu) · [#2682](https://github.com/tenzir/tenzir/pull/2682)

VAST has three new metrics: `catalog.num-partitions-total`, `catalog.num-events-total`, and `ingest-total` that sum up all schema-based metrics by their respective schema-based metric counterparts.

### Enable configuration of the zstd compression level for feather store

Nov 1, 2022 · [@dispanser](https://github.com/dispanser) · [#2623](https://github.com/tenzir/tenzir/pull/2623)

VAST has a new configuration setting, `vast.zstd-compression-level`, to control the compression level of the Zstd algorithm used in both the Feather and Parquet store backends. The default level is set by the Apache Arrow library, and for Parquet is no longer explicitly defaulted to `9`.

### Cloud MISP

Oct 10, 2022 · [@rdettai](https://github.com/rdettai) · [#2548](https://github.com/tenzir/tenzir/pull/2548)

VAST Cloud has now a MISP plugin that enables to add a MISP instance to the cloud stack.

### Rebatch undersized batches when rebuilding partitions

Sep 23, 2022 · [@dominiklohmann](https://github.com/dominiklohmann) · [#2583](https://github.com/tenzir/tenzir/pull/2583)

Rebuilding partitions now additionally rebatches the contained events to `vast.import.batch-size` events per batch, which accelerates queries against partitions that previously had undersized batches.

### PRs 2567-2614-2638-3681

Sep 20, 2022 · [@lava](https://github.com/lava) · [#2567](https://github.com/tenzir/tenzir/pull/2567)

The new experimental web plugin offers a RESTful API to VAST and a bundled web user interface in Svelte.

### Disable building unit tests in Dockerfile

Sep 14, 2022 · [@dominiklohmann](https://github.com/dominiklohmann) · [#2578](https://github.com/tenzir/tenzir/pull/2578)

VAST Cloud can now expose HTTP services using Cloudflare Access.

### Emit metrics from the filesystem actor

Sep 14, 2022 · [@dominiklohmann](https://github.com/dominiklohmann) · [#2572](https://github.com/tenzir/tenzir/pull/2572)

VAST now emits metrics for filesystem access under the keys `posix-filesystem.{checks,writes,reads,mmaps,erases,moves}.{successful,failed,bytes}`.

### PRs 2574-2652

Sep 12, 2022 · [@KaanSK](https://github.com/KaanSK) · [#2574](https://github.com/tenzir/tenzir/pull/2574)

VAST now ships a Docker Compose file. In particular, the Docker Compose stack now has a TheHive integration that can run VAST queries as a Cortex Analyzer.

### PRs 2513-2738

Sep 2, 2022 · [@tobim](https://github.com/tobim) · [#2513](https://github.com/tenzir/tenzir/pull/2513)

We now distribute VAST also as Debian Package with every new release. The Debian package automatically installs a systemd service and creates a `vast` user for the VAST process.

## 🔧 Changes

### Load “all” plugins by default & allow “empty” values

Nov 18, 2022 · [@Dakostu](https://github.com/Dakostu) · [#2689](https://github.com/tenzir/tenzir/pull/2689)

VAST now loads all plugins by default. To revert to the old behavior, explicitly set the `vast.plugins` option to have no value.

### Rename `vast dump` to `vast show`

Nov 10, 2022 · [@dominiklohmann](https://github.com/dominiklohmann) · [#2686](https://github.com/tenzir/tenzir/pull/2686)

The `vast dump` command is now called `vast show`.

### Arrow 10.0.0 support

Nov 9, 2022 · [@Dakostu](https://github.com/Dakostu) · [#2685](https://github.com/tenzir/tenzir/pull/2685)

Building VAST from source now requires [Apache Arrow 10.0](https://arrow.apache.org/blog/2022/10/31/10.0.0-release/) or newer.

### Remove PyVAST in favor of new Python bindings

Oct 29, 2022 · [@dominiklohmann](https://github.com/dominiklohmann) · [#2674](https://github.com/tenzir/tenzir/pull/2674)

We removed PyVAST from the code base in favor of the new Python bindings. PyVAST continues to work as a thin wrapper around the VAST binary, but will no longer be released alongside VAST.

### Make `feather` the default store-backend

Sep 23, 2022 · [@dominiklohmann](https://github.com/dominiklohmann) · [#2587](https://github.com/tenzir/tenzir/pull/2587)

The default store backend of VAST is now `feather`. Reading from VAST’s custom `segment-store` backend is still transparently supported, but new partitions automatically write to the Apache Feather V2 backend instead.

### Move the version string into a central JSON file

Sep 19, 2022 · [@tobim](https://github.com/tobim) · [#2582](https://github.com/tenzir/tenzir/pull/2582)

Building VAST from source now requires CMake 3.19 or greater.

### Add memory-usage to index and catalog telemetry reports

Sep 5, 2022 · [@patszt](https://github.com/patszt) · [#2471](https://github.com/tenzir/tenzir/pull/2471)

VAST now emits per-component memory usage metrics under the keys `index.memory-usage` and `catalog.memory-usage`.

### Change default endpoint to 127.0.0.1

Sep 1, 2022 · [@lava](https://github.com/lava) · [#2512](https://github.com/tenzir/tenzir/pull/2512)

We changed the default VAST endpoint from `localhost` to `127.0.0.1`. This ensures the listening address is deterministic and not dependent on the host-specific IPv4 and IPv6 resolution. For example, resolving `localhost` yields a list of addresses, and if VAST fails to bind on the first (e.g., to due to a lingering socket) it would silently go to the next. Taking name resolution out of the equation fixes such issues. Set the option `vast.endpoint` to override the default endpoint.

## 🐞 Bug Fixes

### Allow read access to user home dir in the systemd unit

Nov 25, 2022 · [@tobim](https://github.com/tobim) · [#2734](https://github.com/tenzir/tenzir/pull/2734)

The systemd service no longer fails if the home directory of the vast user is not in `/var/lib/vast`.

### Add a timeout to the UDS metric sink

Oct 21, 2022 · [@tobim](https://github.com/tobim) · [#2635](https://github.com/tenzir/tenzir/pull/2635)

The UDS metrics sink no longer deadlocks due to suspended listeners.

### Clear failed partitions from the cache

Oct 20, 2022 · [@lava](https://github.com/lava) · [#2642](https://github.com/tenzir/tenzir/pull/2642)

VAST now ejects partitions from the LRU cache if they fail to load with an I/O error.

### Fix a connection error message

Oct 7, 2022 · [@tobim](https://github.com/tobim) · [#2609](https://github.com/tenzir/tenzir/pull/2609)

The error message on connection failure now contains a correctly formatted target endpoint.

### Don’t abort startup if individual partitions fail to load

Sep 23, 2022 · [@tobim](https://github.com/tobim) · [#2515](https://github.com/tenzir/tenzir/pull/2515)

VAST now skips unreadable partitions while starting up, instead of aborting the initialization routine.

### Remove the shutdown grace period

Sep 8, 2022 · [@dominiklohmann](https://github.com/dominiklohmann) · [#2568](https://github.com/tenzir/tenzir/pull/2568)

VAST no longer attempts to hard-kill itself if the shutdown did not finish within the configured grace period. The option `vast.shutdown-grace-period` no longer exists. We recommend setting `TimeoutStopSec=180` in the VAST systemd service definition to restore the previous behavior.

### Remove caf::skip usages

Sep 2, 2022 · [@patszt](https://github.com/patszt) · [#2530](https://github.com/tenzir/tenzir/pull/2530)

Rebuilding of heterogeneous partition no longer freezes the entire rebuilder on pipeline failures.

[ Download on GitHub ](https://github.com/tenzir/tenzir/releases/tag/v2.4.0)

[Get the release artifacts and source code.](https://github.com/tenzir/tenzir/releases/tag/v2.4.0)