File

Tenzir can read from and write to local files, cloud object storage, standard input, standard output, and standard error.

When ~ is the first character in the file path, the operator substitutes it with the $HOME environment variable.

Use from_file to read files with glob patterns, automatic format detection, and file monitoring. For writing, use to_file with a print operator.

Examples

Read a file

Read from a file and parse it in the format implied by the file extension:

from_file "/tmp/file.json"

The operator automatically decompresses the file when the suffix list contains a supported compression algorithm:

from_file "/tmp/file.json.gz"

Read multiple files with a glob pattern

from_file "/var/log/*.json"

Watch a directory for new files

Set watch to a duration to continuously monitor for new files:

from_file "/var/log/app/*.json", watch=10s

Write a file

Write to a file in a specific format:

version
to_file "/tmp/tenzir-version.json" {
  write_json
}

With compression:

version
to_file "/tmp/tenzir-version.json.bz2" {
  write_json
  compress_bz2
}

Append to a file

In case the file exists and you do not want to overwrite it, pass append=true as option:

from {x: 42}
to_file "/tmp/event.csv", append=true {
  write_csv
}

Read or write a Unix domain socket

Use the dedicated Unix domain socket operators for local socket streams. When Tenzir connects to an existing socket, use from_unix_socket or to_unix_socket:

from_unix_socket "/tmp/socket" {
  read_ndjson
}

to_unix_socket "/tmp/socket" {
  write_ndjson
}

When Tenzir owns the listening socket path, use accept_unix_socket:

accept_unix_socket "/tmp/socket" {
  read_ndjson
}