Cloud Storage is Google’s object storage service. Tenzir can treat it like a local filesystem to read and write files.
Configuration
Section titled “Configuration”You need to configure appropriate credentials using Google’s Application Default Credentials.
Examples
Section titled “Examples”Use from_gcs to read files from Cloud
Storage. It supports glob patterns and automatic format detection. For writing,
use save_gcs with a print operator.
Read events from a file in a bucket
Section titled “Read events from a file in a bucket”from_gcs "gs://bucket/path/to/file.json"Read all JSON files from a bucket
Section titled “Read all JSON files from a bucket”from_gcs "gs://bucket/logs/**.json"Write an event to a file in a bucket
Section titled “Write an event to a file in a bucket”from {foo: 42}print_jsonsave_gcs "gs://bucket/path/to/file.json"