# Windows Event Logs

Windows Event Logs record system, security, and application events on Windows. You can collect them into Tenzir for monitoring, troubleshooting, and analysis.

Once Windows Event Logs are flowing in a Tenzir pipeline, you can use any operator to process them. The below examples simply [import all data into a node](https://preview.docs.tenzir.com/375/375/guides/edge-storage/import-into-a-node.md).

## Collect logs with an agent

Installing a third-party agent to ship logs away from a Windows machine is common way to send events to a remote location.

Regardless of the concrete agent you are using for shipping, the high-level setup is always the same: the agent sends events in a push-based to a Tenzir pipeline.

### Winlogbeat

[Winlogbeat](https://www.elastic.co/beats/winlogbeat) is Elastic’s log shipper to get Windows Event Logs out of Windows machines into the Elastic stack.

#### Configure Winlogbeat

After [installing Winlogbeat](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-installation-configuration.html), create a configuration:

winlogbeat.yml

```yaml
# Choose your channels.
winlogbeat.event_logs:
  - name: Application
  - name: System
  - name: Security
  - name: ForwardedEvents
  - name: Windows PowerShell
  - name: Microsoft-Windows-Sysmon/Operational
  - name: Microsoft-Windows-PowerShell/Operational
  - name: Microsoft-Windows-Windows Defender/Operational
  - name: Microsoft-Windows-TaskScheduler/Operational
  - name: Microsoft-Windows-TerminalServices-LocalSessionManager/Operational
  - name: Microsoft-Windows-TerminalServices-RDPClient/Operational


# Send data to a Tenzir pipeline with an Elasticsearch-compatible endpoint.
output.elasticsearch:
  hosts: ["https://10.0.0.1:9200"]
  username: "$USER"
  password: "$PASSWORD"
  ssl:
    enabled: true
    certificate_authorities: [C:\Program Files\Winlogbeat\ca.crt]
    # PEM format
    certificate: C:\Program Files\Winlogbeat\tenzir.crt
    key: C:\Program Files\Winlogbeat\beat-win10\tenzir.key
```

#### Start Winlogbeat as a service

After completing your configuration, start the Winlogbeat service:

```plaintext
C:\Program Files\Winlogbeat> Start-Service winlogbeat
```

#### Run a Tenzir pipeline

Now consume the data via a Tenzir pipeline using the [Elasticsearch](https://preview.docs.tenzir.com/375/375/integrations/elasticsearch.md)/[OpenSearch](https://preview.docs.tenzir.com/375/375/integrations/opensearch.md) integration that mimics a Bulk API endpoint:

```tql
accept_opensearch "10.0.0.1:9200", tls={
  certfile: "server.crt",
  keyfile: "private.key",
}
import
```

### Fluent Bit

Since Tenzir has native Fluent Bit support, collecting logs via the Fluent Bit agent is a simple approach.

#### Configure Fluent Bit

First, install Fluent Bit on Windows according to the [official instructions](https://docs.fluentbit.io/manual/installation/windows).

Then create a [YAML configuration](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/yaml/configuration-file) to send Windows Event Logs out via the [Forward output](https://docs.fluentbit.io/manual/pipeline/outputs/forward), which encodes the events using Fluent Bit’s MsgPack-based wire format:

```yaml
input:
  - name: winevtlog
    channels: Setup,Windows PowerShell
    interval_sec: 1
    db: winevtlog.sqlite
output:
  - name: forward
    match: "*"
    host: 10.0.0.1
```

Adapt `input.channels` according to the Event Log channels you would like Fluent Bit to monitor.

#### Run a Tenzir pipeline

Use the [`from_fluent_bit`](https://preview.docs.tenzir.com/375/375/reference/operators/from_fluent_bit.md) source operator with the [Forward input](https://docs.fluentbit.io/manual/pipeline/inputs/forward):

```tql
from_fluent_bit "forward", options={
  listen: 10.0.0.1,
}
import
```

Ensure that the `listen` parameter matches the `host` value in your Fluent Bit configuration.

#### Test the configuration

Test the setup by running the Fluent Bit command line utility:

```plaintext
C:\Program Files\fluent-bit\bin\fluent-bit.exe -c \fluent-bit\conf\fluent-bit.yaml
```

#### Deploy Fluent Bit as a service

To make the setup permanent, [run Fluent Bit as a service](https://docs.fluentbit.io/manual/installation/windows#windows-service-support).

Create the service:

```plaintext
sc.exe create fluent-bit binpath= "\fluent-bit\bin\fluent-bit.exe -c \fluent-bit\conf\fluent-bit.yaml"
```

Start and check the service:

```plaintext
sc.exe start fluent-bit
sc.exe query fluent-bit
```

Start the service at boot:

```plaintext
sc.exe config fluent-bit start= auto
```

### NXLog

The NXLog agent collects Windows Event Logs and offers numerous [output modules](https://docs.nxlog.co/refman/current/om/index.html). You have several options with Tenzir. We use the JSON extension to format the data in all examples.

#### Ship logs via TCP

To send logs straight to a TCP socket, use the [TCP output module](https://docs.nxlog.co/refman/current/om/tcp.html) with the following configuration:

```plaintext
<Extension json>
  Module    xm_json
</Extension>


<Output tcp>
  Module    om_tcp
  Host      10.0.0.1:1514
  Exec      to_json();
</Output>
```

Import the logs via TCP:

```tql
accept_tcp "10.0.0.1:1514" {
  read_json
}
import
```

#### Ship logs via TLS/SSL

For an encrypted connection, use the [SSL output module](https://docs.nxlog.co/refman/current/om/ssl.html) with the following configuration:

```plaintext
<Extension json>
  Module          xm_json
</Extension>


<Output ssl>
  Module          om_ssl
  Host            example.com:23456
  CAFile          %CERTDIR%/ca.pem
  CertFile        %CERTDIR%/client-cert.pem
  CertKeyFile     %CERTDIR%/client-key.pem
  KeyPass         secret
  AllowUntrusted  TRUE
  OutputType      Binary
  Exec            to_json();
</Output>
```

Import the logs via TCP:

```tql
accept_tcp "127.0.0.1:4000",
  tls={certfile: "key_and_cert.pem", keyfile: "key_and_cert.pem"} {
    read_json
  }
import
```

#### Ship logs via Kafka

The [Kafka output module](https://docs.nxlog.co/refman/current/om/kafka.html) publishes to a Kafka topic that Tenzir can read from. Use the following output configuration to publish to the `nxlog` topic:

```plaintext
<Output out>
  Module          om_kafka
  BrokerList      localhost:9092
  Topic           nxlog
  LogqueueSize    100000
  Partition       0
  Protocol        ssl
  CAFile          %CERTDIR%/ca.pem
  CertFile        %CERTDIR%/client-cert.pem
  CertKeyFile     %CERTDIR%/client-key.pem
  KeyPass         thisisasecret
</Output>
```

Then use [Kafka](https://preview.docs.tenzir.com/375/375/integrations/kafka.md) to read from the topic:

```tql
from_kafka "nxlog"
this = message.parse_json()
import
```

## Collect logs via WEF & WEC

Instead of deploying an agent on a Windows endpoint, you can also use native facilities to collect logs centrally.

To this end, Windows comes with a **Windows Event Forwarding (WEF)** mechanism on the endpoints that uses standard *Windows Remote Management (WinRM)* protocols to transmit events. The **Windows Event Collector (WEC)** is the service running on a server that receives the events sent by clients through WEF. The WEC aggregates these logs and makes them available for review and analysis. Administrators can create subscriptions on the WEC that define which events to collect from which clients, using criteria such as event IDs, keywords, or log levels.

The diagram below illustrates a typical setup:

On the WEC, you typically ship the collected logs away using a third-party agent, as described above. Read below on using [OpenWEC](#collect-logs-via-openwec) as an agent-free alternative.

The following configuration steps are heavily inspired by [SEKOIA](https://docs.sekoia.io/xdr/features/collect/integrations/endpoint/windows/#windows-event-forwarder-to-windows-event-collector-to-a-concentrator)’s instructions.

### Setup the Window Event Collector (WEC)

We begin with setting up the WEC prior to connecting the client to it.

#### Setup Windows Remote Management (WinRM)

Configure WinRM as follows:

```cmd
winrm qc -q
```

The argument `qc` stands for “quick configuration” to perform a basic configuration of WinRM with default settings. This configuration includes starting the WinRM service, setting it to start automatically with the system, creating a listener on HTTP to accept WS-Management protocol requests, and configuring the Windows Firewall to allow WinRM traffic.

The `-q` flag means “quiet mode” to avoid prompting you for any input or confirmation, i.e., it makes the process non-interactive.

:::caution Unencrypted HTTP This command sets up WinRM to listen on HTTP, which is not encrypted. For a secure production environment, it’s advisable to configure WinRM to use HTTPS, which requires additional steps, including setting up an appropriate server certificate for encryption. :::

#### Enable the Event Collector service

Use the `wecutil` command to perform the necessary steps to set up the Windows Event Collector service:

```cmd
wecutil qc /q
```

As above, `qc` stands for “quick configuration” and `/q` for “quiet”. The setup includes actions such as configuring the service to start automatically and ensuring that the service is in a state ready to create and manage event subscriptions.

#### Create a subscription file

Create a new file with the following contents:

DC\_SUBSCRIPTION.xml

```xml
<?xml version="1.0" encoding="UTF-8"?>
<Subscription xmlns="http://schemas.microsoft.com/2006/03/windows/events/subscription">
    <!-- Name of subscription -->
    <SubscriptionId>DC_SUBSCRIPTION</SubscriptionId>
    <!-- Push mode (DC to WEC) -->
    <SubscriptionType>SourceInitiated</SubscriptionType>
    <Description>Source Initiated Subscription from DC_SUBSCRIPTION</Description>
    <!-- Subscription is active -->
    <Enabled>true</Enabled>
    <Uri>http://schemas.microsoft.com/wbem/wsman/1/windows/EventLog</Uri>
    <!-- This mode ensures that events are delivered with minimal delay -->
    <!-- It is an appropriate choice if you are collecting alerts or critical events -->
    <!-- It uses push delivery mode and sets a batch timeout of 30 seconds -->
    <ConfigurationMode>MinLatency</ConfigurationMode>
    <!-- Event log to retrieved -->
    <Query>
        <![CDATA[
            <QueryList>
                <Query Id="0">
                    <Select Path="Application">*</Select>
                    <Select Path="Security">*</Select>
                    <Select Path="System">*</Select>
                </Query>
            </QueryList>
        ]]>
    </Query>
    <!-- Collect events generated since the subscription (not oldest) -->
    <ReadExistingEvents>false</ReadExistingEvents>
    <!-- Protocol and port used (DC to WEC) -->
    <TransportName>http</TransportName>
    <!-- Mandatory value (https://www-01.ibm.com/support/docview.wss?crawler=1&uid=swg1IV71375) -->
    <ContentFormat>RenderedText</ContentFormat>
    <Locale Language="en-US"/>
    <!-- Target Event log on WEC -->
    <LogFile>ForwardedEvents</LogFile>
    <!-- Define which domain computers are allowed or not to initiate subscriptions -->
    <!-- This exemple grants members of the Domain Computers domain group, as well as the local Network Service group (for local forwarder) -->
    <AllowedSourceDomainComputers>O:NSG:NSD:(A;;GA;;;DC)(A;;GA;;;NS)</AllowedSourceDomainComputers>
</Subscription>
```

Key elements are:

* `SubscriptionId`: Give a unique name to your subscription.
* `SubscriptionType`: Choose between `SourceInitiated` (push) or `CollectorInitiated` (pull).
* `Description`: Provide a meaningful description.
* `Query`: Modify the event log query to specify which events to collect.
* `LogFile`: Define the destination log file on the collector.
* `AllowedSourceDomainComputers`: Adjust the [SDDL](https://learn.microsoft.com/en-gb/windows/win32/secauthz/security-descriptor-definition-language?redirectedfrom=MSDN) string to specify which computers can forward events.

There are [several](https://github.com/palantir/windows-event-forwarding/tree/master/wef-subscriptions) [GitHub](https://github.com/nsacyber/Event-Forwarding-Guidance/tree/master/Subscriptions/samples) [repositories](https://github.com/mdecrevoisier/Windows-WEC-server_auto-deploy/tree/master/windows-subscriptions) out there with ideas for additional subscriptions.

#### Activate the subscription

Now that the collector is running, activate the subscription:

```cmd
wecutil cs "<FILE_PATH>\DC_SUBSCRIPTION.xml"
```

The `cs` subcommand stands for “create subscription” and creates a subscription according to the file passed as the next argument.

#### Verify the subscription

Finally, verify that the subscription is active:

```cmd
wecutil gr DC_SUBSCRIPTION
```

The `gr` subcommand stands for “get runtime status” and displays the subscription status for the ID `DC_SUBSCRIPTION`, which corresponds to the `<SubscriptionId>` XML tag in the configuration file.

### Setup Windows Event Forwarding (WEF)

After you completed the server-side configuration, now configure the machines that should log to the WEC.

#### Setup Windows Remote Management (WinRM)

Active WinRM as follows:

```cmd
winrm qc -q \
```

This step is identical to the WinRM configuration on the WEC.

#### Change local group policy settings

Use the Local Group Policy Editor (`gpedit.msc`) to navigate to the `Computer Configuration\Administrative Templates\Windows Components\Event Forwarding` path. Here, you’ll need to open the policy named “Configure the server address, refresh interval, and issuer certificate authority of a target Subscription Manager.”

In the policy settings, enable the policy and click the “Show…” button for SubscriptionManagers. Enter the server details for your WEC:

```plaintext
Server=http://WEC_FQDN:5985/wsman/SubscriptionManager/WEC,Refresh=60
```

Replace `WEC_FQDN` with the actual FQDN of your WEC.

#### Apply the local group policy

Refresh the Local Group Policy settings and apply the changes by running:

```plaintext
gpupdate /force
```

On the WEC, now [verify that the machine forwards events](#verify-the-subscription).

## Collect logs via OpenWEC

Instead of natively running a WEC on a Windows machine, you can also run the third-party implementation [OpenWEC](https://github.com/cea-sec/openwec).

From a functional perspective, this setup is identical to running a native WEC, but it does not require an additional agent at the WEC. In addition, OpenWEC can be scaled redundantly for high availability setups.

### Setup OpenWEC

Refer to the [OpenWEC getting started guide](https://github.com/cea-sec/openwec/blob/main/doc/getting_started.md) for setup instructions.

For running OpenWEC on non-Windows machines that are likely not joined to a Windows domain, it is most useful to configure TLS client authentication. The OpenWEC documentation (see above) recommends to use [a script collection from NXLog](https://gitlab.com/nxlog-public/contrib/-/tree/master/windows-event-forwarding) that creates keys/certificates that can immediately be used to configure both Windows clients and the OpenWEC collector. Make sure to pay attention to specifying the correct hostnames for the sending and receiving machines.

```bash
git clone https://gitlab.com/nxlog-public/contrib
cd contrib/windows-event-forwarding
./genca.sh myca
./gencert-client.sh myclient.domain.com
./gencert-server.sh openwec.domain.com
```

Use the following for the Subscription Manager string:

```plaintext
Server=HTTPS://openwec.domain.com:5985/wsman/,Refresh=14400,IssuerCA=6605742C5400141B76A747E19EA585E29B09F017
```

The string in the last line can be used to configure the Windows Event Forwarding subscription manager on the sending side as described in the section above (“Change local group policy settings”). While you’re at it, also import the `client.pfx` and `ca-cert.pem` into the corresponding stores (see [the documentation](https://github.com/cea-sec/openwec/blob/main/doc/tls.md#client-configuration)).

Then, configure the OpenWEC server as follows (assuming the output files are in `/etc`, the directory `/var/db/openwec` exists and it is writable by the current user):

openwec.conf.toml

```toml
[server]


[logging]
verbosity = "info"


[database]
type = "SQLite"
path = "/var/db/openwec/openwec.sqlite"


[[collectors]]
listen_address = "0.0.0.0"
hostname = "openwec.domain.com"


[collectors.authentication]
type = "Tls"
ca_certificate = "/etc/ca-cert.pem"
server_certificate = "/etc/server-cert.pem"
server_private_key = "/etc/server-key.pem"
```

Create the database (only needs to be done once):

```bash
openwec -c openwec.conf.toml db init
```

Start the server:

```bash
openwecd -c openwec.conf.toml
```

```plaintext
2024-01-30T13:59:26.295792509+01:00 INFO server - Server settings: Server { db_sync_interval: None, flush_heartbeats_interval: None, heartbeats_queue_size: None, node_name: None, keytab: None, tcp_keepalive_time: None, tcp_keepalive_intvl: None, tcp_keepalive_probes: None }
2024-01-30T13:59:26.295947557+01:00 INFO server::subscription - reload_subscriptions task started
2024-01-30T13:59:26.296046314+01:00 INFO server::heartbeat - Heartbeat task started
2024-01-30T13:59:26.297503212+01:00 WARN server::subscription - There are no active subscriptions!
2024-01-30T13:59:26.306151854+01:00 INFO server::tls - Loaded TLS configuration with server certificate /etc/server-cert.pem
2024-01-30T13:59:26.309876793+01:00 INFO server - Server listenning on 0.0.0.0:5985
```

It might make sense to ensure that the server is started and kept up via some automated means, like systemd.

Then, while the server is running, create a subscription in OpenWEC for the desired channels. For example, to match the subscription from the example above, create an XML file like this:

subscription.xml

```xml
<QueryList>
    <Query Id="0">
        <Select Path="Application">*</Select>
        <Select Path="Security">*</Select>
        <Select Path="System">*</Select>
    </Query>
</QueryList>
```

Pass this file to `openwec` to create a subscription, e.g., with name `DC_SUBSCRIPTION`:

```bash
openwec subscriptions new DC_SUBSCRIPTION subscription.xml
```

For the new subscription, configure JSON over TCP as [output](https://github.com/cea-sec/openwec/blob/main/doc/outputs.md):

```bash
openwec subscriptions edit DC_SUBSCRIPTION outputs add --format json tcp 10.0.0.1 1514
```

Finally, enable the subscription:

```bash
openwec subscriptions enable DC_SUBSCRIPTION
```

That’s it! You should now be able read the Windows event logs in JSON format by spinning up a server that listens at `tcp://10.0.0.1:1514`.

### Run a Tenzir pipeline

Accept the logs sent with the configuration above into Tenzir via [TCP](https://preview.docs.tenzir.com/375/375/integrations/tcp.md):

```tql
accept_tcp "10.0.0.1:1514" {
  read_json
}
publish "wec"
```

## Parse Windows Event Log XML

When receiving Windows Event Log data in its native XML format, use [`parse_winlog`](https://preview.docs.tenzir.com/375/375/reference/functions/parse_winlog.md) to convert the XML into structured records. This function is optimized for the [Windows XML Event Log format](https://learn.microsoft.com/en-us/windows/win32/wes/eventschema-schema) and handles the `System`, `EventData`, `UserData`, and `RenderingInfo` sections.

### Parse XML events from a file

```tql
from_file "windows_events.xml" {
  read_delimited "</Event>\n", include_separator=true
}
this = data.parse_winlog()
```

### Filter for specific event IDs

Security monitoring often focuses on specific event types. Filter for logon events (Event ID 4624) and failed logon attempts (Event ID 4625):

```tql
accept_tcp "10.0.0.1:1514" {
  read_delimited "</Event>\n", include_separator=true
}
this = data.parse_winlog()
where System.EventID in [4624, 4625]
```

### Extract EventData fields

The `EventData` section contains event-specific fields. For a successful logon event, extract the relevant information:

```tql
accept_tcp "10.0.0.1:1514" {
  read_delimited "</Event>\n", include_separator=true
}
this = data.parse_winlog()
where System.EventID == 4624
select \
  timestamp = System.TimeCreated.SystemTime,
  computer = System.Computer,
  logon_type = EventData.LogonType,
  target_user = EventData.TargetUserName,
  source_ip = EventData.IpAddress
```

## Increase visibility with Sysmon

[Sysmon](https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon) (System Monitor) is a Windows system service and device driver that, once installed on a system, remains resident across system reboots to monitor and log system activity to the Windows event log. Key features include:

* **Process creation tracking**: Logs details of new processes.
* **Network connection monitoring**: Records incoming and outgoing network connections.
* **File creation time changes**: Tracks changes to file creation times.
* **Driver and image load monitoring**: Logs loading of drivers and DLL files.
* **Registry tracking**: Monitors changes to the Windows registry.

### Download and extract Sysmon via Powershell

Download Sysmon:

```ps
Invoke-WebRequest -Uri "https://download.sysinternals.com/files/Sysmon.zip" -OutFile "Sysmon.zip"
```

Extract the archive:

```plaintext
Expand-Archive -Path Sysmon.zip -DestinationPath Sysmon
```

### Choose a Symon configuration

Choose a suitable Sysmon configuration, e.g., from [Florian Roth](https://github.com/Neo23x0/sysmon-config/) or [SwiftOnSecurity](https://github.com/SwiftOnSecurity/sysmon-config):

```ps
Invoke-WebRequest -Uri "https://raw.githubusercontent.com/Neo23x0/sysmon-config/master/sysmonconfig-export.xml" -OutFile "sysmonconfig-export.xml"
```

### Install Symon with a configuration

```ps
.\Sysmon64.exe -accepteula -i sysmonconfig-export.xml
```

Now use any of the above techniques to collect event logs through the channel `Microsoft-Windows-Sysmon/Operational`.