Skip to content

sumeshi/evtx2es

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

181 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

evtx2es

MIT License PyPI Version

evtx2es logo

A command-line tool and Python library for parsing Windows Event Logs and importing the results into Elasticsearch.

Life is too short to process huge Windows Event Logs using pure Python.
evtx2es leverages the Rust-based parser pyevtx-rs, making it significantly faster than traditional tools. It can also recover as many records as possible from corrupted, partially overwritten, or carved .evtx files.

Usage

evtx2es can be used as a standalone command-line tool or integrated directly into your Python scripts.

$ evtx2es /path/to/your/file.evtx
from evtx2es import evtx2es

evtx2es('/path/to/your/file.evtx')

Arguments

evtx2es can process multiple files at once:

$ evtx2es file1.evtx file2.evtx file3.evtx

evtx2es can recursively process all .evtx files under a specified directory:

$ tree .
evtxfiles/
  β”œβ”€β”€ file1.evtx
  β”œβ”€β”€ file2.evtx
  β”œβ”€β”€ file3.evtx
  └── subdirectory/
    β”œβ”€β”€ file4.evtx
    └── subsubdirectory/
      β”œβ”€β”€ file5.evtx
      └── file6.evtx

$ evtx2es /evtxfiles/ # This recursively processes file1 through file6.

Options

--version, -v

--help, -h

--quiet, -q
  Suppress standard output
  (default: False)

--multiprocess, -m:
  Enable multiprocessing for faster execution
  (default: False)

--size:
  Number of records to process per chunk (default: 500)

--host:
  Elasticsearch host address (default: localhost)

--port:
  Elasticsearch port number (default: 9200)

--index:
  Destination index name (default: evtx2es)

--scheme:
  Protocol scheme to use (http or https) (default: http)

--pipeline:
  Elasticsearch Ingest Pipeline to use (default: )

--datasetdate:
  Date of the latest record in the dataset, extracted from the `TimeCreated` field (MM/DD/YYYY.HH:MM:SS). If omitted, timestamps are not shifted.

--login:
  Username for Elasticsearch authentication

--pwd:
  Password for Elasticsearch authentication

--no-verify-certs:
  Disable TLS certificate verification for Elasticsearch connections (default: False)

Examples

When using from the command line:

$ evtx2es /path/to/your/file.evtx --host=localhost --port=9200 --index=foobar --size=500

When using from a Python script:

evtx2es("/path/to/your/file.evtx", host="localhost", port=9200, index="foobar", chunk_size=500)

With credentials for Elastic Security:

$ evtx2es /path/to/your/file.evtx --host=localhost --port=9200 --index=foobar --login=elastic --pwd=******

Warning

TLS certificate verification is enabled by default for Elasticsearch connections. Use --no-verify-certs only when connecting to a trusted cluster with self-signed or otherwise unverifiable certificates.

Appendix

evtx2json

evtx2es also includes evtx2json, a command-line tool for converting Windows Event Logs into JSON files. 🍣 🍣 🍣

$ evtx2json /path/to/your/file.evtx /path/to/output/target.json

You can also convert .evtx files directly into a Python List[dict] object:

from evtx2es import evtx2json

result: List[dict] = evtx2json('/path/to/your/file.evtx')

Output Format Example

The following example uses a sample .evtx file from JPCERT/CC:LogonTracer.

[
  {
    "@timestamp": "2016-10-06T01:47:07.509504Z",
    "event": {
      "action": "eventlog-security-1102",
      "category": [
        "host"
      ],
      "type": [
        "info"
      ],
      "kind": "event",
      "provider": "microsoft-windows-eventlog",
      "module": "windows",
      "dataset": "windows.eventlog",
      "code": 1102,
      "created": "2016-10-06T01:47:07.509504Z"
    },
    "winlog": {
      "channel": "Security",
      "computer_name": "WIN-WFBHIBE5GXZ.example.co.jp",
      "event_id": 1102,
      "opcode": 0,
      "record_id": 227126,
      "task": 104,
      "version": 0,
      "provider": {
        "name": "Microsoft-Windows-Eventlog",
        "guid": "{fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}"
      }
    },
    "userdata": {
      "LogFileCleared": {
        "#attributes": {
          "xmlns:auto-ns3": "http://schemas.microsoft.com/win/2004/08/events",
          "xmlns": "http://manifests.microsoft.com/win/2004/08/windows/eventlog"
        },
        "SubjectUserSid": "S-1-5-21-1524084746-3249201829-3114449661-500",
        "SubjectUserName": "Administrator",
        "SubjectDomainName": "EXAMPLE",
        "SubjectLogonId": "0x32cfb"
      }
    },
    "process": {
      "pid": 960,
      "thread": {
        "id": 3020
      }
    },
    "log": {
      "file": {
        "path": "/path/to/your/Security.evtx"
      }
    },
    "tags": [
      "eventlog"
    ]
  },
  ...
]

Performance Evaluation (v1.8.0)

Performance was evaluated using a sample .evtx file from JPCERT/CC:LogonTracer (approx. 30MB of binary data).

$ time uv run evtx2es Security.evtx 
Currently Importing Security.evtx.
1it [00:08,  8.09s/it]
Bulk import completed: 1 batches processed
Successfully indexed: 62031 documents
Import completed.

________________________________________________________
Executed in    8.60 secs    fish           external
   usr time    4.85 secs  481.00 micros    4.85 secs
   sys time    0.40 secs    0.00 micros    0.40 secs

Running Environment

OS: Ubuntu 20.04 (Dev Container on WSL2)
CPU: Intel Core i5-12400F
RAM: DDR4 32GB

The tests were conducted within the provided development container, pushing data into a local Elasticsearch 9.0.2 Docker container.

Installation

From PyPI

$ pip install evtx2es

With uv

$ uv add evtx2es

From GitHub Releases

Standalone binaries built with Nuitka are available from GitHub Releases for systems without a Python environment.

$ chmod +x ./evtx2es
$ ./evtx2es {{options...}}
> evtx2es.exe {{options...}}

Contributing

The source code for evtx2es is hosted on GitHub: https://github.com/sumeshi/evtx2es. Please report issues and feature requests. 🍣 🍣 🍣

Included in

Thank you for your interest in evtx2es!

License

Released under the MIT License.

Third-party licenses

The standalone binaries distributed via GitHub Releases may bundle the following third-party libraries. These libraries remain under their original licenses.

Apache-2.0

MIT

Apache-2.0 OR MIT, with MPL-2.0 components

MIT and MPL-2.0

About

A command-line tool and Python library for parsing Windows Event Logs and importing the results into Elasticsearch.

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Contributors