actfite

actfite is a SQLite run-time loadable extension to query file-based Common Trace Format 2 (CTF) traces. It supports opening a CTF trace as a virtual table where each row is an event and uses the following schema:

CREATE TABLE x(
    timestamp_ns INTEGER, -- nanoseconds from origin
    namespace TEXT, -- namespace of the event
    name TEXT, -- name of the event
    uid TEXT, -- unique id of the event within the scope of the namespace and name
    packet_header NULL, -- header of the packet
    packet_context NULL, -- context of the packet
    header NULL, -- header of the event
    common_context NULL, -- common context of the event
    specific_context NULL, -- specific context of the event
    payload NULL -- payload of the event
);

The fields are represented by the NULL datatype and are not directly accessible, instead you are expected to use specific functions (ctf/ctf_extract) to access them. This is deliberate to allow extraction of individual fields without having to deserialize all fields of an event.

Compiling

The extension is dependent on headers from sqlite3 and the library actf (which in turn depends on json-c), so depending on how you have installed them, you might need to modify the import paths -I and -L. The command will be something like this:

gcc -fPIC -O3 -shared actfite.c -o libactfite.so -lactf

Loading actfite and a trace

Assuming actfite was built as libactfite.so, it can be loaded from the sqlite3 command-line shell and be used to create a virtual table from a CTF trace like this:

.load ./libactfite.so
CREATE VIRTUAL TABLE temp.t1 USING actfite(tracedir);
select * from t1;

Each argument given to the virtual table will be interpreted as a path to a CTF trace.

The ctf function

The ctf(X) function takes a single CTF field column X and returns it as a string.

Example: ctf(payload) -> { name: "setting the table", args: "" }

Suppose you have a CTF table called myctf, then you could print all fields like this:

SELECT
    timestamp_ns, name, uid,
    ctf(packet_header), ctf(packet_context),
    ctf(header),ctf(common_context),ctf(specific_context),ctf(payload)
FROM myctf;

The ctf_extract function

The ctf_extract(X,P) function returns the CTF field at PATH P from CTF column X. The PATH is inspired by SQLite's JSON PATH. If the PATH is invalid, an error is thrown. If the PATH does not point to an actual field in the event, NULL is returned.

A valid PATH is a text value that begins with a $ and is followed by zero or more .objectlabel or [arrayindex]. The arrayindex is either a non-negative integer N, where the N-th element is selected, or the special form #-N, where the N-th from the right element selected (last element is #-1).

If objectlabel contains a character with special meaning (., [, \), you must escape the offending character with \.

Example: ctf_extract(payload,'$.name') -> setting the table

Implementation details

There is no indexing, meaning all queries will result in a full table/trace scan.

Each cursor will walk through the CTF directory and open the datastream files which could be slow if you have lots of datastreams and/or have them on a slow disk.