Add PostHog LLM Analytics plugin for OpenCode#1
Conversation
src/events.ts
Outdated
| trace: TraceState, | ||
| config: PluginConfig, | ||
| ): CaptureEvent { | ||
| const spanId = randomUUID() |
There was a problem hiding this comment.
trace.currentGenerationSpanId is only assigned inside buildAiGeneration(), i.e. after the step-finish part arrives. OpenCode emits tool-part updates during the step and only emits step-finish afterwards (tool-result is processed before finish-step: https://github.com/sst/opencode/blob/main/packages/opencode/src/session/processor.ts#L215-L285). That means any $ai_span you emit for a tool call in the current turn will get null here or, worse, the previous generation's span ID. The reference @posthog/pi implementation avoids this by allocating the generation span ID at turn start, before tool events begin: https://github.com/PostHog/posthog-pi/blob/0.1.1/src/index.ts#L167-L189
There was a problem hiding this comment.
Fixed — generation span ID is now allocated eagerly on step-start (before any tool events fire), so tool spans always get the correct $ai_parent_id. The buildAiGeneration function reuses the pre-allocated ID rather than generating a new one. See edfc0be.
| const omitted = value.length - maxLength | ||
| return `${value.slice(0, maxLength)}...[truncated ${omitted} chars]` | ||
| } | ||
|
|
There was a problem hiding this comment.
serializeAttribute() only redacts object keys; if the value is already a string it just truncates and returns it verbatim. OpenCode's SDK defines completed tool output as a plain string (output: string): https://github.com/sst/opencode/blob/main/packages/sdk/js/src/gen/types.gen.ts#L259-L268. So a tool result like {"api_key":"secret"} or a shell command that prints a token is forwarded to PostHog unredacted, which leaks exactly the kind of secrets this helper is supposed to scrub.
There was a problem hiding this comment.
Fixed — redactSensitive now handles string values by scanning for inline sensitive patterns (both JSON-style "api_key":"..." and password=... formats). Previously strings passed straight through to truncation, so tool output containing secrets would leak. See edfc0be.
Allocate generation span ID on step-start instead of step-finish so tool spans emitted during the step reference the correct parent. Scan plain strings for sensitive key-value patterns (JSON fragments, key=value) since tool output arrives as a string and was previously passed through unredacted.
Covers mapStopReason, buildAiGeneration, buildAiSpan, buildAiTrace, redactForPrivacy, serializeAttribute (including string-level redaction), and serializeError. Tests privacy mode, custom tags, error handling, pre-allocated span IDs, truncation, circular refs, and deep nesting.
…text 1. String redaction now consumes full multi-word values after sensitive keys (e.g. "Authorization: Bearer secret-token") instead of stopping at the first whitespace boundary. 2. All $ai_error fields and trace.lastError now route through serializeAttribute instead of raw JSON.stringify, so sensitive keys in error payloads are redacted before being sent to PostHog. 3. Generations now use per-step accumulated input messages (including tool results) rather than just the initial user prompt, making multi-step/tool-assisted analytics accurate.
OpenCode installs plugin dependencies at startup, so posthog-node is always available when the plugin runs. Replace the lazy ensureClient pattern with a straightforward static import and direct construction.
1. Tool output stored in stepInputMessages now goes through serializeAttribute before being appended, so secrets in tool results are redacted in $ai_generation.$ai_input too. 2. Track message IDs per trace and delete entries from messageRoles and assistantMessages on session.idle, preventing unbounded memory growth in long-lived processes. 3. Fix local development instructions in README — the plugin has multiple source files, so copying just index.ts doesn't work.
- Version 0.0.1, author Nejc Drobnič <nejc@nejc.dev> - Reorder package.json fields: identity → module config → files → keywords → scripts → dependencies - Add GitHub Actions CI workflow (typecheck + test) with setup action from posthog-pi, add .nvmrc for Node 22
- Add oxlint (lint) and oxfmt (format) with configs matching posthog-pi - Reformat all source files to single quotes, 4-space indent, 120 cols - Add MIT license headers to CI workflow and setup action - Derive VERSION from package.json instead of hardcoding it - Add lint step to CI workflow - Fix no-unneeded-ternary lint warning in events.ts
OpenCode runs under Bun, so align the toolchain. Replace pnpm with bun install/lockfile, update CI setup action to use oven-sh/setup-bun, remove .nvmrc and packageManager field, simplify version.ts to use a direct JSON import instead of createRequire.
1. Snapshot stepInputMessages at step-start into stepInputSnapshot. buildAiGeneration reads from the snapshot, so tool results from the current step don't leak into the same generation's $ai_input. Tool results are still appended to stepInputMessages for the next step's snapshot. 2. Rename $ai_cache_read_input_tokens → cache_read_input_tokens and $ai_cache_creation_input_tokens → cache_creation_input_tokens to match PostHog's LLM Analytics schema (and posthog-pi reference).
Add exports map, module/types fields, build script (bun build), prepublishOnly hook, repository/bugs/homepage URLs. Update tsconfig with outDir/declarationDir/rootDir and bun-types. Replace @types/node with @types/bun to match the runtime.
1 participant
Summary
Implements an OpenCode plugin that captures LLM generations, tool executions, and conversation traces as structured
$ai_*events for PostHog's LLM Analytics dashboard. This is a port of@posthog/piadapted to OpenCode's plugin system and event model.The plugin emits
$ai_generation(per LLM roundtrip),$ai_span(per tool call), and$ai_trace(per user prompt) with full token counts (input, output, reasoning, cache), model/provider tracking, cost, and latency. Includes privacy mode for content redaction and always-on sensitive key redaction. Configured entirely via environment variables; no-op whenPOSTHOG_API_KEYis unset.