Skip to content

feat: document model alias logging and wire debugTokens through config#3782

Merged
lpcox merged 7 commits into
mainfrom
docs/model-alias-logging
May 25, 2026
Merged

feat: document model alias logging and wire debugTokens through config#3782
lpcox merged 7 commits into
mainfrom
docs/model-alias-logging

Conversation

@lpcox
Copy link
Copy Markdown
Collaborator

@lpcox lpcox commented May 25, 2026

Summary

Adds first-class config support and documentation for model alias logging in the API proxy.

Config changes

  • apiProxy.logging.debugTokens (boolean): enables detailed token and model-alias diagnostic logging (AWF_DEBUG_TOKENS=1)
  • apiProxy.logging.tokenLogDir (string): directory for token-usage.jsonl and token-diag.jsonl (AWF_TOKEN_LOG_DIR)

Both are now wirable through the AWF config file (previously only available as raw env vars on the container).

File renames (JSONL extension consistency)

  • token-diag.logtoken-diag.jsonl (writes JSONL records)
  • cli-proxy access.logaccess.jsonl (writes JSON records)

Documentation (config spec §13)

  • §13.1: Always-on stdout events (model_resolution, model_rewrite, model_fallback_*)
  • §13.2: Diagnostic events (MODEL_ALIAS_RESOLUTION_STEP, MODEL_ALIAS_REWRITE)
  • §13.3: Configuration example
  • §13.4: Comprehensive log file inventory — all JSONL and text log files produced by Squid, API proxy, and CLI proxy
  • §13.5: Availability (since v0.25.40)
  • Updated Runtime JSONL Schemas table to list all 5 JSONL files

Files changed

  • src/awf-config-schema.json + docs/awf-config.schema.json — schema
  • src/types/api-proxy-options.ts — type definitions
  • src/config-file.ts — config mapping
  • src/commands/build-config.ts — build config wiring
  • src/services/api-proxy-service.ts — env var injection
  • containers/api-proxy/token-persistence.js — .jsonl rename
  • containers/api-proxy/server.models.test.js — test update
  • containers/cli-proxy/server.js — .jsonl rename
  • docs/awf-config-spec.md — documentation

- Add apiProxy.logging.debugTokens and apiProxy.logging.tokenLogDir to
  the config schema, types, and env var wiring (AWF_DEBUG_TOKENS,
  AWF_TOKEN_LOG_DIR)
- Document all model alias logging events (stdout and token-diag) in
  config spec §13
- Add comprehensive log file inventory (§13.4) listing all JSONL and
  text log files produced by Squid, API proxy, and CLI proxy
- Rename token-diag.log → token-diag.jsonl (it writes JSONL records)
- Rename cli-proxy access.log → access.jsonl (it writes JSON records)
- Update Runtime JSONL Schemas table to include token-diag.jsonl,
  otel.jsonl, and cli-proxy access.jsonl

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 25, 2026 16:33
@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

Documentation Preview

Documentation build failed for this PR. View logs.

Built from commit 405e660

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions github-actions Bot mentioned this pull request May 25, 2026
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds config-file support and documentation for API-proxy model-alias/token diagnostic logging, and standardizes structured log file extensions to .jsonl for consistency across components.

Changes:

  • Add apiProxy.logging.debugTokens and apiProxy.logging.tokenLogDir to the AWF config schema, config-file mapping, and runtime env injection for the api-proxy container.
  • Rename diagnostics/audit log filenames to .jsonl where the content is JSONL (api-proxy token-diag.*, cli-proxy access.*) and update the related test.
  • Expand docs/awf-config-spec.md with a new “Model Alias Logging” section and an inventory of runtime log artifacts.
Show a summary per file
File Description
src/types/api-proxy-options.ts Adds typed options for debugTokens / tokenLogDir with JSDoc.
src/services/api-proxy-service.ts Wires debugTokens / tokenLogDir into api-proxy container environment variables.
src/config-file.ts Extends config-file shape and maps apiProxy.logging.* into internal options.
src/commands/build-config.ts Plumbs debugTokens / tokenLogDir into WrapperConfig assembly.
src/awf-config-schema.json Adds apiProxy.logging object to the config schema.
docs/awf-config.schema.json Mirrors schema changes for documentation consumption.
docs/awf-config-spec.md Documents model-alias logging events, configuration, and log file inventory.
containers/cli-proxy/server.js Renames structured access log output to access.jsonl.
containers/api-proxy/token-persistence.js Renames diagnostic log output to token-diag.jsonl.
containers/api-proxy/server.models.test.js Updates test expectations for the new token-diag.jsonl filename.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 10/10 changed files
  • Comments generated: 5

Comment thread src/commands/build-config.ts Outdated
Comment thread src/awf-config-schema.json
Comment thread docs/awf-config.schema.json
Comment thread docs/awf-config-spec.md Outdated
Comment thread src/types/api-proxy-options.ts Outdated
lpcox and others added 5 commits May 25, 2026 09:40
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Update test assertions to match the current smoke-claude.md (max-turns
changed from 2 to 10) and the current lock file (gh-aw-actions/setup
at v0.74.4).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 96.53% 96.58% 📈 +0.05%
Statements 96.37% 96.42% 📈 +0.05%
Functions 97.99% 97.99% ➡️ +0.00%
Branches 90.88% 90.84% 📉 -0.04%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/config-writer.ts 89.3% → 90.9% (+1.65%) 89.3% → 90.9% (+1.65%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

Smoke Test Results

PR #3779: Increase smoke-claude max-turns from 2 to 10

✅ GitHub MCP connectivity
✅ github.com connectivity
✅ File write/read test

Status: PASS

cc: @lpcox

📰 BREAKING: Report filed by Smoke Copilot

@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

📡 OTEL Tracing Smoke Test Results

All scenarios validated successfully:

Module Loading — otel.js loaded, isEnabled=true, exports 11 functions
Test Suite — 32/32 tests passed (span creation, attributes, export, parent context)
Env Var Forwarding — OTEL_EXPORTER_OTLP_ENDPOINT and GITHUB_AW_OTEL_TRACE_ID forwarded
Token Tracker Integration — onUsage callback present in token-tracker-http.js
i️ OTEL Diagnostics — No spans exported (no OTEL endpoint configured in this run, expected)

Status: Infrastructure ready for OTEL tracing when endpoint is configured.

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

Smoke Codex: FAIL
PRs: Increase smoke-claude max-turns from 2 to 10; fix: remove unused export from normalizeDockerHostPathPrefix
✅ GitHub PR review; ❌ safeinputs-gh query unavailable
✅ Playwright title contains GitHub; ❌ Tavily exposed no search tool
✅ file/bash check; ✅ npm ci && npm run build
Discussion comment skipped: github-discussion-query unavailable

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

Smoke Test: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

Chroot Environment Version Comparison

The chroot test compared runtime versions between the host and the chrooted environment:

Runtime Host Version Chroot Version Match?
Python Python 3.12.13 Python 3.12.3 ❌ NO
Node.js v24.15.0 v22.22.3 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Overall Result: ❌ Tests failed - version mismatches detected

Python and Node.js versions differ between the host and chrooted environment, which may indicate the chroot is not properly inheriting the host system binaries or has separate installations.

Tested by Smoke Chroot

@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

Service Connectivity Test Results

Redis: Connection timeout (no response)
PostgreSQL pg_isready: Connection timeout (no response)
PostgreSQL query: Not tested (pg_isready failed)

Overall: FAIL

Services on host.docker.internal are not reachable from the AWF sandbox.

🔌 Service connectivity validated by Smoke Services

@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx 1/1 passed ✅ PASS
Node.js execa 1/1 passed ✅ PASS
Node.js p-limit 1/1 passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

All build and test operations completed successfully across all 18 projects in 8 ecosystems.

Generated by Build Test Suite for issue #3782 · ● 13.6M ·

@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 25, 2026

Copilot BYOK Smoke Test Results

GitHub MCP - API connectivity verified
GitHub.com - Network access confirmed
File I/O - Write/read test passed: smoke-test-copilot-byok-26410803429.txt
BYOK Inference - Offline mode working via api-proxy → api.githubcopilot.com

Mode: BYOK offline (COPILOT_OFFLINE=true) - agent uses dummy key, api-proxy holds real COPILOT_GITHUB_TOKEN

Status: ✅ PASS

@lpcox - All systems operational

Note

🔒 Integrity filter blocked 1 item

The following item was blocked because it doesn't meet the GitHub integrity level.

  • #3783 list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

🔑 BYOK report filed by Smoke Copilot BYOK

@lpcox lpcox merged commit 08697d0 into main May 25, 2026
67 of 70 checks passed
@lpcox lpcox deleted the docs/model-alias-logging branch May 25, 2026 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants