Skip to content

README.md

Latest commit

 

History

History
146 lines (104 loc) · 29 KB

README.md

File metadata and controls

146 lines (104 loc) · 29 KB

Codemap — documentation index

Technical docs for @stainless-code/codemap.

Start here: ../README.md (install, CLI, API, dev commands). This folder is deeper reference — pick a row below.

File Ownership

Each topic has exactly one canonical file. Other files cross-reference by relative path, never duplicate.

File Topic
why-codemap.md Why index + SQL for agents (speed, tokens, accuracy). Anti-pitch (When to reach for something else) and alternatives comparison. Good first read after the readme.
architecture.md Schema, layering, CLI internals, API, User config (Zod), parsers, Key Files.
glossary.md Canonical term definitions. Disambiguates pairs like FileRow vs files table, recipe vs query, schema vs DDL, hub vs barrel.
agents.md codemap agents init — bundled templates/agents/ (thin pointer files) → .agents/ in consumer projects; full content served live by codemap skill / codemap rule + codemap://skill / codemap://rule from templates/agent-content/; section assembler + *.gen.md renderers, pointer protocol + staleness nag, per-file IDE symlink/copy, --interactive, --mcp, <state-dir>/.gitignore reconciler (root .gitignore untouched).
benchmark.md Indexing another project · Benchmark script · Query stdout (table vs JSON) · Custom scenarios (CODEMAP_BENCHMARK_CONFIG) · Agent eval harness · fixtures/minimal/.
golden-queries.md Golden query design & policy (Tier A/B, no proprietary trees); runner: scripts/query-golden.ts.
fixtures/golden/ scenarios.json + minimal/ — bun run test:golden; Tier B: scenarios.external.example.json + bun run test:golden:external (benchmark § Fixtures).
fixtures/benchmark/ Tracked scenarios.example.json — copy to *.local.json (gitignored) for CODEMAP_BENCHMARK_CONFIG.
fixtures/qa/ prompts.external.template.md — optional chat QA prompts for an external index (*.local.md gitignored).
packaging.md CHANGELOG.md / dist/ / templates/ on npm, engines, Node vs Bun, Releases (Changesets; bun run version + oxfmt CHANGELOG.md).
roadmap.md Forward-looking Backlog and Non-goals (not a src/ inventory).
plans/ One <feature-name>.md per in-flight plan. Created on demand — don't add the -plan suffix; the folder provides context. See folder contents for the current in-flight set; avoid maintaining a duplicate inline list.
audits/ Targeted architecture / performance / lifecycle audits. None open.
research/ Dated snapshot notes for open evaluations. Closed adopted notes delete after lift; rejected notes keep a one-line status header only.

Rules for Agents

These rules are normative — cite them by number in PR review. Ordered by how often they fire, not severity.

  1. One source of truth — every topic lives in exactly one file. Other files cross-reference by relative path; never duplicate prose. See the Single source of truth table for cross-cutting topics.

  2. When a backlog item ships — move the description from roadmap.md to its canonical home (architecture.md, why-codemap.md, or root README.md). Remove the item from roadmap.md entirely; the roadmap is forward-looking.

  3. When adding a feature plan — create plans/<feature-name>.md. Don't embed plans in roadmap.md; link from there.

  4. Keep ownership tables current — when creating or deleting a doc file, update the File Ownership and Single source of truth tables in the same PR. A stale table is worse than no table.

  5. Cross-references use relative paths — [architecture.md § Section](./architecture.md#section) or [plans/foo.md](./plans/foo.md). Prefer section-deep links over file-only links.

  6. No inventory counts in narrative — don't hardcode counts of files, symbols, recipes, or other code-derived quantities. Use qualitative descriptors or a codemap query --json example. Decision values (cache TTLs, batch sizes, schema version) are fine — those are decisions, not inventory.

  7. No line-number references — line numbers (e.g. parser.ts:241) rot on every edit. Reference by function name, section heading, or symbol from codemap query instead. Methodology tables in benchmark.md are exempt.

  8. Research notes get closed — default: lift + delete. When adopt items ship, move decisions-of-record into canonical homes (architecture.md, glossary.md, roadmap.md, a plan, .agents/rules/). Delete the research file once nothing in source cites it by path. Rejected-only keep: add Status: Rejected (YYYY-MM-DD) — <one-line reason> and stop — no "analytical history" appendices. Slim + keep only when inbound source cites (rule numbers, NOTE(...), tests) would orphan — then keep cited sections + status header, not the full evaluation prose. Do not retain "What shipped" inventory tables or git log / git show recovery rows in living docs.

  9. New term ⇒ update glossary.md in the same PR — when a PR introduces a new domain noun / verb / acronym (table name, recipe id, parser name, schema column), add or update its entry. Disambiguations (e.g. FileRow TS shape vs files SQLite table) take priority over single defs.

  10. Core surface change ⇒ check which agent-content layer it belongs to — the v1 pointer pattern split the agent surface in two:

    • Auto-flows (no template edit needed) — recipe additions (templates/recipes/*.{sql,md}), schema additions (src/db.ts createTables()). Both surfaces via *.gen.md renderers in src/application/agent-content.ts; codemap skill re-assembles every call, while MCP/HTTP memoize the assembled skill/rule/schema/mcp-instructions body per server process.
    • Narrative changes — new CLI flag / output mode / MCP tool / HTTP route / output-shape change → edit the relevant hand-written section in templates/agent-content/skill/*.md (single source of truth; codemap skill (CLI), codemap://skill (MCP), and GET /resources/{encoded-uri} against codemap serve (HTTP) all serve the same assembled body).
    • Pointer-shape changes (frontmatter schema, fetch instructions, marker comments) → edit templates/agents/{rules/codemap,skills/codemap/SKILL}.md AND bump EXPECTED_POINTER_VERSION in agent-content.ts so consumers see the staleness nag and re-run codemap agents init --force.
    • Consumer-only surfaces — templates/agent-content/, templates/agents/, .changeset/ bodies, and user-facing CLI text describe user-visible behavior only. No maintainer internals (Action wiring, src/ module names, dual-file sync, dogfood spawn paths). Full policy: .agents/rules/consumer-surfaces.md.

    This repo's .agents/{rules/codemap,skills/codemap/SKILL}.md are thin pointers too (regenerate via bun src/index.ts agents init --force if pointer shape drifts) — they used to be the dev-side "second copy" to keep in sync; that obligation is gone.

Single source of truth (do not duplicate)

Cross-cutting topics that span multiple files. Each has exactly one canonical home; other files link, never copy.

Topic Canonical doc Elsewhere
Runtime splits (SQLite, workers, globs, JSON config I/O) packaging § Node vs Bun — the table lives here architecture § Runtime links here; do not copy the table
<state-dir>/config.{ts,js,json} shape / Zod validation architecture § User config Root README points here for config shape
codemap agents init: pointer pattern (templates/agents/ → consumer-disk pointers; templates/agent-content/ → live-served full content), codemap skill / codemap rule, codemap://skill / codemap://rule, section assembler + *.gen.md renderers, EXPECTED_POINTER_VERSION + staleness nag, --force, IDE matrix, per-file symlink/copy agents.md Link here; do not paste the integration table into README or packaging
CLAUDE.md / AGENTS.md / GEMINI.md / Copilot — managed codemap-pointer sections, merge vs --force agents.md § Pointer files Link here; do not duplicate the situation table
End-user CLI (index, query --json, query --recipe, query --recipes-json, query --print-sql, skill, rule, agents, flags, env) — query has no row cap; use SQL LIMIT; --json errors include SQL, DB open, and bootstrap failures; bundled templates/agent-content/skill/*.md examples default to --json ../README.md § CLI architecture § CLI usage summarizes and links back; agents.md
Golden query regression (test:golden, test:golden:external, --update) golden-queries.md CONTRIBUTING § Golden queries; benchmark § Fixtures
Agent eval harness (test:agent-eval, scripts/agent-eval/) benchmark § Agent eval harness Reuses golden scenarios via goldenId; probe + live + log structural cost A/B; exploratory MCP vs agent findings in research/agent-eval-findings-2026-05.md
CODEMAP_BENCHMARK_CONFIG (per-repo benchmark JSON) benchmark § Custom scenarios fixtures/benchmark/scenarios.example.json only
bun run qa:external — index + disk checks + benchmark.ts on CODEMAP_* .github/CONTRIBUTING.md scripts/qa-external-repo.ts (invocation only)
Non-goals (v1) — deliberate product floors (FTS default-off, no LSP shim, predicate-as-API not verdict engine, no one-shot CLI daemon, no deep intent classification) roadmap.md § Non-goals why-codemap.md § When to reach for something else (consumer-facing framing) — links here; research/ notes link here, never re-list
Domain term definitions (FileRow vs files, recipe vs query, schema vs DDL, hub vs barrel, fan-in vs fan-out, …) glossary.md Other docs link to a glossary entry on first use; never inline a definition that conflicts

Document Lifecycle

Every doc here falls into one of five types. New content fits an existing type, or absorbs into an existing file — it does not spawn a new top-level doc by default.

Types

Type Folder Lifecycle
Reference root (architecture.md, agents.md, benchmark.md, golden-queries.md, packaging.md, glossary.md, why-codemap.md) Lives forever. Kept current per Rules 4, 7, 9.
Roadmap root (roadmap.md, single file) Lives forever. Items move in (new findings) and out (per Rule 2).
Plan plans/<name>.md Created when work commits. Deleted when work ships (per Rule 3).
Audit audits/<topic>.md Created at audit time. Closed per the docs-governance skill's audit lifecycle.
Research research/<topic>.md Created on demand for a third-party scan or evaluation. Closed per Closing research.

Backlogs, frameworks, and decisions don't get their own top-level file. They fold into one of the five:

  • Backlogs of open items → a section in roadmap.md.
  • Frameworks / playbooks → architecture.md if Codemap-internal, or .agents/rules/ / .agents/skills/ if project-wide policy.
  • Decisions of record from concluded research → lift into the relevant reference doc; the research file's job is the evaluation, not the decision.

Existence test (apply on every doc-touching PR)

A file earns its place if it meets at least one of:

  1. Source code or another doc cites it (grep finds the path).
  2. It documents durable policy or framework unavailable elsewhere.
  3. It tracks open work (open audit findings, in-flight plan, roadmap items).
  4. Inbound source cites require a slim stub — JSDoc, rules, tests, or plans link to this file by path/anchor; deletion would orphan them. (Not "interesting history" — if only git could reconstruct it, delete.).

If none → fold any salvageable content into roadmap / architecture / glossary, fix the cross-refs, delete the file.

Closing research

A research note's job is the evaluation. When it concludes:

  • Adopted → lift the decision-of-record into the relevant reference doc; delete the research file when nothing cites it by path. Rejected-only keep per Rule 8 above.
  • Rejected → add Status: Rejected (YYYY-MM-DD) — <one-line reason> at the top. Keep the file. Don't delete; the rejection rationale saves the next agent from re-litigating.
  • Open → stays in research/ with no status header (open is the default).

Top-level cap

Adding a new top-level doc requires:

  1. The topic doesn't fit any existing root-level doc.
  2. The new file passes the existence test on day one.
  3. File Ownership table updated in the same PR.

When in doubt, default to absorbing into the closest existing root-level file (usually roadmap.md for forward-looking work, architecture.md for shipped behavior, glossary.md for terminology, research/ for snapshot notes).

Closing audits

When an audit closes, lift shipped work into canonical homes (architecture.md, a plan, .agents/lessons.md, roadmap.md backlog). Do not leave tombstones — no pointer table, no "recover via git log --follow -- <deleted-path>" rows in living docs. Deleted audit text lives in git history only; cite the shipping PR or commit when closure needs a durable anchor.

  • Delete when the re-derivable test passes (findings visible in source / no source-cites / no unique policy) — see docs-governance § Closing an audit.
  • Slim + keep in audits/ when the file carries decisions-of-record, source back-references, or methodology not captured elsewhere — add a Status: Closed header.
  • Absorb into a plan when the audit is the synthesis substrate for in-flight work — the plan's provenance block owns recovery (git show <sha> -- docs/audits/… belongs there, not in docs/README.md).

Naming Conventions

  • plans/ files: <feature-name>.md — the folder provides "plan" context, don't add a -plan suffix.
  • research/ files: <topic>-YYYY-MM.md for dated snapshots; <tool-name>.md for ongoing tool evaluations. Delete after lift when adopted (Rule 8).
  • Top-level files: descriptive domain noun (architecture.md, glossary.md, roadmap.md) — no prefix or suffix.

Conventions

Stylistic addendum to the rules above:

  • CLI flags and examples — canonical README.md § CLI. Other docs summarize and link; do not copy full flag lists. Implementation paths (src/cli/…, QUERY_RECIPES) belong in architecture.md § CLI usage only.
  • This repo: bun run dev is bun src/index.ts; bun run build → tsdown → dist/; bun run clean / bun run check-updates — see .github/CONTRIBUTING.md.
  • Contributors: branch + PR into main (CI), bun run check, JSDoc on public API.