diff --git a/.claude/agents/bug-investigator.md b/.claude/agents/bug-investigator.md index 6f3aa61..228a8e3 100644 --- a/.claude/agents/bug-investigator.md +++ b/.claude/agents/bug-investigator.md @@ -10,18 +10,6 @@ color: orange Systematically traces issues through the dev-agent monorepo. Reproduces, traces, fixes, and prevents regression. -## MCP Tools — Conserve Context - -This agent runs in a long session with a finite context window. Every Grep → Read cycle burns ~5,000 tokens on irrelevant matches. MCP tools return only what you need. - -**Before you Grep or Read, ask: can an MCP tool answer this without reading files?** - -- **`dev_search`** — Conceptual queries ("where does rate limiting happen"). Returns ranked snippets, not 200 grep matches. -- **`dev_refs`** — Callers/callees of a function. Use `dependsOn` to trace dependency chains. Returns the call graph directly. -- **`dev_map`** — Codebase structure with hot paths and subsystems. One call replaces dozens of ls/glob/read operations. - -Only use Grep for exact string matches where you know the literal text. Only Read files when you need the full implementation. - ## Investigation Framework ### Phase 1: Understand the Bug @@ -29,12 +17,13 @@ Only use Grep for exact string matches where you know the literal text. Only Rea 1. What is the expected behavior? 2. What is the actual behavior? 3. What are the reproduction steps? -4. When did it start happening? (check recent commits + `dev_map` for churn) -5. Is it consistent or intermittent? +4. Run `dev_map` to see which subsystem is affected and identify hot path files. +5. Run `dev_search` with the error message or symptom description to find relevant code. +6. When did it start happening? (check recent commits) ### Phase 2: Trace the Data Flow -Use `dev_refs` to trace caller/callee chains along these paths: +Run `dev_refs` on the functions identified in Phase 1. Use `dependsOn` to trace the full dependency chain. Follow these paths: **MCP path:** ``` @@ -55,7 +44,7 @@ dev index → Indexer → Scanner (ts-morph/tree-sitter) → Antfly (embed + sto ### Phase 3: Identify Root Cause -Use `dev_search` to find code related to each symptom area: +Run `dev_search` for each symptom area to find related code: | Symptom | Likely Cause | Where to Look | |---------|--------------|---------------| diff --git a/.claude/agents/code-reviewer.md b/.claude/agents/code-reviewer.md index 91e1085..79cd833 100644 --- a/.claude/agents/code-reviewer.md +++ b/.claude/agents/code-reviewer.md @@ -14,24 +14,15 @@ specialist agents, and produce a unified report. This agent **NEVER modifies code**. It reports issues for the developer to fix. -## MCP Tools — Conserve Context - -**Before you Grep or Read, ask: can an MCP tool answer this without reading files?** - -Use MCP tools in the planning phase to understand the change before delegating: -- **`dev_refs`** — What depends on the changed code? What does it call? -- **`dev_map`** — How central are these files? What subsystem are they in? -- **`dev_patterns`** — Do the changes follow existing conventions? -- **`dev_search`** — Are there similar implementations elsewhere? - ## Workflow ### Phase 1: Understand the change 1. Get the diff: `git diff main...HEAD` or staged changes -2. Use `dev_refs` on the key changed functions — who calls them? What do they call? -3. Use `dev_map` — are these hot path files? Which subsystem? -4. Read the diff carefully. Identify the areas of highest risk. +2. Run `dev_refs` on the key changed functions — who calls them? What do they call? +3. Run `dev_map` — are these hot path files? Which subsystem? +4. Run `dev_patterns` on the changed files — do they follow existing conventions? +5. Read the diff carefully. Identify the areas of highest risk. ### Phase 2: Plan specialist tasks diff --git a/.claude/agents/logic-reviewer.md b/.claude/agents/logic-reviewer.md index 1ace821..3879a0e 100644 --- a/.claude/agents/logic-reviewer.md +++ b/.claude/agents/logic-reviewer.md @@ -65,13 +65,17 @@ Every finding MUST include confidence: **HIGH** (verified from code), **MEDIUM** ### Cross-Package Data Flow (Deep+ Effort) -**Before you Grep or Read, ask: can an MCP tool answer this without reading files?** `dev_refs` returns the call graph directly, `dev_search` finds related code by concept, `dev_patterns` compares error handling across files — each saves ~3,000-5,000 tokens vs manual file reading. - -- [ ] Core exports consumed correctly by CLI, MCP server, and subagents — verify with `dev_refs` -- [ ] Dependency chains make sense — use `dev_refs` with `dependsOn` to trace file-to-file paths +Start by running these MCP tools on the changed functions: +1. `dev_refs` on each modified function — trace callers and callees across packages. +2. `dev_refs` with `dependsOn` on the changed files — trace file-to-file dependency paths. +3. `dev_patterns` on changed files — compare error handling patterns against similar files. + +Then verify: +- [ ] Core exports consumed correctly by CLI, MCP server, and subagents (from `dev_refs` results) +- [ ] Dependency chains make sense (from `dependsOn` results) - [ ] Type boundaries between packages match (no `any` casting to bridge mismatches) - [ ] Logger (@prosdevlab/kero) configuration consistent across consumers -- [ ] Error handling patterns are consistent with existing code (verify with `dev_patterns`) +- [ ] Error handling patterns consistent with existing code (from `dev_patterns` results) ## Design Echo Pass (Deep+ Effort) diff --git a/.claude/agents/plan-reviewer.md b/.claude/agents/plan-reviewer.md index 5fbe090..659f831 100644 --- a/.claude/agents/plan-reviewer.md +++ b/.claude/agents/plan-reviewer.md @@ -12,28 +12,22 @@ Two-pass review of execution plans in `.claude/da-plans/`. Validates completenes This agent **NEVER modifies plans**. It reports issues for the author to fix. -## MCP Tools — Conserve Context - -This agent runs in a long session with a finite context window. Every Grep → Read cycle burns ~5,000 tokens on irrelevant matches. MCP tools return only what you need. - -**Before you Grep or Read, ask: can an MCP tool answer this without reading files?** - -- **`dev_map`** — Verify structure claims in the plan against actual codebase layout. -- **`dev_refs`** — Confirm dependency assertions. Use `dependsOn` to trace dependency chains between files. -- **`dev_patterns`** — Check if proposed patterns match existing conventions. - ## Two-Pass Review ### Pass 1: Engineer Review -Read the plan as a senior engineer. Use `dev_map` to verify structure claims, `dev_refs` to confirm dependency assertions, and `dev_patterns` to check if proposed patterns match existing conventions. - -1. **Context** — Does it accurately describe what exists today? (Verify with `dev_map` and reading actual code) -2. **Architecture** — Does the proposed design fit the existing monorepo structure? -3. **Parts breakdown** — Are parts sized correctly? (Each should be 1-2 commits) -4. **Dependencies** — Are cross-package dependencies identified? (Verify with `dev_refs`. Use `dependsOn` to trace dependency chains between files.) -5. **Build order** — Does the implementation order respect the build dependency chain? -6. **Breaking changes** — Are they identified and migration paths described? +Read the plan as a senior engineer. Start by gathering context with MCP tools before evaluating. + +1. Run `dev_map` to see the current codebase structure. Compare against the plan's architecture claims. +2. Run `dev_refs` on the key functions the plan modifies. Use `dependsOn` to trace dependency chains between files the plan touches. +3. Run `dev_patterns` on files the plan proposes to change. Check if the proposed code follows existing conventions. +4. Now evaluate with the context you gathered: + - **Context** — Does the plan accurately describe what exists today? + - **Architecture** — Does the proposed design fit the actual structure you saw in `dev_map`? + - **Parts breakdown** — Are parts sized correctly? (Each should be 1-2 commits) + - **Dependencies** — Do the `dev_refs` results confirm the plan's dependency claims? + - **Build order** — Does the implementation order respect the dependency chain? + - **Breaking changes** — Are they identified and migration paths described? ### Pass 2: Test Engineer Review diff --git a/.claude/agents/quality-reviewer.md b/.claude/agents/quality-reviewer.md index 8aa5748..24df872 100644 --- a/.claude/agents/quality-reviewer.md +++ b/.claude/agents/quality-reviewer.md @@ -48,9 +48,9 @@ Maximum **5 SUGGESTION items** per review. If more found, pick the top 5 and not ### Readability & Simplification -**Before you Grep or Read, ask: can an MCP tool answer this without reading files?** `dev_patterns` compares patterns across similar files (~500 tokens vs ~3,000 for manual reads). `dev_search` checks if a utility exists by meaning, not just name. +Run `dev_patterns` on changed files to find similar code and detect duplication. Run `dev_search` to check if a utility already exists before flagging missing abstractions. -- [ ] No code duplicating existing utilities — verify with `dev_patterns` and `dev_search` +- [ ] No code duplicating existing utilities (from `dev_patterns` and `dev_search` results) - [ ] Functions reasonably sized (consider splitting if >50 lines) - [ ] Complex logic has comments explaining "why", not "what" - [ ] No premature abstractions for one-time operations diff --git a/.claude/agents/quick-scout.md b/.claude/agents/quick-scout.md index 7c0d3f1..f01900b 100644 --- a/.claude/agents/quick-scout.md +++ b/.claude/agents/quick-scout.md @@ -10,12 +10,6 @@ color: blue Lightweight explorer optimized for speed and cost. Finds code, traces flows, maps dependencies. -## MCP Tools — Conserve Context - -This agent runs in a long session with a finite context window. Every Grep → Read cycle burns ~5,000 tokens on irrelevant matches. MCP tools return only what you need. - -**Before you Grep or Read, ask: can an MCP tool answer this without reading files?** - ## Capability Boundaries You excel at: diff --git a/.claude/agents/research-planner.md b/.claude/agents/research-planner.md index a7e5b48..76e9ab0 100644 --- a/.claude/agents/research-planner.md +++ b/.claude/agents/research-planner.md @@ -15,15 +15,6 @@ sub-agents for external evidence. This agent **NEVER writes code**. It produces research plans backed by evidence. -## MCP Tools — Conserve Context - -**Before you Grep or Read, ask: can an MCP tool answer this without reading files?** - -- **`dev_search`** — Find relevant code areas by meaning. Returns ranked snippets. -- **`dev_map`** — Codebase structure with hot paths and subsystems. -- **`dev_patterns`** — Compare patterns across similar files without reading each one. -- **`dev_refs`** — Trace cross-package dependencies. Use `dependsOn` to trace chains. - ## When to Use - Before starting a feature that touches unfamiliar parts of the codebase diff --git a/.claude/agents/security-reviewer.md b/.claude/agents/security-reviewer.md index 06eafbe..4697ec8 100644 --- a/.claude/agents/security-reviewer.md +++ b/.claude/agents/security-reviewer.md @@ -12,15 +12,11 @@ Security-focused review for a TypeScript monorepo that processes repository data This agent **NEVER modifies code**. It reports issues for the developer to fix. -## MCP Tools — Conserve Context +## Before the Checklist -This agent runs in a long session with a finite context window. Every Grep → Read cycle burns ~5,000 tokens on irrelevant matches. MCP tools return only what you need. - -**Before you Grep or Read, ask: can an MCP tool answer this without reading files?** - -- **`dev_search`** — Find security-sensitive code ("user input", "shell execution", "token handling"). Returns ranked snippets. -- **`dev_patterns`** — If one injection vector exists, the same pattern likely appears elsewhere. Scan for similar patterns across files. -- **`dev_refs`** — Trace how user input flows through the system. Use `dependsOn` to trace dependency chains. +1. Run `dev_search "user input"`, `dev_search "shell execution"`, `dev_search "token handling"` to find security-sensitive code in the diff area. +2. Run `dev_refs` on any function that handles user input — trace how that input flows through the system. +3. Run `dev_patterns` on files with injection vectors — if one exists, the same pattern likely appears elsewhere. ## Checklist diff --git a/.claude/da-plans/README.md b/.claude/da-plans/README.md index 7384217..b49962f 100644 --- a/.claude/da-plans/README.md +++ b/.claude/da-plans/README.md @@ -13,7 +13,8 @@ Implementation deviations are logged at the bottom of each plan file. | [Core](core/) | Phase 2 | Indexing rethink (Linear Merge, incremental) | Merged | | [Core](core/) | Phase 3 | Cached dependency graph for scale | Merged | | [Core](core/) | Phase 4 | Python language support | Merged | -| [Core](core/) | Phase 5 | Go callee extraction + Rust language support | Draft | +| [Core](core/) | Phase 5 | Go callee extraction + Rust language support | Merged | +| [Core](core/) | Phase 6 | Reverse callee index for dev_refs callers | Draft | | [MCP](mcp/) | Phase 1 | Tools improvement (patterns, consolidation, AST, graph algorithms) | Merged | | [MCP](mcp/) | Phase 2 | Composite tools — dev_review and dev_research | Draft | diff --git a/.claude/da-plans/core/phase-6-reverse-callee-index/6.1-build-and-persist.md b/.claude/da-plans/core/phase-6-reverse-callee-index/6.1-build-and-persist.md new file mode 100644 index 0000000..07dcb3c --- /dev/null +++ b/.claude/da-plans/core/phase-6-reverse-callee-index/6.1-build-and-persist.md @@ -0,0 +1,319 @@ +# Part 6.1: Build and Persist Reverse Callee Index + +See [overview.md](overview.md) for architecture context. + +## Goal + +Build a reverse callee index from scan results at index time. Persist it +inside the dependency graph artifact (v2), not as a separate file. + +## What changes + +### `packages/core/src/map/types.ts` + +Add `CallerEntry` type: + +```typescript +export interface CallerEntry { + name: string; + file: string; + line: number; + type: string; +} +``` + +### `packages/core/src/map/graph.ts` + +Update `CachedGraph` (defined here at line 26, NOT in types.ts): + +```typescript +// Update existing CachedGraph (graph.ts:26) — version 1→2 +export interface CachedGraph { + version: 1 | 2; + generatedAt: string; + nodeCount: number; + edgeCount: number; + graph: Record; + // v2 additions + reverseIndex?: Record; + reverseIndexEntryCount?: number; +} +``` + +### NEW `packages/core/src/map/reverse-index.ts` + +```typescript +import type { CalleeInfo } from '../scanner/types.js'; +import type { SearchResult } from '../vector/types.js'; +import type { CallerEntry } from './types.js'; + +/** + * Build reverse callee index from indexed documents. + * Key format: "file:name" (compound) for unique identity. + * When callee has no resolved file, key is just the name. + */ +export function buildReverseCalleeIndex( + docs: SearchResult[] +): Map { + const index = new Map(); + + for (const doc of docs) { + const callees = doc.metadata.callees as CalleeInfo[] | undefined; + if (!callees || callees.length === 0) continue; + + const callerName = typeof doc.metadata.name === 'string' + ? doc.metadata.name : 'unknown'; + const callerFile = typeof doc.metadata.path === 'string' + ? doc.metadata.path : ''; + const callerType = typeof doc.metadata.type === 'string' + ? doc.metadata.type : 'unknown'; + + for (const callee of callees) { + // Compound key when file is resolved, bare name when not + const key = callee.file + ? `${callee.file}:${callee.name}` + : callee.name; + + const entry: CallerEntry = { + name: callerName, + file: callerFile, + line: callee.line, + type: callerType, + }; + + const existing = index.get(key); + if (existing) { + existing.push(entry); + } else { + index.set(key, [entry]); + } + } + } + + return index; +} + +/** + * Build secondary name index for bare-name lookups. + * Maps the last segment of compound keys to full keys. + * "src/validate.ts:validateArgs" → lastSegment "validateArgs" + * "new CompactFormatter" → lastSegment "CompactFormatter" + * Built in memory at load time, not persisted. + */ +export function buildNameIndex( + reverseIndex: Map +): Map { + const nameIndex = new Map(); + + for (const key of reverseIndex.keys()) { + // Extract name from compound key "file:name" or bare "name" + const colonIdx = key.lastIndexOf(':'); + const name = colonIdx >= 0 ? key.slice(colonIdx + 1) : key; + // Handle "new Foo" → "Foo" + const cleaned = name.startsWith('new ') ? name.slice(4) : name; + // Also handle qualified: "this.service.search" → "search" + const dotIdx = cleaned.lastIndexOf('.'); + const lastSegment = dotIdx >= 0 ? cleaned.slice(dotIdx + 1) : cleaned; + + // Index under the full name, last segment, and class prefix + const segments = new Set([cleaned, lastSegment]); + + // For "ClassName.method", also index under "ClassName" for class aggregation + if (cleaned.includes('.')) { + const classPrefix = cleaned.split('.')[0]; + segments.add(classPrefix); + } + + for (const segment of segments) { + const existing = nameIndex.get(segment); + if (existing) { + existing.push(key); + } else { + nameIndex.set(segment, [key]); + } + } + } + + return nameIndex; +} +``` + +### `packages/core/src/map/graph.ts` + +Update serialization to include reverse index in v2: + +```typescript +export function serializeGraph( + graph: Map, + reverseIndex?: Map, + generatedAt?: string +): string { + let edgeCount = 0; + const graphObj: Record = {}; + for (const [key, edges] of graph) { + graphObj[key] = edges; + edgeCount += edges.length; + } + + let reverseObj: Record | undefined; + let reverseEntryCount: number | undefined; + if (reverseIndex) { + reverseObj = {}; + reverseEntryCount = 0; + for (const [key, entries] of reverseIndex) { + reverseObj[key] = entries; + reverseEntryCount += entries.length; + } + } + + return JSON.stringify({ + version: 2, // Always v2 once deployed — never downgrade + generatedAt: generatedAt ?? new Date().toISOString(), + nodeCount: graph.size, + edgeCount, + graph: graphObj, + reverseIndex: reverseObj, + reverseIndexEntryCount: reverseEntryCount, + } satisfies CachedGraph); +} +``` + +Update deserialization to handle v1 and v2: + +```typescript +export function deserializeGraph(json: string): { + graph: Map; + reverseIndex: Map | null; +} | null { + try { + const cached: CachedGraph = JSON.parse(json); + if (cached.version !== 1 && cached.version !== 2) return null; + + const graph = new Map(); + for (const [key, edges] of Object.entries(cached.graph)) { + graph.set(key, edges); + } + + let reverseIndex: Map | null = null; + if (cached.reverseIndex) { + reverseIndex = new Map(); + for (const [key, entries] of Object.entries(cached.reverseIndex)) { + reverseIndex.set(key, entries); + } + } + + return { graph, reverseIndex }; + } catch { + return null; + } +} +``` + +### Wire into `packages/core/src/indexer/index.ts` + +At line 191-206 where the graph is built during full index: + +```typescript +// Existing: +const graph = buildDependencyGraph(graphDocs); + +// NEW — build reverse index from same docs: +const reverseIndex = buildReverseCalleeIndex(graphDocs); + +// Updated — serialize both together: +const graphPath = getStorageFilePaths(storagePath).dependencyGraph; +await fs.writeFile(graphPath, serializeGraph(graph, reverseIndex), 'utf-8'); +``` + +Single file, single write, no drift. + +## Tests + +In `packages/core/src/map/__tests__/reverse-index.test.ts`: + +```typescript +describe('buildReverseCalleeIndex', () => { + it('should map compound keys to caller components', () => { + const docs = [ + mockDoc('src/a.ts', 'funcA', 'function', [ + { name: 'validateArgs', line: 5, file: 'src/validate.ts' }, + { name: 'console.log', line: 10 }, + ]), + mockDoc('src/b.ts', 'funcB', 'function', [ + { name: 'validateArgs', line: 3, file: 'src/validate.ts' }, + ]), + ]; + + const index = buildReverseCalleeIndex(docs); + + // Compound key for resolved file + expect(index.get('src/validate.ts:validateArgs')).toHaveLength(2); + // Bare name key for unresolved + expect(index.get('console.log')).toHaveLength(1); + }); + + it('should handle docs with no callees', () => { + const docs = [mockDoc('src/a.ts', 'MyInterface', 'interface', [])]; + const index = buildReverseCalleeIndex(docs); + expect(index.size).toBe(0); + }); +}); + +describe('buildNameIndex', () => { + it('should map last segment to compound keys', () => { + const reverseIndex = new Map([ + ['src/validate.ts:validateArgs', []], + ['src/search.ts:this.searchService.search', []], + ['new CompactFormatter', []], + ]); + + const nameIndex = buildNameIndex(reverseIndex); + + expect(nameIndex.get('validateArgs')).toContain( + 'src/validate.ts:validateArgs' + ); + expect(nameIndex.get('search')).toContain( + 'src/search.ts:this.searchService.search' + ); + expect(nameIndex.get('CompactFormatter')).toContain( + 'new CompactFormatter' + ); + }); +}); +``` + +In `packages/core/src/map/__tests__/graph.test.ts`: + +```typescript +describe('serializeGraph v2', () => { + it('should round-trip graph + reverse index', () => { + const graph = new Map([['a.ts', [{ target: 'b.ts', weight: 1 }]]]); + const reverseIndex = new Map([ + ['b.ts:funcB', [{ name: 'funcA', file: 'a.ts', line: 5, + type: 'function' }]], + ]); + + const json = serializeGraph(graph, reverseIndex); + const result = deserializeGraph(json); + + expect(result!.graph).toEqual(graph); + expect(result!.reverseIndex).toEqual(reverseIndex); + }); + + it('should deserialize v1 graph with null reverse index', () => { + const v1Json = JSON.stringify({ + version: 1, generatedAt: '', nodeCount: 0, edgeCount: 0, graph: {}, + }); + const result = deserializeGraph(v1Json); + + expect(result!.graph.size).toBe(0); + expect(result!.reverseIndex).toBeNull(); + }); +}); +``` + +## Commit + +``` +feat(core): build and persist reverse callee index in graph v2 +``` diff --git a/.claude/da-plans/core/phase-6-reverse-callee-index/6.2-lookup-callers.md b/.claude/da-plans/core/phase-6-reverse-callee-index/6.2-lookup-callers.md new file mode 100644 index 0000000..0c14e2d --- /dev/null +++ b/.claude/da-plans/core/phase-6-reverse-callee-index/6.2-lookup-callers.md @@ -0,0 +1,191 @@ +# Part 6.2: Shared Caller Lookup Functions + +See [overview.md](overview.md) for architecture context. + +## Goal + +Add shared `lookupCallers()` and `lookupClassCallers()` functions in core +that both CLI and MCP can use. Compound keys give O(1) exact match. +Name index gives O(1) bare-name resolution. + +## What changes + +### `packages/core/src/map/reverse-index.ts` + +Add lookup functions. All are pure — same inputs produce same outputs, +no side effects, no I/O, no Date.now. + +```typescript +/** + * Deduplicate caller entries by file+name, cap at limit. + * Pure helper — extracted to avoid duplication across lookup functions. + */ +function deduplicateCallers( + candidates: CallerEntry[], + limit: number +): CallerEntry[] { + const seen = new Set(); + const results: CallerEntry[] = []; + for (const entry of candidates) { + const key = `${entry.file}:${entry.name}`; + if (seen.has(key)) continue; + seen.add(key); + results.push(entry); + if (results.length >= limit) break; + } + return results; +} + +/** + * Look up callers of a target from the reverse callee index. + * + * 1. Try compound key: "targetFile:targetName" → O(1) + * 2. Fall back to nameIndex for bare-name resolution → O(1) + * 3. Deduplicate by caller file+name, cap at limit + */ +export function lookupCallers( + reverseIndex: Map, + nameIndex: Map, + targetName: string, + targetFile: string, + limit = 50 +): CallerEntry[] { + const candidates: CallerEntry[] = []; + + // 1. Compound key — exact match, O(1) + const compoundKey = `${targetFile}:${targetName}`; + const exact = reverseIndex.get(compoundKey); + if (exact) candidates.push(...exact); + + // 2. Bare name — use name index for O(1) resolution + const fullKeys = nameIndex.get(targetName) ?? []; + for (const key of fullKeys) { + if (key === compoundKey) continue; // already collected + const entries = reverseIndex.get(key); + if (entries) candidates.push(...entries); + } + + return deduplicateCallers(candidates, limit); +} + +/** + * Look up callers of a class, aggregating across constructor and methods. + * Searches for "new ClassName" and all "ClassName.method" entries. + */ +export function lookupClassCallers( + reverseIndex: Map, + nameIndex: Map, + className: string, + classFile: string, + limit = 50 +): CallerEntry[] { + const candidates: CallerEntry[] = []; + + // nameIndex indexes "ClassName.method" keys under "ClassName" prefix, + // and "new ClassName" keys under "ClassName". Single O(1) lookup + // returns all constructor + method compound keys. + const fullKeys = nameIndex.get(className) ?? []; + for (const key of fullKeys) { + const entries = reverseIndex.get(key); + if (entries) candidates.push(...entries); + } + + return deduplicateCallers(candidates, limit); +} +``` + +## Tests + +In `packages/core/src/map/__tests__/reverse-index.test.ts`: + +```typescript +describe('lookupCallers', () => { + const reverseIndex = new Map([ + ['src/validate.ts:validateArgs', [ + { name: 'SearchAdapter.execute', file: 'src/search-adapter.ts', + line: 124, type: 'method' }, + { name: 'RefsAdapter.execute', file: 'src/refs-adapter.ts', + line: 168, type: 'method' }, + ]], + ['this.searchService.search', [ + { name: 'SearchAdapter.execute', file: 'src/search-adapter.ts', + line: 141, type: 'method' }, + ]], + ]); + + const nameIndex = buildNameIndex(reverseIndex); + + it('should find callers by compound key', () => { + const callers = lookupCallers( + reverseIndex, nameIndex, 'validateArgs', 'src/validate.ts' + ); + expect(callers).toHaveLength(2); + }); + + it('should find callers by bare name via nameIndex', () => { + const callers = lookupCallers( + reverseIndex, nameIndex, 'validateArgs', 'unknown-file.ts' + ); + // Falls back to nameIndex, still finds callers + expect(callers).toHaveLength(2); + }); + + it('should deduplicate by file+name', () => { + const callers = lookupCallers( + reverseIndex, nameIndex, 'validateArgs', 'src/validate.ts' + ); + const keys = callers.map(c => `${c.file}:${c.name}`); + expect(new Set(keys).size).toBe(keys.length); + }); + + it('should respect limit', () => { + const callers = lookupCallers( + reverseIndex, nameIndex, 'validateArgs', 'src/validate.ts', 1 + ); + expect(callers).toHaveLength(1); + }); + + it('should return empty for unknown name', () => { + const callers = lookupCallers( + reverseIndex, nameIndex, 'nonexistent', 'x.ts' + ); + expect(callers).toHaveLength(0); + }); +}); + +describe('lookupClassCallers', () => { + const reverseIndex = new Map([ + ['new CompactFormatter', [ + { name: 'SearchAdapter.execute', file: 'search.ts', + line: 154, type: 'method' }, + ]], + ['search.ts:CompactFormatter.formatResults', [ + { name: 'SearchAdapter.execute', file: 'search.ts', + line: 161, type: 'method' }, + ]], + ['other.ts:CompactFormatter.estimateTokens', [ + { name: 'OtherService.run', file: 'other.ts', + line: 20, type: 'method' }, + ]], + ]); + + const nameIndex = buildNameIndex(reverseIndex); + + it('should aggregate constructor and method callers', () => { + const callers = lookupClassCallers( + reverseIndex, nameIndex, 'CompactFormatter', 'formatters.ts' + ); + // SearchAdapter.execute (deduped) + OtherService.run + expect(callers.length).toBeGreaterThanOrEqual(2); + const names = callers.map(c => c.name); + expect(names).toContain('SearchAdapter.execute'); + expect(names).toContain('OtherService.run'); + }); +}); +``` + +## Commit + +``` +feat(core): add shared lookupCallers + lookupClassCallers functions +``` diff --git a/.claude/da-plans/core/phase-6-reverse-callee-index/6.3-wire-cli-and-mcp.md b/.claude/da-plans/core/phase-6-reverse-callee-index/6.3-wire-cli-and-mcp.md new file mode 100644 index 0000000..f6290c0 --- /dev/null +++ b/.claude/da-plans/core/phase-6-reverse-callee-index/6.3-wire-cli-and-mcp.md @@ -0,0 +1,184 @@ +# Part 6.3: Wire into CLI refs + MCP refs adapter + +See [overview.md](overview.md) for architecture context. + +## Goal + +Replace the broken semantic-search caller logic in both CLI and MCP adapter +with the shared `lookupCallers`/`lookupClassCallers` functions from core. + +## What changes + +### `packages/cli/src/commands/refs.ts` + +Replace the callers block (lines 112-140): + +```typescript +// Before (broken — semantic search returns wrong candidates): +const candidates = await indexer.search(targetName, { limit: 100 }); +for (const candidate of candidates) { ... } + +// After: +import { + loadOrBuildGraph, buildReverseCalleeIndex, buildNameIndex, + lookupCallers, lookupClassCallers +} from '@prosdevlab/dev-agent-core'; + +// Load graph (already loaded for dependsOn) — now returns reverseIndex too +const { graph, reverseIndex } = await loadGraph(filePaths.dependencyGraph); + +let callers = []; +if (reverseIndex) { + const nameIndex = buildNameIndex(reverseIndex); + const targetFile = (target.metadata.path as string) || ''; + const targetType = target.metadata.type as string; + + callers = targetType === 'class' + ? lookupClassCallers(reverseIndex, targetName, targetFile, + { nameIndex, limit }) + : lookupCallers(reverseIndex, targetName, targetFile, + { nameIndex, limit }); +} else { + // v1 graph — no reverse index. Show message to re-index. + // No legacy fallback in CLI — just empty callers. +} +``` + +Note: The CLI does NOT fall back to semantic search when the reverse index +is missing. It shows empty callers. The old semantic search approach was +unreliable anyway — better to give no results than wrong results and prompt +the user to re-index. + +### `packages/mcp-server/src/adapters/built-in/refs-adapter.ts` + +Replace `getCallers` method (lines 326-365). The adapter already caches +the dependency graph with a 60s TTL. Extend this to also cache the +reverse index and name index (they come from the same file now): + +```typescript +private cachedReverseIndex: Map | null = null; +private cachedNameIndex: Map | null = null; + +// Update existing ensureGraph to also extract reverseIndex +private async ensureGraph(): Promise { + if (this.cachedGraph && Date.now() - this.graphLoadedAt < CACHE_TTL) { + return; + } + // loadOrBuildGraph now returns { graph, reverseIndex } + const result = await loadGraph(this.config.graphPath); + this.cachedGraph = result.graph; + this.cachedReverseIndex = result.reverseIndex; + this.cachedNameIndex = result.reverseIndex + ? buildNameIndex(result.reverseIndex) + : null; + this.graphLoadedAt = Date.now(); +} + +// Replace getCallers with: +private getCallersFromIndex( + target: SearchResult, + limit: number +): RefResult[] { + if (!this.cachedReverseIndex) return []; + + const targetName = target.metadata.name as string; + const targetFile = (target.metadata.path as string) || ''; + const targetType = target.metadata.type as string; + + const callers = targetType === 'class' + ? lookupClassCallers(this.cachedReverseIndex, targetName, targetFile, + { nameIndex: this.cachedNameIndex ?? undefined, limit }) + : lookupCallers(this.cachedReverseIndex, targetName, targetFile, + { nameIndex: this.cachedNameIndex ?? undefined, limit }); + + return callers.map(c => ({ + name: c.name, + file: c.file, + line: c.line, + type: c.type, + })); +} +``` + +No separate config for reverse index path — it's in the same file as the graph. + +### `packages/core/src/map/graph.ts` + +Update `loadOrBuildGraph` to return both graph and reverse index: + +```typescript +export async function loadOrBuildGraph( + graphPath: string, + fallbackDocs: () => Promise +): Promise<{ + graph: Map; + reverseIndex: Map | null; +}> { + try { + const { readFile } = await import('node:fs/promises'); + const json = await readFile(graphPath, 'utf-8'); + const result = deserializeGraph(json); + if (result) return result; + } catch { + // File doesn't exist or is corrupt — rebuild + } + const docs = await fallbackDocs(); + return { + graph: buildDependencyGraph(docs), + reverseIndex: null, // Don't rebuild on hot path — prompt re-index + }; +} +``` + +Note: when the file is missing, we return `reverseIndex: null` instead +of rebuilding from 50k docs. The caller sees empty callers and the user +is prompted to run `dev index`. + +### Breaking changes — ALL call sites for modified functions + +`deserializeGraph` return type changes. Direct callers: +- `packages/mcp-server/src/watcher/incremental-indexer.ts:137` — calls + `deserializeGraph(json)` directly. Must destructure: `const { graph } = ...` + +`loadOrBuildGraph` return type changes. Direct callers: +- `packages/core/src/map/index.ts:126` — `const graph = await loadOrBuildGraph(...)`. + Must destructure: `const { graph } = await loadOrBuildGraph(...)`. + MapService only needs the graph, discard reverseIndex. +- `packages/mcp-server/src/adapters/built-in/refs-adapter.ts` — graph caching. + Already addressed in this part. + +`serializeGraph` signature changes (adds optional reverseIndex param): +- `packages/core/src/indexer/index.ts:201` — already addressed in Part 6.1 +- `packages/mcp-server/src/watcher/incremental-indexer.ts` — addressed in Part 6.4 + +v1 graph files load fine — reverseIndex is null, callers return empty. + +## Tests + +### `packages/mcp-server/src/adapters/__tests__/refs-adapter.test.ts` + +```typescript +it('should find callers from reverse index', async () => { + // Mock graph file with v2 data including reverseIndex + // Execute refs query for "validateArgs" + // Verify callers returned from index +}); + +it('should return empty callers when v1 graph (no reverse index)', async () => { + // Mock graph file with v1 data (no reverseIndex) + // Execute refs query + // Verify empty callers, no error +}); + +it('should aggregate callers for class target', async () => { + // Mock target with type: 'class' + // Mock reverse index with constructor + method entries + // Verify both types of callers returned +}); +``` + +## Commit + +``` +feat(mcp): wire reverse callee index into CLI refs and MCP adapter +``` diff --git a/.claude/da-plans/core/phase-6-reverse-callee-index/6.4-incremental-updates.md b/.claude/da-plans/core/phase-6-reverse-callee-index/6.4-incremental-updates.md new file mode 100644 index 0000000..82a98b4 --- /dev/null +++ b/.claude/da-plans/core/phase-6-reverse-callee-index/6.4-incremental-updates.md @@ -0,0 +1,211 @@ +# Part 6.4: Incremental Reverse Index Updates + +See [overview.md](overview.md) for architecture context. + +## Goal + +Keep the reverse callee index in sync when files change. Both graph and +reverse index are updated atomically in a single write. + +## What changes + +### `packages/core/src/map/reverse-index.ts` + +Add incremental update function: + +```typescript +/** + * Incrementally update the reverse callee index. + * + * 1. Deep copy existing (don't mutate original) + * 2. Remove entries where caller file is in changedFiles or deletedFiles + * 3. Remove entries where compound key's file is in deletedFiles + * 4. Rebuild entries from changedDocs + */ +export function updateReverseIndexIncremental( + existing: Map, + changedDocs: SearchResult[], + deletedFiles: string[] +): Map { + // Deep copy — shallow Map copy shares CallerEntry[] references. + // Without this, .push() on merge mutates the original map's arrays. + const updated = new Map(); + for (const [key, entries] of existing) { + updated.set(key, [...entries]); + } + + const removedFiles = new Set(deletedFiles); + + // Collect files from changed docs + const changedFiles = new Set(); + for (const doc of changedDocs) { + const file = doc.metadata.path as string; + if (file) changedFiles.add(file); + } + + // Remove stale entries + for (const [key, entries] of updated) { + // Remove compound keys whose file is deleted + const colonIdx = key.lastIndexOf(':'); + if (colonIdx >= 0) { + const keyFile = key.slice(0, colonIdx); + if (removedFiles.has(keyFile)) { + updated.delete(key); + continue; + } + } + + // Filter out caller entries from changed/deleted files + const filtered = entries.filter(entry => + !changedFiles.has(entry.file) && !removedFiles.has(entry.file) + ); + + if (filtered.length === 0) { + updated.delete(key); + } else { + updated.set(key, filtered); + } + } + + // Rebuild entries from changed docs + const newEntries = buildReverseCalleeIndex(changedDocs); + for (const [key, entries] of newEntries) { + const existing = updated.get(key); + if (existing) { + existing.push(...entries); + } else { + updated.set(key, [...entries]); + } + } + + return updated; +} +``` + +### `packages/mcp-server/src/watcher/incremental-indexer.ts` + +Update the incremental pipeline to atomically write both indexes: + +Note: the incremental indexer calls `deserializeGraph(json)` directly +(line 137), NOT `loadOrBuildGraph`. Must update for the new return type. + +```typescript +// Current (around line 134-152): +if (graphPath) { + const json = await fs.readFile(graphPath, 'utf-8'); + const existing = deserializeGraph(json); // returns Map | null + if (existing) { + const updated = updateGraphIncremental(existing, docs, deletedFiles); + await fs.writeFile(graphPath, serializeGraph(updated)); + } +} + +// Updated — destructure new return type, atomic write of both: +if (graphPath) { + const json = await fs.readFile(graphPath, 'utf-8'); + const result = deserializeGraph(json); // returns { graph, reverseIndex } | null + if (result) { + const updatedGraph = updateGraphIncremental(result.graph, docs, deletedFiles); + const updatedReverse = result.reverseIndex + ? updateReverseIndexIncremental(result.reverseIndex, docs, deletedFiles) + : buildReverseCalleeIndex(docs); + await fs.writeFile(graphPath, serializeGraph(updatedGraph, updatedReverse)); + } +} +``` + +Single `writeFile` call — both graph and reverse index are serialized into +the same JSON file. No partial-write drift possible. + +### No new config needed + +The reverse index path was removed — it lives inside `dependency-graph.json`. +The incremental indexer config stays the same (`graphPath` covers both). + +## Tests + +In `packages/core/src/map/__tests__/reverse-index.test.ts`: + +```typescript +describe('updateReverseIndexIncremental', () => { + it('should remove entries from changed files', () => { + const existing = new Map([ + ['src/validate.ts:funcA', [ + { name: 'caller1', file: 'old.ts', line: 5, type: 'function' }, + { name: 'caller2', file: 'other.ts', line: 10, type: 'function' }, + ]], + ]); + + const changedDocs = [ + mockDoc('old.ts', 'caller1New', 'function', [ + { name: 'funcB', line: 5 }, + ]), + ]; + + const updated = updateReverseIndexIncremental(existing, changedDocs, []); + + expect(updated.get('src/validate.ts:funcA')).toHaveLength(1); + expect(updated.get('src/validate.ts:funcA')![0].name).toBe('caller2'); + expect(updated.get('funcB')).toHaveLength(1); + }); + + it('should remove entries for deleted files', () => { + const existing = new Map([ + ['src/validate.ts:funcA', [ + { name: 'caller1', file: 'deleted.ts', line: 5, type: 'function' }, + { name: 'caller2', file: 'kept.ts', line: 10, type: 'function' }, + ]], + ]); + + const updated = updateReverseIndexIncremental(existing, [], ['deleted.ts']); + + expect(updated.get('src/validate.ts:funcA')).toHaveLength(1); + expect(updated.get('src/validate.ts:funcA')![0].file).toBe('kept.ts'); + }); + + it('should remove compound keys whose file is deleted', () => { + const existing = new Map([ + ['deleted.ts:funcA', [ + { name: 'caller1', file: 'kept.ts', line: 5, type: 'function' }, + ]], + ]); + + const updated = updateReverseIndexIncremental(existing, [], ['deleted.ts']); + + expect(updated.has('deleted.ts:funcA')).toBe(false); + }); + + it('should not mutate the original map', () => { + const original = new Map([ + ['a.ts:funcA', [ + { name: 'caller1', file: 'old.ts', line: 5, type: 'function' }, + ]], + ]); + const originalArray = original.get('a.ts:funcA')!; + const originalLength = originalArray.length; + + updateReverseIndexIncremental(original, [], ['old.ts']); + + expect(original.get('a.ts:funcA')).toHaveLength(originalLength); + expect(original.get('a.ts:funcA')).toBe(originalArray); + }); + + it('should clean up empty keys', () => { + const existing = new Map([ + ['a.ts:funcA', [ + { name: 'caller1', file: 'old.ts', line: 5, type: 'function' }, + ]], + ]); + + const updated = updateReverseIndexIncremental(existing, [], ['old.ts']); + + expect(updated.has('a.ts:funcA')).toBe(false); + }); +}); +``` + +## Commit + +``` +feat(mcp): atomic incremental update for graph + reverse index +``` diff --git a/.claude/da-plans/core/phase-6-reverse-callee-index/6.5-verification.md b/.claude/da-plans/core/phase-6-reverse-callee-index/6.5-verification.md new file mode 100644 index 0000000..ee99263 --- /dev/null +++ b/.claude/da-plans/core/phase-6-reverse-callee-index/6.5-verification.md @@ -0,0 +1,84 @@ +# Part 6.5: End-to-End Verification + +See [overview.md](overview.md) for architecture context. + +## Goal + +Verify the full feature works end-to-end. + +## Verification steps + +### 1. Full index writes v2 graph + +```bash +dev index +cat ~/.dev-agent/indexes/*/dependency-graph.json | \ + node -e "const d=JSON.parse(require('fs').readFileSync('/dev/stdin','utf8')); \ + console.log('version:', d.version, 'entries:', d.reverseIndexEntryCount)" +# Should show: version: 2 entries: +``` + +### 2. CLI refs finds callers + +```bash +# validateArgs is called by all 5 adapters +dev refs "validateArgs" +# Should show callers: SearchAdapter.execute, RefsAdapter.execute, etc. + +# CompactFormatter is instantiated +dev refs "CompactFormatter" +# Should show callers of "new CompactFormatter" + +# Class-level aggregation +dev refs "SearchAdapter" +# Should aggregate callers across constructor + methods + +# Function with resolved file +dev refs "startFileWatcher" +# Should show callers from dev-agent-mcp.ts +``` + +### 3. MCP refs finds callers (after MCP server restart) + +``` +dev_refs({ name: "validateArgs" }) → callers listed +dev_refs({ name: "SearchAdapter" }) → class callers aggregated +dev_refs({ name: "execute" }) → disambiguated by target file +``` + +### 4. Backward compatibility + +```bash +# Simulate v1 graph (no reverse index) +node -e "const fs=require('fs'); const g=JSON.parse(fs.readFileSync( + '~/.dev-agent/indexes/*/dependency-graph.json','utf8')); + delete g.reverseIndex; g.version=1; + fs.writeFileSync('/tmp/v1-graph.json', JSON.stringify(g))" + +# Copy to index dir, run refs — should show empty callers, no crash +dev refs "validateArgs" +# Callers section shows "No callers found" — not an error + +# Re-index to restore +dev index +``` + +### 5. Incremental updates + +```bash +# Edit a file that calls validateArgs, save +# Wait for watcher debounce (500ms) +# Check dependency-graph.json was updated (mtime + version 2 + reverseIndex) +# Run dev refs "validateArgs" — should reflect changes +``` + +### 6. All tests pass + +```bash +pnpm build && pnpm test && pnpm typecheck +``` + +## Commit + +No code in this part — just manual verification and any test gaps +discovered during e2e testing. diff --git a/.claude/da-plans/core/phase-6-reverse-callee-index/overview.md b/.claude/da-plans/core/phase-6-reverse-callee-index/overview.md new file mode 100644 index 0000000..01f5308 --- /dev/null +++ b/.claude/da-plans/core/phase-6-reverse-callee-index/overview.md @@ -0,0 +1,226 @@ +# Phase 6: Reverse Callee Index for dev_refs + +## Status: Draft (revised after 4 review passes) + +## Context + +`dev_refs` can find callees (what a function calls) but **callers are broken**. +Both CLI and MCP adapter find callers by semantic-searching the target name, +then scanning each candidate's callees list. Semantic search returns +*similar concepts*, not *call sites* — so `validateArgs` returns other +validators, not the 5 adapters that call it. The search is also capped at +100 candidates, meaning callers outside that window are invisible. + +### What exists + +- Each indexed component stores `callees: CalleeInfo[]` with name, file, line +- Dependency graph (`dependency-graph.json`) tracks file→file edges only +- Callee names vary by language: `this.searchService.search` (TS), + `fmt.Println` (Go), `self.validate` (Python), `Vec::new` (Rust) +- Incremental indexer updates graph on file changes via `updateGraphIncremental` +- CLI (`refs.ts:112-140`) and MCP (`refs-adapter.ts:326-365`) duplicate + the same broken caller logic + +### What we're building + +A **reverse callee index** — a map from compound callee key (`file:name`) +to the components that call it. Built at index time, persisted inside the +dependency graph artifact (v2), updated atomically with the graph, used by +both CLI and MCP via a shared lookup function in core. + +## Parts + +| Part | Description | Risk | +|------|-------------|------| +| 6.1 | Build + persist reverse index in core | Low | +| 6.2 | Shared caller lookup function in core | Low | +| 6.3 | Wire into CLI refs + MCP refs adapter | Medium | +| 6.4 | Incremental updates | Medium | +| 6.5 | Tests + verification | Low | + +## Architecture + +``` +dev index + │ + ▼ +scanRepository() → EmbeddingDocument[] (with callees metadata) + │ + ▼ +buildIndexes(docs) + ├─► buildDependencyGraph() → graph (file→file edges) + └─► buildReverseCalleeIndex() → reverseIndex (file:name → callers) + │ + ▼ +serializeCachedGraph(graph, reverseIndex) → dependency-graph.json v2 + (single artifact, atomic write, shared lifecycle) + +dev refs "validateArgs" + │ + ├─► findBestMatch() → target component (via search, unchanged) + ├─► callees → from target.metadata.callees (unchanged) + └─► callers → lookupCallers(reverseIndex, target) + Compound key: O(1) exact match + Bare name: O(1) via name→keys secondary map +``` + +## Key Design Decisions + +| Decision | Choice | Rationale | Alternatives | +|----------|--------|-----------|-------------| +| Index key format | Compound `file:name` when file resolved, bare name otherwise | Unique identity per callee for TS (ts-morph resolves files). Tree-sitter languages (Go, Rust, Python) don't resolve callee files — degrades to bare name keys with name-index fallback | Bare name only (collisions), full symbol ID (not available) | +| Bare-name lookup | Secondary `Map` built in memory at load | Agent queries `"search"` not `"search-service.ts:search"`. Secondary map resolves bare→compound in O(1) | endsWith scan O(n) — slower, same result | +| Storage | Inside `dependency-graph.json` v2, not a separate file | Same source data, same lifecycle, atomic write. Prevents drift between graph and reverse index | Separate file (two caches, one truth — will drift) | +| Graph version | Bump CachedGraph version 1→2. v2 adds `reverseIndex` field. Deserializer handles v1 (no reverse index) gracefully | Backward compatible — v1 files load fine, just no callers | New file format (drift risk) | +| Module location | New `packages/core/src/map/reverse-index.ts` | graph.ts is already ~350 lines. Reverse index is conceptually distinct | graph.ts (God module) | +| Missing index fallback | Return empty callers + log warning to re-index | No expensive rebuild on hot path. Old repos just show "no callers" until re-indexed | Rebuild from 50k docs on every call (multi-second blocking) | +| Dedup granularity | By caller file+name, not by call site line | Agents want "who calls this" not "every call site" | Per call site (noisier) | +| Class-level queries | `lookupClassCallers()` helper in core | Aggregates callers of `new ClassName` + all `ClassName.*` methods. In core, not duplicated in callers | Push to adapter (duplication) | +| CLI/MCP duplication | Consolidate into shared functions in core | Single implementation, tested once | Keep duplicated (drift risk) | +| Incremental updates | Atomic update of both graph + reverse index in single write | Mirrors `updateGraphIncremental` pattern, prevents partial-write drift | Independent writes (drift risk) | + +## Data Structures + +### CallerEntry (stored in reverse index) + +```typescript +interface CallerEntry { + name: string; // Caller component name ("SearchAdapter.execute") + file: string; // Caller file path + line: number; // Call site line in caller + type: string; // Caller component type (function, method, class) +} +``` + +No `calleeFile` needed — disambiguation is built into the compound key. + +### CachedGraph v2 (updated serialization envelope) + +```typescript +interface CachedGraph { + version: 2; // bumped from 1 + generatedAt: string; + nodeCount: number; + edgeCount: number; + graph: Record; + reverseIndex?: Record; // NEW — compound key + reverseIndexEntryCount?: number; // NEW +} +``` + +v1 files (no `reverseIndex`) deserialize fine — callers just return empty. + +### lookupCallers (shared function) + +```typescript +function lookupCallers( + reverseIndex: Map, + targetName: string, + targetFile: string, + options?: { limit?: number; nameIndex?: Map } +): CallerEntry[] +``` + +Matching logic: +1. Try compound key: `reverseIndex.get("${targetFile}:${targetName}")` +2. If no compound match, use nameIndex for bare-name lookup +3. Deduplicate by caller file+name, cap at limit + +### lookupClassCallers (class aggregation) + +```typescript +function lookupClassCallers( + reverseIndex: Map, + className: string, + classFile: string, + options?: { limit?: number; nameIndex?: Map } +): CallerEntry[] +``` + +Aggregates callers of `new ClassName` + all `ClassName.*` methods found +in the name index. + +### buildNameIndex (secondary bare-name map) + +```typescript +function buildNameIndex( + reverseIndex: Map +): Map +``` + +Maps last segment of compound key to full keys: +`"search"` → `["search-service.ts:search", "indexer.ts:search"]` + +Built in memory at load time. Not persisted. + +## Risk Register + +| Risk | Likelihood | Impact | Mitigation | +|------|-----------|--------|------------| +| Large repos produce huge index | Medium | Low | JSON fine up to 250k entries. Single file with graph keeps overhead contained | +| Stale entries after incremental update | Medium | Medium | Remove all entries by caller file before rebuilding; test with add/edit/delete | +| Dangling edges to deleted files | Medium | Medium | Prune entries where compound key's file is in deletedFiles | +| File path format mismatch | Medium | Medium | Invariant: all paths relative to repo root. Normalize at build time | +| v1→v2 graph migration | Low | Low | Deserializer handles v1 gracefully (no reverse index). `dev index` rebuilds v2 | +| Concurrent incremental writes | Low | Medium | Watcher serializes flushes via promise chain. Single atomic write for both | + +## Test Strategy + +| Test | Priority | File | +|------|----------|------| +| `buildReverseCalleeIndex` from mock docs | P0 | `reverse-index.test.ts` | +| Serialization round-trip (v2 graph with reverse index) | P0 | `graph.test.ts` | +| Deserialize v1 graph (no reverse index) → empty callers | P0 | `graph.test.ts` | +| `lookupCallers` compound key exact match | P0 | `reverse-index.test.ts` | +| `lookupCallers` bare-name via nameIndex | P0 | `reverse-index.test.ts` | +| `lookupClassCallers` aggregation | P1 | `reverse-index.test.ts` | +| `buildNameIndex` mapping | P0 | `reverse-index.test.ts` | +| Incremental update: add file | P0 | `reverse-index.test.ts` | +| Incremental update: delete file | P0 | `reverse-index.test.ts` | +| Incremental update: change file (rename callees) | P0 | `reverse-index.test.ts` | +| Incremental update does not mutate original map | P1 | `reverse-index.test.ts` | +| MCP refs adapter uses lookupCallers | P1 | `refs-adapter.test.ts` | +| CLI refs uses lookupCallers | P1 | manual verification | +| Empty callers when reverse index missing (v1 graph) | P1 | `refs-adapter.test.ts` | +| dev_refs MCP tool returns callers for validateArgs | P0 | manual e2e | + +## Verification Checklist + +- [ ] `dev index` writes dependency-graph.json with version 2 + reverseIndex +- [ ] `dev refs "validateArgs"` shows callers (SearchAdapter, RefsAdapter, etc.) +- [ ] `dev refs "CompactFormatter"` shows callers of `new CompactFormatter` +- [ ] `dev refs "SearchAdapter"` aggregates callers across class methods +- [ ] MCP `dev_refs` returns callers (restart MCP server, test via tool call) +- [ ] File edit triggers atomic incremental update of graph + reverse index +- [ ] File delete removes stale caller entries +- [ ] Old v1 graph files load fine — callers return empty, no error +- [ ] `pnpm test` passes, `pnpm typecheck` passes + +## Files Modified + +| File | Change | +|------|--------| +| `packages/core/src/map/reverse-index.ts` | NEW — build, lookup, lookupClass, nameIndex, incremental update | +| `packages/core/src/map/types.ts` | Add CallerEntry type | +| `packages/core/src/map/graph.ts` | Update CachedGraph to v2, serialize/deserialize for v2 | +| `packages/core/src/map/index.ts` | Export reverse-index functions, destructure loadOrBuildGraph in MapService | +| `packages/core/src/indexer/index.ts` | Wire reverse index build alongside dependency graph (line 191) | +| `packages/cli/src/commands/refs.ts` | Replace caller logic with lookupCallers from core | +| `packages/mcp-server/src/adapters/built-in/refs-adapter.ts` | Replace getCallers with lookupCallers from core | +| `packages/mcp-server/src/watcher/incremental-indexer.ts` | Atomic update of graph + reverse index together | +| `packages/mcp-server/bin/dev-agent-mcp.ts` | No new config — reverse index comes from graph | +| `packages/core/src/map/__tests__/reverse-index.test.ts` | NEW — tests for all reverse index functions | +| `packages/core/src/map/__tests__/graph.test.ts` | Update for v2 serialization | +| `packages/mcp-server/src/adapters/__tests__/refs-adapter.test.ts` | Update caller tests | + +## References + +- Current callers logic (CLI): `packages/cli/src/commands/refs.ts:112-140` +- Current callers logic (MCP): `packages/mcp-server/src/adapters/built-in/refs-adapter.ts:326-365` +- Dependency graph builder: `packages/core/src/map/graph.ts:46-78` +- Graph serialization: `packages/core/src/map/graph.ts:280-340` +- Incremental graph update: `packages/core/src/map/graph.ts:353-372` +- Incremental indexer: `packages/mcp-server/src/watcher/incremental-indexer.ts` +- Storage paths: `packages/core/src/storage/path.ts:118` +- CalleeInfo type: `packages/core/src/scanner/types.ts:30-37` +- PR #34: fix(mcp) drop search scores — fixed scoreThreshold blocking refs