Skip to content

feat: Comparator Phase 2 — persistence, diffs, evaluation, history#22

Merged
siracusa5 merged 3 commits intomainfrom
feat/comparator-phase-2
Mar 16, 2026
Merged

feat: Comparator Phase 2 — persistence, diffs, evaluation, history#22
siracusa5 merged 3 commits intomainfrom
feat/comparator-phase-2

Conversation

@siracusa5
Copy link
Collaborator

Summary

Adds everything needed to make Comparator comparisons useful beyond a single run: SQLite persistence, git worktree isolation, diff view, manual evaluation with radar charts, JSON export, comparison replay, history browser, and analytics dashboard.

Changes

  • SQLite persistence (rusqlite bundled) — comparisons, panels, evaluations, and file diffs stored at ~/.harness-kit/comparator.db with WAL mode
  • Git worktree isolation — each panel runs in its own worktree pinned to a commit; diffs computed post-completion against the base commit
  • Diff view — side-by-side file listing per panel with unified diff rendering (green/red line coloring)
  • Evaluation panel — 10-dimension rubric (correctness, completeness, code quality, efficiency, reasoning, speed, safety, context awareness, autonomy, adherence) with sliders, numeric input, and Recharts radar chart overlay
  • Export — full JSON export of comparison data (metadata + panels + evaluations + diffs), download or copy to clipboard
  • Replay — "Re-run" button pre-fills setup page from a saved comparison
  • History page — list saved comparisons, click to review, delete
  • Analytics page — win rate bar chart, dimension comparison chart, stat cards (requires 2+ evaluated comparisons)
  • Review mode/comparator/review/:id loads from DB in read-only tabs
  • Navigation — History and Analytics sub-items under Comparator sidebar

New files (11)

File Purpose
src-tauri/src/db.rs SQLite init + schema
src-tauri/src/commands/comparator_db.rs 8 CRUD commands
src-tauri/src/commands/git.rs 4 git worktree commands
src-tauri/src/commands/evaluation.rs 3 evaluation commands
src-tauri/src/commands/export.rs JSON export + analytics
src/components/comparator/DiffView.tsx Diff viewer
src/components/comparator/EvaluationPanel.tsx 10-dimension scoring
src/components/comparator/ScoreRadar.tsx Radar chart
src/components/comparator/ExportMenu.tsx Export dropdown
src/pages/comparator/ComparatorHistoryPage.tsx History list
src/pages/comparator/ComparatorAnalyticsPage.tsx Analytics dashboard

Test Plan

  • cargo check — compiles with 0 warnings
  • vitest run — 152 tests pass
  • CI checks pass
  • Manual: Setup → Run → panels complete → Output/Diffs/Evaluate tabs
  • Manual: Save → History → Review → Re-run → setup pre-filled
  • Manual: Export JSON → valid JSON with all data
  • Manual: Analytics after 2+ evaluated comparisons

Notes

  • LLM-as-judge deferred — "Score with AI" button is present but disabled with "Coming soon" tooltip
  • HTML report export deferred — JSON only for Phase 2
  • pinnedCommit field accepted by Rust backend but worktree creation/cleanup not yet wired into the start_comparison command handler (frontend calls createWorktrees / removeWorktrees directly) — planned for Phase 3

🤖 Generated with Claude Code

@siracusa5 siracusa5 added the enhancement New feature or request label Mar 16, 2026
siracusa5 and others added 3 commits March 15, 2026 23:45
…luation, history

Add SQLite persistence (rusqlite) for comparisons, panels, evaluations,
and file diffs. Add git worktree isolation so each tool runs in its own
checkout pinned to a commit. Add post-completion diff view with
side-by-side file listing and unified diff rendering. Add 10-dimension
manual evaluation panel with radar chart (Recharts). Add JSON export,
comparison replay, history browser, and analytics dashboard with win
rates and dimension averages.

New Rust modules: db.rs, comparator_db.rs, git.rs, evaluation.rs,
export.rs. New frontend: DiffView, EvaluationPanel, ScoreRadar,
ExportMenu, ComparatorHistoryPage, ComparatorAnalyticsPage. Updated
run page with Output/Diffs/Evaluate tabs and review mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lResult

- Create git worktrees before spawning panels when pinnedCommit is set
- Pass per-panel workingDir to Rust start_comparison for cwd isolation
- Collect diffs via getDiffAgainstCommit after all panels complete
- Persist diffs to DB and load into DiffView tab
- Clean up worktrees after diff collection
- Fix savePanelResult firing on every render by tracking saved panels
- Remove invalid capabilities (fs:write-scope, security command)
- Remove leaked security module references from mod.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…olish

- Fix N+1 query in list_comparisons: single batch load + HashMap grouping
- Wrap save_file_diffs in BEGIN/COMMIT transaction for atomicity
- Consolidate duplicated Rust types into shared commands/types.rs
  (FileDiffEntry, Evaluation used by git.rs, evaluation.rs, comparator_db.rs)
- Add reusable CSS classes (badge, btn, tab, harness-card, form, eval-table)
- Replace inline styles across all comparator components with CSS classes
- Fix version text overflow in HarnessSelector with text-overflow: ellipsis
- Switch harness cards from flex-wrap to CSS grid for predictable sizing
- Normalize monospace font stack in DiffView to match comparator-output

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@siracusa5 siracusa5 force-pushed the feat/comparator-phase-2 branch from 0adef9b to 986ea62 Compare March 16, 2026 03:45
@siracusa5 siracusa5 merged commit 40ad584 into main Mar 16, 2026
3 checks passed
@siracusa5 siracusa5 deleted the feat/comparator-phase-2 branch March 16, 2026 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant