Skip to content

fix(retro): warn on stale date windows#1636

Open
jbetala7 wants to merge 1 commit into
garrytan:mainfrom
jbetala7:oss/fix-1624-retro-stale-window
Open

fix(retro): warn on stale date windows#1636
jbetala7 wants to merge 1 commit into
garrytan:mainfrom
jbetala7:oss/fix-1624-retro-stale-window

Conversation

@jbetala7
Copy link
Copy Markdown
Contributor

@jbetala7 jbetala7 commented May 21, 2026

Fixes #1624

Summary

  • Move the /retro date/window guard into a dedicated pre-flight before normal data gathering.
  • Use a window-aware stale-base threshold: warn when the latest default-branch commit gap is greater than the current window length (gap > window_days), not a fixed 3 days.
  • Print the system date, current window length, latest origin default-branch commit date/gap, and commit count before narrative generation.
  • Apply the same pre-flight to /retro compare and to each discovered repo in /retro global; global mode surfaces affected repos before aggregate totals.
  • Warn for mismatched today anchors and empty windows so /retro does not produce a clean-looking empty report from bad date/window evidence.

Tests

  • bun run gen:skill-docs --host all
  • bun test test/gen-skill-docs.test.ts -t 'retro SKILL.md warns on stale base or empty windows'
  • bun run scripts/gen-skill-docs.ts --dry-run --host claude
  • git diff --check
  • git diff --check origin/main...HEAD
  • git merge-tree --write-tree HEAD origin/main
  • Temporary 9-day-gap/7d proof: latest origin/main commit dated 2026-05-12, system date 2026-05-21, RETRO_WINDOW_DAYS: 7, RETRO_WINDOW_COMMIT_COUNT: 0, emitted both STALE-BASE WARNING and EMPTY-WINDOW WARNING.

Collision checks

  • Open PR search for 1624 retro stale window returned only this PR.
  • Open PR search for STALE-BASE WARNING returned only this PR.
  • Open PR searches for RETRO_WINDOW_DAYS and retro global empty window found no competing PR.
  • Fresh origin/main merge-tree completed cleanly.

@poppinpixels
Copy link
Copy Markdown

Issue author. Four questions from #1624 I don’t see resolved in this PR:

@xg-gh-25
Copy link
Copy Markdown

Smart catch on the stale-window issue — this is exactly the kind of silent failure that undermines trust in retro tools.

Why this matters:
When /retro runs with an outdated anchor date, it produces plausible-looking but meaningless output. Users assume the tool is working, waste time analyzing fake trends, and only realize the issue days later when retrospectives don't align with reality.

Implementation suggestion:
Your warning approach is solid. I'd recommend:

  1. Threshold: Flag if today - anchor_date > 48 hours (aggressive but prevents most drift)
  2. Visibility: Surface the warning above the output, not just in logs — make it impossible to miss
  3. Auto-correction option: Offer a quick action: "Anchor is stale. Re-run with today's date?" (reduces friction)

The tricky part is handling legitimate use cases where someone intentionally runs historical retros (e.g., auditing last month). Maybe:

  • Default: auto-warn
  • Escape hatch: --allow-stale-anchor flag for intentional historical runs

Have you considered caching the last anchor date per workspace to detect drift automatically?


Posted by SwarmAI — an agentic community engagement system

@jbetala7 jbetala7 force-pushed the oss/fix-1624-retro-stale-window branch from f342979 to 79e4724 Compare May 21, 2026 10:28
@jbetala7
Copy link
Copy Markdown
Contributor Author

Addressed in 79e4724d.

  • Threshold: changed from fixed > 3d to window-aware gap > window_days. The pre-flight now resolves default 7d, Nd, Nw * 7, and ceil(Nh / 24) with a minimum of 1 day, then emits RETRO_WINDOW_DAYS.
  • Scope: yes, this now explicitly covers normal /retro, /retro compare current-window metrics, and /retro global per discovered repo. Global mode runs the same pre-flight after repo/default-branch detection and lists affected repos before aggregate totals.
  • Placement: moved to a dedicated ### Date/window pre-flight section before Step 1, rather than leaving it inline inside raw data gathering.
  • Reproducer: tested the 9-day-gap/7d case from /retro silently produces empty/misleading output when 'today' anchor is wrong (no stale-base warning) #1624 with a temporary git repo, not only a synthetic empty-window. The proof had system date 2026-05-21, latest origin/main commit 2026-05-12, RETRO_WINDOW_DAYS: 7, RETRO_WINDOW_COMMIT_COUNT: 0, and emitted both STALE-BASE WARNING and EMPTY-WINDOW WARNING.

Verification also passed: bun run gen:skill-docs --host all, focused generated-doc regression, Claude dry-run freshness, git diff --check, and merge-tree against fresh origin/main. I updated the PR body with the exact proof and collision searches.

@poppinpixels
Copy link
Copy Markdown

Good turnaround. Brief read on each:

  1. Window-aware gap > window_days matches what my local patch does. Better than fixed 3d.
  2. Scope extension to compare and global per discovered repo closes the gap I flagged.
  3. Pre-flight section reads cleaner than the inline option.
  4. The 9-day case with system date 2026-05-21, commit 2026-05-12, both warnings emitted is exactly the reproducer.

Will verify against my local repo and reproducer.

The three failing E2E Eval checks (e2e-review, llm-judge, report) — related to this change, or pre-existing flakes?

@jbetala7
Copy link
Copy Markdown
Contributor Author

Not related to this change from what I can see.

  • llm-judge: the job starts with Secret source: None, then the Anthropic SDK fails before the judged eval can run because no API auth is available. That is an external PR/secrets issue, not branch logic.
  • e2e-review: the failing case is /retro default branch detection, but the eval comparison reports FAIL -> FAIL, Status: 3 unchanged, and No regressions. So it is the existing baseline failure, not a new regression from this branch.
  • report: it downloads the eval artifacts and builds the PR comment body, then fails only while posting the report comment with GraphQL: Resource not accessible by integration (addComment). That is the external PR token permission issue.

The branch-specific checks remain green: actionlint, build-image, check-freshness, windows-free-tests, and the rest of the E2E suites all passed. The local verification for this change also passed as noted above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

/retro silently produces empty/misleading output when 'today' anchor is wrong (no stale-base warning)

3 participants