Skip to content

[FIX] Rectify: Filesystem Snapshot Modification Blind Spot#3336

Merged
Trecek merged 5 commits into
developfrom
fs-writes-detected-blind-spot-snapshot-watches-skill-specifi/3309
May 30, 2026
Merged

[FIX] Rectify: Filesystem Snapshot Modification Blind Spot#3336
Trecek merged 5 commits into
developfrom
fs-writes-detected-blind-spot-snapshot-watches-skill-specifi/3309

Conversation

@Trecek
Copy link
Copy Markdown
Collaborator

@Trecek Trecek commented May 30, 2026

Summary

_recursive_snapshot() returns set[str] (path names only), making the post-session set-difference comparison (_post - _pre) structurally blind to in-place file modifications. This is a type-level defect: the snapshot's return type cannot represent file state, only file presence. The fix upgrades the snapshot to capture (mtime_ns, size) per file — a stat-based comparison that detects creation, modification, deletion, and truncation. The approach mirrors the proven _LoadCacheEntry pattern already in production at recipe/_api_cache.py.

Closes #3309

Implementation Plan

Plan file: /home/talon/projects/autoskillit-runs/remediation-20260530-130303-981480/.autoskillit/temp/rectify/rectify_fs_writes_snapshot_blind_spot_2026-05-30_131500.md

🤖 Generated with Claude Code via AutoSkillit

Token Usage Summary

Step Model count uncached output cache_read peak_ctx turns cache_write time
rectify* opus[1m] 1 2.6k 17.7k 1.9M 108.7k 241 99.8k 15m 36s
review_approach* sonnet 1 62 6.9k 170.4k 51.8k 81 110.0k 5m 18s
dry_walkthrough* opus 1 50 9.3k 675.4k 62.4k 81 46.4k 4m 47s
implement* sonnet 1 286 24.2k 2.2M 81.7k 106 65.9k 7m 0s
audit_impl* sonnet 1 627 11.9k 354.5k 50.4k 32 36.0k 4m 4s
prepare_pr* sonnet 1 135.7k 3.1k 337.2k 30.9k 23 15.8k 1m 14s
compose_pr* sonnet 1 38.3k 1.2k 182.6k 30.9k 14 15.5k 36s
review_pr* sonnet 3 609 135.2k 3.6M 99.1k 190 294.4k 32m 27s
resolve_review* opus 3 120 22.9k 4.2M 89.0k 142 188.3k 27m 3s
Total 178.3k 232.4k 13.5M 108.7k 872.2k 1h 38m

* Step used a non-Anthropic provider; caching behavior may differ.

Token Efficiency

Step LoC Changed cache_read/LoC cache_write/LoC output/LoC
rectify 0
review_approach 0
dry_walkthrough 0
implement 194 11127.7 339.7 124.5
audit_impl 0
prepare_pr 0
compose_pr 0
review_pr 0
resolve_review 28 149026.4 6726.5 816.5
Total 222 60955.8 3928.9 1046.7

Model Usage Breakdown

Model steps uncached output cache_read cache_write time
opus[1m] 1 2.6k 17.7k 1.9M 99.8k 15m 36s
sonnet 6 175.6k 182.5k 6.8M 537.7k 50m 41s
opus 2 170 32.2k 4.8M 234.7k 31m 51s

@Trecek Trecek force-pushed the fs-writes-detected-blind-spot-snapshot-watches-skill-specifi/3309 branch from a5477a2 to 9fedb5c Compare May 30, 2026 21:17
Copy link
Copy Markdown
Collaborator Author

@Trecek Trecek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: approved_with_comments

Comment thread tests/execution/test_write_evidence.py
Comment thread src/autoskillit/execution/headless/_headless_execute.py
Comment thread src/autoskillit/execution/headless/_headless_helpers.py
Copy link
Copy Markdown
Collaborator Author

@Trecek Trecek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit review: warning-only findings detected. See inline comments — no blocking changes required.

⚠️ Outside Diff Range

These findings target lines not in the diff and could not be posted as inline comments:

tests/execution/test_write_evidence.py

  • L741 [warning/tests]: test_snapshot_empty_when_no_new_files asserts only that no new keys appear (post.keys() - pre.keys() == set()), but _stat_snapshot now also captures mtime+size. An existing file modified between pre and post would pass the key-diff assertion while the production comparison (_post != _pre) would detect it. Assert post == pre to match production semantics.

Trecek and others added 5 commits May 30, 2026 14:55
Adds tests proving _recursive_snapshot misses in-place modifications,
plus TestStatSnapshot and TestSnapshotTypeContract for the _stat_snapshot
fix target. Integration test covers run_headless_core modification path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_recursive_snapshot returned set[str] (path names only), making it
structurally blind to in-place file modifications. _stat_snapshot returns
dict[str, tuple[int, int]] (mtime_ns, size per file), so the _post != _pre
dict comparison detects creation, modification, and deletion.

Updates _headless_execute.py to use the new function and dict fallbacks.
Updates __init__.py re-export and all test references accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…hes, weak assertion

- Use None sentinel for failed scans instead of {} to distinguish
  scan failure from empty directory; continue on post-scan OSError
  to avoid false-positive _fs_writes_detected
- Rename TestRecursiveSnapshot to TestSubdirSnapshot and update docstring
- Remove dead branches in test_snapshot_return_type_is_dict_not_set
  (from __future__ annotations makes sig.return_annotation always str)
- Strengthen test_snapshot_empty_when_no_new_files to assert post == pre
  (detects modifications, not just new file creation)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Combine the None guard and inequality check into one condition to stay
within the 595-line budget for _headless_execute.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- _stat_snapshot: log skipped paths at debug level instead of silently
  swallowing OSError, improving diagnostic visibility
- test_stat_snapshot_detects_modified_file: remove time.sleep(0.01) that
  implied mtime reliance when detection actually relies on size difference

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Trecek Trecek force-pushed the fs-writes-detected-blind-spot-snapshot-watches-skill-specifi/3309 branch from b78f91d to 809841d Compare May 30, 2026 21:56
@Trecek Trecek added this pull request to the merge queue May 30, 2026
Merged via the queue into develop with commit 9d71516 May 30, 2026
3 checks passed
@Trecek Trecek deleted the fs-writes-detected-blind-spot-snapshot-watches-skill-specifi/3309 branch May 30, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant