diff --git a/.claude/settings.json b/.claude/settings.json index 4aea0ed..e79389b 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -13,6 +13,10 @@ "Bash(cargo clippy:*)", "Bash(cargo fmt:*)", "Bash(cargo fetch:*)", + "Bash(git rev-parse:*)", + "Bash(git ls-files:*)", + "Bash(git merge-base:*)", + "Bash(git remote:*)", "mcp__github__pull_request_read", "mcp__github__issue_read", "mcp__github__list_pull_requests" diff --git a/CHANGELOG.md b/CHANGELOG.md index b59e27f..e0996e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,7 @@ this project aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- feat(sidecar): cross-process advisory write locking + atomic-rename durability for the JSON sidecar, and wire the `provenance`/`history` CLI subcommands against sqlite + json sidecars (V-L2-F4, ADR-0013, closes #150) (#151) - feat(sidecar): JSON-family sidecar storage backend — plain JSON / JSON-LD / NDJSON with SQLite-parity octad runtime (provenance incl. forks, temporal, drift, gc); new `[sidecar].format` key and a single `StorageKind::resolve` backend resolver (V-L2-F3, ADR-0012, closes #146) (#148) - feat(codegen): split sidecar DDL by dialect; reject json sidecar (#45) (#133) - feat(codegen): split sidecar DDL by dialect; reject json sidecar (#45) (#131) diff --git a/docs/decisions/0012-json-family-sidecar-storage.adoc b/docs/decisions/0012-json-family-sidecar-storage.adoc index 5e69816..99aaec7 100644 --- a/docs/decisions/0012-json-family-sidecar-storage.adoc +++ b/docs/decisions/0012-json-family-sidecar-storage.adoc @@ -57,8 +57,10 @@ relational invariants in application code with weaker guarantees* — is answered rather than simply contradicted: #148 introduces the abstraction (`StorageKind` + the `SidecarData` codec) and re-encodes those invariants in Rust with documented parity. The one acknowledged difference is concurrency: -the JSON store is single-writer (atomic whole-file rewrite) where SQLite -serialises via `BEGIN IMMEDIATE` locks. The guarantees SQL enforced +the JSON store serialises writers itself — single-writer-by-assumption as of +F3, and since V-L2-F4 (#151, ADR-0013) via a cross-process advisory lock held +across the whole load→mutate→save cycle — where SQLite serialises via +`BEGIN IMMEDIATE` locks. The guarantees SQL enforced structurally (one current temporal row, monotonic versions, fork-aware chains, hash identity) are now enforced in code for the json path and covered by tests mirroring the SQLite suite, across all three formats. @@ -75,8 +77,9 @@ covered by tests mirroring the SQLite suite, across all three formats. ### Negative - The json path enforces invariants in application code rather than via SQL - constraints, so it carries a heavier test burden and a documented - single-writer limitation. + constraints, so it carries a heavier test burden. (The single-writer + constraint noted here was subsequently hardened into a cross-process + advisory lock by V-L2-F4 — see ADR-0013.) - Two storage families to maintain in lockstep (the hash function is shared via `abi`, which mitigates the highest-risk drift). diff --git a/docs/decisions/0013-json-sidecar-cross-process-locking.adoc b/docs/decisions/0013-json-sidecar-cross-process-locking.adoc new file mode 100644 index 0000000..0e8e18a --- /dev/null +++ b/docs/decisions/0013-json-sidecar-cross-process-locking.adoc @@ -0,0 +1,75 @@ += Architecture Decision Record: 0013-json-sidecar-cross-process-locking + + + +# 13. Cross-process locking + atomic writes for the JSON sidecar + +Date: 2026-05-31 + +## Status + +Accepted (2026-05-31) — implemented in #151. Closes #150 (V-L2-F4). +**Refines [ADR-0012](0012-json-family-sidecar-storage.adoc)** (V-L2-F3): it +does not reverse the JSON-family decision, it hardens its concurrency story. + +## Context + +ADR-0012 implemented the JSON-family sidecar with crash-safe writes +(temp file + same-directory atomic `rename`) but a **single-writer-by- +assumption** concurrency model: it noted that the SQLite path serialises +writers via the database write lock, whereas the JSON path assumed one +writer at a time. Two processes that each ran the load→mutate→save cycle +against the same sidecar could interleave and silently lose writes — a real +hazard for `gc` running alongside an appending process. + +The `provenance` and `history` CLI subcommands were also still stubs, so the +JSON (and SQLite) sidecars could be written but not yet *queried* from the +CLI. + +## Decision + +Harden the JSON store and wire the read-side CLI (#151): + +* **Cross-process advisory write lock.** A dependency-free lock file + (`.lock`) is created exclusively, with bounded backoff and a + stale-lock *steal* heuristic, and released via RAII on drop. It is held + across the **entire** load→mutate→save cycle, so a concurrent `gc` and + `append` serialise instead of clobbering each other. (`src/sidecar/lock.rs`.) +* **Atomic durability, formalised.** The temp-file + same-directory `rename` + in `save()` is documented as the durability boundary and is now performed + under the write lock. +* **`provenance` / `history` wired.** `provenance ` lists the chain + and reports `verify_chain` plus any fork points; `history + [--at T]` lists temporal versions (or the point-in-time snapshot at `T`). + Both work against the SQLite *and* JSON backends; `postgres` reads remain + an explicit "not yet implemented". +* **Release.** Version `0.1.0` → `0.2.0`, with a CHANGELOG entry. + +## Consequences + +### Positive + +- Concurrent cross-process writers to a JSON sidecar are safe: serialised by + the advisory lock and crash-safe via atomic rename. This closes the main + parity gap ADR-0012 called out against the SQLite store. +- No new dependencies — the lock is built on the standard library. +- `provenance` and `history` now function against real sidecars, for both + storage families. + +### Negative + +- The lock is **advisory** (cooperative) with a stale-steal heuristic: a + writer wedged past the staleness window can have its lock stolen. This is + the documented trade-off of a dependency-free, portable lock. +- Writes remain whole-file `O(n)` rewrites (no incremental append for the + JSON encodings) — acceptable for the small-to-medium sidecars this store + targets, but not for very large histories. +- `postgres` provenance/history reads are still unimplemented (tracked + separately). + +## References + +- Issue #150 (V-L2-F4); PR #151. +- Refines ADR-0012 (V-L2-F3, the JSON-family backend; issue #146 / #148). +- ADR-0010 (provenance forks are first-class) — the chain semantics the + JSON store mirrors.