Skip to content

V-L2-F3: implement JSON-family sidecar storage backend (plain JSON / JSON-LD / NDJSON) #146

@hyperpolymath

Description

@hyperpolymath

Context

#112 (V-L2-F2) dropped the never-implemented json sidecar store because the octad runtime is relational and a JSON store wasn't on the roadmap. That decision shipped in #144.

This issue reverses that for a deliberately-scoped JSON family backend, at the maintainer's request: a real json store covering three on-disk encodings — plain JSON, JSON-LD, and NDJSON — with full parity to the runtime operations the SQLite path implements today.

Design

Manifest

  • [sidecar].storage = "json" is accepted again.
  • New [sidecar].format key selects the encoding: "plain" (default) | "ld" | "ndjson". Only meaningful when storage = "json"; ignored for sqlite/postgres.
  • A single resolver sidecar::StorageKind::resolve(storage, format) becomes the one source of truth (Sqlite | Postgres | Json(JsonFormat)), used by validate, generate, drift, and gc. SqlDialect::from_storage is folded into it.

Storage model & codec

  • One internal SidecarData model mirrors the verisimdb_* tables (provenance log + chain-head set, temporal versions, lineage edges, access policies).
  • Format is purely a codec over that model — operations are written once and are format-independent:
    • plain: one object keyed by table name, arrays of rows.
    • ld: @context + @graph of typed (@type/@id) nodes — genuine linked data.
    • ndjson: one {"_table": …, …} record per line.
  • Writes are crash-safe (temp file + atomic rename). Logical append-only semantics (provenance/temporal never mutate history except gc); physical representation is rewritten atomically. Single-writer model documented (the one explicit difference from SQLite's lock-based serialization).

Octad parity (matches what the SQLite runtime does today)

  • Provenance: linear append, append_fork, verify_chain, fork_points — reusing abi::ProvenanceEntry::compute_hash so hashing is identical across backends.
  • Temporal: append_version (monotonic, exactly-one-current invariant enforced in code), read_current, read_at, rollback_to.
  • Drift: temporal drift via the existing tier1::drift::temporal_drift_score kernel.
  • GC: purge provenance/temporal(superseded)/lineage by age.
  • Generate: emit a JSON sidecar scaffold (sidecar_schema.{json,jsonld,ndjson}) for the enabled dimensions instead of SQL DDL; interceptors still emitted.

Acceptance

  • [sidecar].storage = "json" + [sidecar].format = plain|ld|ndjson parse, validate, and round-trip; bad format rejected with a clear message.
  • StorageKind::resolve is the single resolver; validate/doctor accept json and reject typos.
  • JSON store passes provenance (incl. fork + tamper-detection), temporal (incl. monotonicity + one-current + rollback), drift, and gc tests mirroring the SQLite suite — for all three formats.
  • generate emits a format-appropriate scaffold; drift and gc honour the json store.
  • SidecarConfig docs + manifest template document json and format.

Re-opens the capability dropped in #112 / #144.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions