Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.git
node_modules
npm-debug.log
.gitignore
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [5.3.2] — Unreleased

### Changed
- **Vitest workspace split** — unit, integration, and benchmark suites now live in explicit workspace projects so the integration suite always runs with `fileParallelism: false`, regardless of the exact CLI invocation shape.
- **Status semantics** — `STATUS.md` now distinguishes the last released version (`v5.3.1`) from the current branch version (`v5.3.2`).

### Fixed
- **CLI version drift** — `bin/git-cas.js` now reads the package version instead of carrying a stale hardcoded literal, so `git-cas --version` tracks the in-repo release line correctly.

## [5.3.1] — 2026-03-15

### Fixed
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ We use the object database.
- **Lifecycle management** `readManifest`, `inspectAsset`, `collectReferencedChunks` — inspect trees, plan deletions, audit storage.
- **Vault** GC-safe ref-based storage. One ref (`refs/cas/vault`) indexes all assets by slug. No more silent data loss from `git gc`.
- **Interactive dashboard** `git cas inspect` with chunk heatmap, animated progress bars, and rich manifest views.
- **Verify & JSON output** `git cas verify` checks integrity; `--json` on all commands for CI/scripting.
- **Verify & JSON output** `git cas verify` checks integrity; `--json` on all current human-facing commands provides convenient structured output for CI/scripting.

**Use it for:** binary assets, build artifacts, model weights, data packs, secret bundles, weird experiments, etc.

Expand Down
1,896 changes: 184 additions & 1,712 deletions ROADMAP.md

Large diffs are not rendered by default.

128 changes: 62 additions & 66 deletions STATUS.md
Original file line number Diff line number Diff line change
@@ -1,88 +1,84 @@
# @git-stunts/cas — Project Status

**Current version:** v5.1.0 (Locksmith)
**Last release:** 2026-02-28
**Test suite:** 757 tests (vitest)
**Current release:** `v5.3.1`
**Current branch version:** `v5.3.2`
**Last release:** `2026-03-15`
**Current line:** M16 Capstone shipped in `v5.3.0`; `v5.3.1` fixed repeated-chunk tree emission; `v5.3.2` is the next maintenance/doc/test follow-up in flight.
**Runtimes:** Node.js 22.x, Bun, Deno

---

## What's shipped
## Interface Strategy

| Version | Codename | Highlights |
|---------|----------|------------|
| v5.1.0 | Locksmith | Envelope encryption (DEK/KEK), multi-recipient APIs, `--recipient` CLI, recipient management |
| v5.0.0 | Hydra | Content-defined chunking (CDC), `ChunkingPort`, buzhash engine, 98% dedup on edits |
| v4.0.1 | Spit Shine + Cockpit | CryptoPort refactor, `verify` command, `--json` mode, `runAction`, vault list filtering |
| v4.0.0 | Conduit | ObservabilityPort, `restoreStream()`, parallel chunk I/O, `concurrency` option |
| v3.1.0 | Bijou | Interactive vault dashboard, animated progress bars, `git cas inspect`, chunk heatmap |
| v3.0.0 | Vault | GC-safe ref-based storage (`refs/cas/vault`), slug-based addressing, vault CLI |
| v2.0.0 | Horizon | Compression (gzip), KDF (pbkdf2/scrypt), Merkle manifests |
| v1.x | — | Core CAS, AES-256-GCM encryption, fixed chunking, Git ODB persistence |
- **Human CLI/TUI:** the current public operator surface. Existing `git cas ...` commands, Bijou formatting, prompts, dashboards, and `--json` convenience output stay here.
- **Agent CLI:** planned next as `git cas agent`. It will be JSONL-first, non-interactive by default, and independent from Bijou rendering or TTY-only behavior.

---

## What's next
## Recently Shipped

One open milestone remains.
| Version | Milestone | Highlights |
|---------|-----------|------------|
| `v5.3.1` | Maintenance | Repeated-chunk tree integrity fix; unique chunk tree entries; `git fsck` regression coverage |
| `v5.3.0` | M16 Capstone | Audit remediation, `.casrc`, passphrase-file support, restore guards, `encryptionCount`, lifecycle rename |
| `v5.2.0` | M12 Carousel | Key rotation without re-encrypting data |
| `v5.1.0` | M11 Locksmith | Envelope encryption and recipient management |
| `v5.0.0` | M10 Hydra | Content-defined chunking |
| `v4.0.1` | M8 + M9 | Review hardening, `verify`, `--json`, CLI polish |
| `v4.0.0` | M14 Conduit | Streaming restore, observability, parallel chunk I/O |
| `v3.1.0` | M13 Bijou | Interactive dashboard and animated progress |

### M12 — Carousel (~13h)
Key rotation without re-encrypting data. Now unblocked by M11 Locksmith.

- [ ] **12.1** Key rotation workflow (`rotateKey()`)
- [ ] **12.2** Key version tracking in manifest
- [ ] **12.3** CLI key rotation commands
- [ ] **12.4** Vault-level key rotation
Milestone labels are thematic and non-sequential; the versions above are listed in release order.

---

## Dependency graph
## Next Up

### M17 — Ledger (`v5.3.2`)

Planning and ops reset:

- Reconcile `ROADMAP.md`, `STATUS.md`, and release messaging
- Add review automation (`CODEOWNERS` or equivalent)
- Document Git tree ordering test conventions
- Define release-prep workflow for changelog/version timing
- Automate test-count injection into release notes or changelog prep
- Add property-based fuzz coverage for envelope encryption

### M18 — Relay (`v5.4.0`)

```
M8 Spit Shine ──────── ✅ v4.0.1
M9 Cockpit ─────────── ✅ v4.0.1
M10 Hydra ──────────── ✅ v5.0.0
M11 Locksmith ──────── ✅ v5.1.0
└──► M12 Carousel ── (ready)
```
LLM-native CLI foundation:

- Introduce `git cas agent`
- Define the JSONL protocol envelope and exit codes
- Add machine-facing parity for the current operational command set
- Enforce strict non-interactive input handling

### M19 — Nouveau (`v5.5.0`)

Human UX refresh:

- Upgrade Bijou packages to `3.0.0`
- Move the inspector shell to the v3 `ViewOutput` model
- Split the dashboard into sub-apps
- Add better styling, motion, layout persistence, and richer heatmap/detail rendering

---

## Backlog (unscheduled ideas)

- Named vaults (`refs/cas/vaults/<name>`)
- Export vault to archive
- Publish to working tree / branch
- Duplicate detection on store
- Repo scan / dedup advisor
- Add `CODEOWNERS` or reviewer auto-assignment for PRs
- Document Git tree filename ordering semantics in test conventions
- Define release-prep workflow for CHANGELOG/version bump timing
- Automate test count injection into CHANGELOG from CI output
- Property-based fuzz tests for envelope encryption round-trips
- Investigate HSM/Vault key management as a future `KeyManagementPort`

## Visions (researched, not committed)

- **V1** Snapshot trees — directory-level store (~410 LoC, ~19h)
- **V2** Portable bundles — air-gap transfer (~340 LoC, ~15h)
- **V3** Manifest diff engine (~180 LoC, ~8h)
- **V4** CompressionPort — zstd, brotli, lz4 (~180 LoC, ~8h)
- **V5** Watch mode — continuous sync (~220 LoC, ~10h)
- **V6** Interactive passphrase prompt (~90 LoC, ~4h)

## Known concerns

| # | Issue | Severity | Summary |
|---|-------|----------|---------|
| C1 | Memory amplification | High | Encrypted/compressed restore buffers entire file |
| C2 | Orphaned blobs | Medium | STREAM_ERROR leaves unreferenced blobs in ODB |
| C3 | No chunk size cap | Medium | No upper bound on configured chunk size |
| C4 | Web Crypto buffering | Medium | Deno adapter silently buffers entire file |
| C5 | Passphrase exposure | High | `--vault-passphrase` visible in shell history |
| C6 | KDF no rate limit | Low | No brute-force detection on failed decryption |
| C7 | GCM nonce collision | Low | 96-bit random nonce, safe to ~2^32 encryptions |
## Sequenced Roadmap

| Version | Milestone | Theme |
|---------|-----------|-------|
| `v5.3.2` | M17 Ledger | Planning and ops reset |
| `v5.4.0` | M18 Relay | LLM-native CLI foundation |
| `v5.5.0` | M19 Nouveau | Bijou v3 human UX refresh |
| `v5.6.0` | M20 Sentinel | Vault health and safety |
| `v5.7.0` | M21 Atelier | Vault ergonomics and publishing |
| `v5.8.0` | M22 Cartographer | Repo intelligence and change analysis |
| `v5.9.0` | M23 Courier | Artifact sets and transfer |
| `v5.10.0` | M24 Spectrum | Storage and observability extensibility |
| `v5.11.0` | M25 Bastion | Enterprise key-management research |

---

*Full task cards: [ROADMAP.md](./ROADMAP.md) | Completed: [COMPLETED_TASKS.md](./COMPLETED_TASKS.md) | Superseded: [GRAVEYARD.md](./GRAVEYARD.md)*
*Future details: [ROADMAP.md](./ROADMAP.md) | Shipped detail: [COMPLETED_TASKS.md](./COMPLETED_TASKS.md) | Superseded: [GRAVEYARD.md](./GRAVEYARD.md)*
9 changes: 6 additions & 3 deletions bin/actions.js
Original file line number Diff line number Diff line change
Expand Up @@ -70,19 +70,22 @@ function defaultDelay(ms) {
*
* @param {(...args: any[]) => Promise<void>} fn - The async action function.
* @param {() => boolean} getJson - Lazy getter for --json flag value.
* @param {{ delay?: (ms: number) => Promise<void> }} [options] - Injectable dependencies.
* @param {{ delay?: (ms: number) => Promise<void>, setExitCode?: (code: number) => void }} [options] - Injectable dependencies.
* @returns {(...args: any[]) => Promise<void>} Wrapped action.
*/
export function runAction(fn, getJson, { delay = defaultDelay } = {}) {
export function runAction(fn, getJson, {
delay = defaultDelay,
setExitCode = (code) => { process.exitCode = code; },
} = {}) {
return async (/** @type {any[]} */ ...args) => {
try {
await fn(...args);
} catch (/** @type {any} */ err) {
if (err?.code === 'INTEGRITY_ERROR') {
await delay(1000);
}
setExitCode(1);
writeError(err, getJson());
process.exitCode = 1;
}
};
}
Expand Down
14 changes: 11 additions & 3 deletions bin/git-cas.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
#!/usr/bin/env node

import { readFileSync } from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { program, Option } from 'commander';
import GitPlumbing, { ShellRunnerFactory } from '@git-stunts/plumbing';
import ContentAddressableStore, { EventEmitterObserver, CborCodec } from '../index.js';
Expand All @@ -11,16 +13,23 @@ import { renderHistoryTimeline } from './ui/history-timeline.js';
import { renderManifestView } from './ui/manifest-view.js';
import { renderHeatmap } from './ui/heatmap.js';
import { runAction } from './actions.js';
import { flushStdioAndExit, installBrokenPipeHandlers } from './io.js';
import { filterEntries, formatTable, formatTabSeparated } from './ui/vault-list.js';
import { readPassphraseFile, promptPassphrase } from './ui/passphrase-prompt.js';
import { loadConfig, mergeConfig } from './config.js';

const __dirname = path.dirname(fileURLToPath(import.meta.url));
const { version: CLI_VERSION } = JSON.parse(
readFileSync(path.resolve(__dirname, '../package.json'), 'utf8'),
);

const getJson = () => program.opts().json;
installBrokenPipeHandlers();

program
.name('git-cas')
.description('Content Addressable Storage backed by Git')
.version('5.2.2')
.version(CLI_VERSION)
.option('-q, --quiet', 'Suppress progress output')
.option('--json', 'Output results as JSON');

Expand Down Expand Up @@ -751,5 +760,4 @@ await program.parseAsync();

// Flush stdout/stderr before exiting — spawned git child processes leave
// libuv handles that prevent natural exit in containerized environments.
const code = process.exitCode || 0;
process.stdout.write('', () => process.stderr.write('', () => process.exit(code)));
await flushStdioAndExit();
86 changes: 86 additions & 0 deletions bin/io.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import { setTimeout as delay } from 'node:timers/promises';

/**
* @param {unknown} err
* @returns {err is NodeJS.ErrnoException}
*/
function isBrokenPipeError(err) {
return Boolean(err && typeof err === 'object' && /** @type {NodeJS.ErrnoException} */ (err).code === 'EPIPE');
}

/**
* Install stdout/stderr error handlers that exit cleanly when the downstream
* consumer closes the pipe before the CLI finishes writing.
*
* @param {Object} [options]
* @param {{ on(event: string, listener: (...args: any[]) => void): any, removeListener(event: string, listener: (...args: any[]) => void): any }} [options.stdout]
* @param {{ on(event: string, listener: (...args: any[]) => void): any, removeListener(event: string, listener: (...args: any[]) => void): any }} [options.stderr]
* @param {(code?: number) => never} [options.exit]
* @param {() => number} [options.getExitCode]
* @returns {{ dispose(): void }}
*/
export function installBrokenPipeHandlers({
stdout = process.stdout,
stderr = process.stderr,
exit = process.exit,
getExitCode = () => process.exitCode || 0,
} = {}) {
const onError = (/** @type {unknown} */ err) => {
if (isBrokenPipeError(err)) {
exit(getExitCode());
}
};

stdout.on('error', onError);
stderr.on('error', onError);

return {
dispose() {
stdout.removeListener('error', onError);
stderr.removeListener('error', onError);
},
};
}

/**
* Flush stdout/stderr before exit so the CLI does not hang on open handles in
* containerized test environments.
*
* @param {Object} [options]
* @param {{ write(chunk: string, callback?: () => void): boolean }} [options.stdout]
* @param {{ write(chunk: string, callback?: () => void): boolean }} [options.stderr]
* @param {(code?: number) => void} [options.exit]
* @param {number} [options.code]
* @returns {Promise<void>}
*/
export async function flushStdioAndExit({
stdout = process.stdout,
stderr = process.stderr,
exit = process.exit,
code = process.exitCode || 0,
} = {}) {
await flushStream(stdout);
await flushStream(stderr);
exit(code);
}

/**
* @param {{ write(chunk: string, callback?: () => void): boolean }} stream
* @returns {Promise<void>}
*/
async function flushStream(stream) {
try {
await new Promise((resolve) => {
stream.write('', resolve);
});
} catch (err) {
if (!isBrokenPipeError(err)) {
throw err;
}
}

// Give stream error handlers one turn to observe late EPIPE events before exit.
await delay(0);
}

export { isBrokenPipeError };
2 changes: 1 addition & 1 deletion jsr.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@git-stunts/git-cas",
"version": "5.3.1",
"version": "5.3.2",
"exports": {
".": "./index.js",
"./service": "./src/domain/services/CasService.js",
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@git-stunts/git-cas",
"version": "5.3.1",
"version": "5.3.2",
"description": "Content-addressed storage backed by Git's object database, with optional encryption and pluggable codecs",
"type": "module",
"main": "index.js",
Expand Down
14 changes: 12 additions & 2 deletions test/integration/round-trip.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { mkdtempSync, rmSync, writeFileSync, readFileSync } from 'node:fs';
import { randomBytes } from 'node:crypto';
import { execSync, spawnSync } from 'node:child_process';
import { spawnSync } from 'node:child_process';
import path from 'node:path';
import os from 'node:os';
import GitPlumbing from '@git-stunts/plumbing';
Expand All @@ -31,9 +31,19 @@ let repoDir;
let cas;
let casCbor;

function initBareRepo(cwd) {
const result = spawnSync('git', ['init', '--bare'], { cwd, encoding: 'utf8' });
if (result.error) {
throw result.error;
}
if (result.status !== 0) {
throw new Error(`${result.stderr ?? result.stdout ?? 'git init --bare failed'}`.trim());
}
}

beforeAll(() => {
repoDir = mkdtempSync(path.join(os.tmpdir(), 'cas-integ-'));
execSync('git init --bare', { cwd: repoDir, stdio: 'ignore' });
initBareRepo(repoDir);

const plumbing = GitPlumbing.createDefault({ cwd: repoDir });
cas = new ContentAddressableStore({ plumbing });
Expand Down
Loading
Loading