feat(rustledger): in-process Component-Model engine behind a flag#161
Open
robcohen wants to merge 2 commits into
Open
feat(rustledger): in-process Component-Model engine behind a flag#161robcohen wants to merge 2 commits into
robcohen wants to merge 2 commits into
Conversation
Adds `RustledgerComponentEngine`, an experimental successor to the JSON-RPC `RustledgerEngine` that drives rustledger's typed WASI Preview 2 / Component Model component (`rustledger-ffi-component`, rustledger beancount#1384) directly via `wasmtime-py` — no `wasmtime` CLI subprocess per call, no hand-mirrored JSON DTOs. - `component_engine.py`: loads the component once, instantiates with WASI p2, and calls the typed `ledger`/`util`/`format` exports. A generic, type-driven marshaller (`_marshal`) walks the component's own `RecordType.fields` / `VariantType.cases` to turn typed `Record`/`Variant`/`list` values into plain dicts/lists (variants as discriminated `{"type": ...}`), so there are no hand-maintained per-type field lists. `load_full` pre-opens the entry file's directory into the WASI sandbox for include resolution. - `get_engine()` dispatcher selects the backend via `RUSTFAVA_RUSTLEDGER_BACKEND=component`; default stays JSON-RPC. The component module (and `wasmtime`) is imported lazily, so the default path gains no dependency. `wasmtime` is a new optional `component` extra. - Smoke tests (`test_rustledger_component_engine.py`) cover version/load/query/ account-type/load-file; skipped unless `wasmtime` + the wasip2 artifact are present. Gated behind the flag because the component is `publish = false` upstream (rustledger ADR-0006 Phase 3); this proves the in-process typed-host path works end-to-end ahead of that release. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses the review of #161 — the generic marshaller diverged from the JSON-RPC engine's shapes, so downstream loader/types/options mis-parsed nearly every multi-word field: - Map WIT kebab-case identifiers to snake_case keys / variant tags (the whole downstream reads snake_case; only single-word fields survived before). - Render WIT string-keyed maps (`list<tuple<string, V>>` — display-precision, meta.user, …) as dicts so `options_from_json`'s `.items()` doesn't crash. - Flatten `meta.user` into the meta object with scalar values (mirroring the JSON-RPC `Meta`'s `#[serde(flatten)]` + meta-value flattening). - Fix the `result<ok, err>` branch: it lifts to a Variant and does NOT raise — surface `err` as an exception, unwrap `ok` (was returning a raw Variant). - Serialize the shared, non-thread-safe wasmtime `Store` behind a lock (Fava is multi-threaded; `load_full` uses a fresh per-call instance). Also: - Regenerate `uv.lock` to include `wasmtime` (the `component` extra) — fixes the `check-lockfile` CI failure. - mypy: treat optional `wasmtime` as untyped and allow Any returns from the FFI-boundary engine (two scoped overrides). - Strengthen the tests to round-trip marshalled output through the real `options_from_json` / `directives_from_json`, which is what would have caught these bugs. Verified end-to-end against a locally-built component wasm. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
robcohen
force-pushed
the
feat/component-engine
branch
from
66c1e52 to
b2c35c4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an experimental, flag-gated rustledger engine that drives the typed
WASI Preview 2 / Component-Model component (
rustledger-ffi-component,rustledger #1384)
in-process via
wasmtime-py— the successor to the current JSON-RPC engine,which spawns the
wasmtimeCLI once per call and exchanges hand-mirrored JSONDTOs over stdin/stdout.
What's here
RustledgerComponentEngine(src/rustfava/rustledger/component_engine.py)loads the component once, instantiates it with WASI p2, and calls the typed
ledger/util/formatexports directly (no JSON envelope, nosubprocess). It mirrors
RustledgerEngine's surface:version,load,query,validate,format_entries,get_account_type,is_encrypted,load_full._marshal) converts the component'styped
Record/Variant/listvalues into plain Python dicts/lists bywalking the component's own type metadata (
RecordType.fields,VariantType.cases). Variants render as discriminated{"type": <case>, ...}(mirroring the JSON-RPC tagged unions), so there are no hand-maintained
per-type field lists.
get_engine()(rustfava.rustledger) selects the backend viaRUSTFAVA_RUSTLEDGER_BACKEND=component; the default stays the JSON-RPC engine.The component module — and therefore
wasmtime— is imported lazily, sothe default path gains no new dependency.
wasmtimeis a new optionalcomponentextra.Why now (and why behind a flag)
This is the consumer-migration proof for the rustledger Component-Model work.
The component is still
publish = falseupstream (rustledger ADR-0006 Phase 3),so rustfava can't ship against a released artifact yet — but the in-process
typed-host path is now proven end-to-end, ready to flip on when it releases.
The in-process model also removes the per-call subprocess cost and the JSON DTO
drift.
Testing
tests/test_rustledger_component_engine.pycovers version / load (typeddirective marshalling) / query / account-type /
load_full(include resolutionover a WASI pre-open). They skip unless
wasmtimeis installed and thewasip2 artifact is locatable (
RUSTLEDGER_COMPONENT_WASMor bundled). Build itwith:
rustfava.rustledgerpulls in nowasmtime.ruff check+ruff formatclean.Follow-ups (not in scope here)
.wasmand flip the default once it's a release artifact.load_fullcurrently pre-opens only the entry file's directory tree, socross-tree (
../) includes are unreachable; a wider pre-open root would liftthat (documented in the method).
🤖 Generated with Claude Code