Transactions silently dropped when chain started via 'pnpm dev' (turbo); 'node' direct works

[zkokio]

Transactions silently dropped when chain is started via pnpm dev (turbo) — node direct start works

Summary

When the in-memory chain is started via pnpm dev (which invokes turbo run dev), every transaction submitted by a node client is silently dropped: blocks are produced normally but every block reports 0 txs. Starting the same chain directly with node ./src/start.ts start ./core/environments/inmemory/chain.config.ts makes the exact same transactions land in blocks and execute successfully. No error is logged in either case.

This is a hard-to-find class of bug — there is no failure signal, just 0 txs forever — so writing it up in case it helps anyone else hitting it.

Environment

• starter-kit cloned earlier in 2026, ~16 commits ahead of develop with custom runtime modules added
• @proto-kit/* 0.2.0
• o1js 2.14.0-dev.e1080
• tsyringe ^4.10.0
• Node 18.18.0
• pnpm 9.x (whatever the starter ships with)
• turbo 2.6.1
• Ubuntu 24, Hetzner CX/CPX instance
• Custom runtime modules added beyond the starter (Treasury, Ledger, UnitRegistry, Tax, Sales, DevelopmentRegistry) — not relevant to the reproduction; the bug also affects starter modules.

What we observed

Running the chain via pnpm dev:

chain:dev: Produced block #1 (0 txs)
chain:dev: Produced block #2 (0 txs)
chain:dev: Produced block #3 (0 txs)
...

Submitting a transaction from a node test client — await tx.sign(); await tx.send(); — completes without throwing. The chain log keeps producing blocks at 0 txs. State reads return empty/null because the writes never happened. No error message anywhere — stdout, stderr, GraphQL response, or test client.

GraphQL schema introspection works (the chain is reachable). Module registration happens correctly at startup (visible with --logLevel debug). The test client successfully calls .start() against the GraphQL endpoint. Everything looks fine.

Reproduction

1. Clone the starter and add a couple of runtime modules with a few @runtimeMethods and StateMap state (or just use the stock starter — the failure is independent of the runtime contents).
2. Start the chain via pnpm dev.
3. Wait for it to produce blocks.
4. Run any test that submits a tx via a node client (buildNodeClient from core/environments/node.config.ts, client.transaction(...), tx.sign(), tx.send()).
5. Observe: every block produced after the tx submission is still 0 txs. State remains empty. No errors anywhere.

Workaround

Start the chain process directly, bypassing pnpm dev / turbo:

cd packages/chain
nohup node \
  --loader ts-node/esm \
  --experimental-vm-modules \
  --experimental-wasm-modules \
  --es-module-specifier-resolution=node \
  ./src/start.ts \
  start ./core/environments/inmemory/chain.config.ts \
  --logLevel debug \
  > /tmp/protokit.log 2>&1 &

The same environment variables are still required:

export PROTOKIT_ENV_FOLDER=inmemory
export PROTOKIT_GRAPHQL_PORT=8080
export PROTOKIT_TRANSACTION_FEE_RECIPIENT_PUBLIC_KEY=B62q...

With this direct invocation, txs land in blocks as expected — same code, same env, same tests, same everything except the wrapper.

Suspected cause

We don't know. Things we noticed but didn't confirm:

• turbo.json has "envMode": "loose" so it isn't a basic env-stripping issue.
• The dev task in turbo.json is marked persistent: true and cache: false, which is correct for long-running processes.
• The pnpm dev chain is: top-level pnpm dev → turbo run dev --env-mode=loose → chain:dev → pnpm run sequencer:dev → the actual node ./src/start.ts ... command.
• There are at least three levels of process indirection (turbo → pnpm → node-with-loaders). One of them appears to be interfering with tx submission, but we couldn't pin which.

Things we ruled out:

• Build failure (build is clean in both modes).
• GraphQL schema mismatch (introspection succeeds, mutations are discoverable).
• Module registration (debug log shows all modules registered correctly under both invocations).
• Auth/signing errors (no errors logged anywhere; the test client's .send() resolves without throwing).
• Env variables (the same env vars are exported in both cases; loose mode is on).

A guess worth testing: turbo may be capturing or modifying stdio in a way that breaks the GraphQL transport between the node test client and the sequencer's mempool. But this is speculation.

Impact

For anyone using the starter as-is (pnpm dev) and running tests against it, every test that exercises chain mutations will silently fail. The tests will appear to "run" — assertions just fire against empty state — and the failure mode is consistent (everything returns null/0). It took us several hours of debugging code that turned out to be fine before we suspected the wrapper.

Suggestions

A startup banner from the chain process saying "submitting txs via X; using transport Y" would help users confirm the path is working. Or: a simple chain:dev: warning: 30 blocks produced with 0 txs and 0 pending in mempool — is your test client connecting to the right endpoint? periodic check would have saved us. Anything that breaks the silence.

Happy to share more detail / repo state / chain logs if helpful. Real working example of the starter + custom modules running under the direct-node invocation is at https://github.com/zkokio/minalia-protokit.

— Pete (zkokio), MINALIA

[zkokio]

Follow-up: workaround confirmed for real workloads

Posting an update in case it helps anyone landing here.

The direct-node workaround I described in the issue has now held up under sustained development. Since switching off pnpm dev, I've:

• Added new state fields (State.from<PublicKey>) to an existing module
• Added a new @runtimeMethod (setAuthority) to that module
• Gated existing methods with an assertAuthority() helper
• Added a second new @runtimeMethod (forceTransfer) with different access rules
• Switched a downstream module (MinaliaTax) to call the new method via @inject
• Updated three test scripts with new bootstrap steps

Every one of these landed cleanly with txs being processed normally — blocks reporting (1 tx) per submission, state mutating as expected. The same changes had silently failed under pnpm dev no matter what I tried.

So whatever the wrapper is doing, it's not a one-off — it's a consistent, reproducible block on all tx submission. And the workaround isn't a one-off either; it's stable enough to do real iterative development on.

The --logLevel debug flag (already documented in start.ts but not surfaced anywhere user-facing that I could find) was the second key piece — it showed the chain happily registering modules and preparing tasks under both invocations, which is how we narrowed the problem down to outside the chain itself.

Still happy to share more if useful. The example repo is at https://github.com/zkokio/minalia-protokit — the README now documents the workaround as a hard requirement for running the tests there.