Skip to content

feat(solana-indexer): PR 4 — Component structs (skeleton declarations)#4514

Open
tilacog wants to merge 2 commits into
solana-indexer/PR3-bootstrapfrom
solana-indexer/PR4-bootstrap
Open

feat(solana-indexer): PR 4 — Component structs (skeleton declarations)#4514
tilacog wants to merge 2 commits into
solana-indexer/PR3-bootstrapfrom
solana-indexer/PR4-bootstrap

Conversation

@tilacog

@tilacog tilacog commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Description

Adds the indexer/ module with skeleton declarations for the four components that will do the actual work of the indexer. Each struct declares its fields, a constructor, and doc comments describing what it will do; the run bodies are unimplemented! and the behavior lands in later PRs.

The four components and their roles:

  • Ingester: subscribes to the Yellowstone gRPC stream and drains it as fast as updates arrive, forwarding them to the decoder. It does no decoding itself, so the socket never backs up behind slow processing. It is also the single writer of the "latest chain slot" counter that the other components use to know how far the chain has advanced.
  • Decoder: receives the raw stream updates, picks out transactions belonging to the settlement and SolFlow programs, matches each transaction with its corresponding account-update snapshot, and persists the resulting typed events to the store.
  • Partial-event watchdog: some events arrive in two halves (a transaction update and an account update) that don't always land together. The decoder parks the half it has in a map shared with the watchdog; the watchdog periodically scans that map and dead-letters any entry whose other half never showed up within the slot window, recording which half went missing.
  • Finalization worker: rows are first written at the confirmed commitment level. This worker re-checks them against the chain and promotes them to finalized, or marks them rolled back if the transaction disappeared. It uses a cheap batched RPC call for recent rows and falls back to one-call-per-row lookups for rows old enough that the batched method no longer reports them.

The field declarations also pin down how the components talk to each other: the ingester feeds the decoder over a bounded channel, while the decoder and the watchdog share a concurrent map with no message passing between them. That second decision reshaped the partial-event types, so types/channel.rs became types/shared.rs.

Changes

  • Added indexer/ingester.rs: the Ingester struct. Generic over the connection so unit tests can drive it with a mock (the trait-bound approach chosen in PR 3 instead of a third in-crate trait). Also declares the shared latest-chain-slot counter and the constants for reconnect backoff and channel capacity.
  • Added indexer/decoder.rs: the Decoder struct, holding the receiving end of the channel, the shared partial-event map, and the two program ids it filters for.
  • Added indexer/watchdog.rs: the PartialEventWatchdog struct, holding the store and its view of the shared partial-event map.
  • Added indexer/finalization.rs: the FinalizationWorker struct, with module docs explaining the two promotion flows and the constants that bound them (finalization window, batch size, retention horizon).
  • Renamed types/channel.rs to types/shared.rs and reworked the partial-event types: instead of payloads sent over a channel, a partial event is now a key (slot + signature) and a value (whichever half arrived first) in the map shared by the decoder and the watchdog.
  • Dependencies: added yellowstone-grpc-client to the workspace (source of the connection trait the ingester is generic over), plus dashmap and a few tokio features to the crate.

How to test

  1. cargo check -p solana-indexer
  2. cargo clippy --locked -p solana-indexer --all-features --all-targets -- -D warnings

No unit tests are included: the structs declare shape only, and every run body is unimplemented! until the behavior PRs.

@tilacog tilacog requested a review from a team as a code owner June 10, 2026 17:33

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the skeleton structure for the Solana settlement indexer components, including the Ingester, Decoder, PartialEventWatchdog, and FinalizationWorker, along with necessary dependency updates and shared type definitions. Feedback on the changes suggests replacing the global static LATEST_CHAIN_SLOT with a shared Arc passed via constructors to avoid process-wide shared mutable state and potential race conditions during parallel testing.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

/// The sole writer is the ingester, on every slot-filter message. Anchors the
/// partial-event watchdog and the finalization worker. Cold start is zero; the
/// watchdog skips its comparison on the first tick.
pub static LATEST_CHAIN_SLOT: AtomicU64 = AtomicU64::new(0);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using a global static AtomicU64 for LATEST_CHAIN_SLOT introduces shared mutable state across the entire process. This causes race conditions and flakiness when running unit/integration tests in parallel (Cargo's default behavior). It also prevents running multiple indexer instances in the same process.

Actionable Suggestion:
Remove the global static and instead pass an Arc<AtomicU64> (or a shared state struct) to the constructors of Ingester, Decoder, PartialEventWatchdog, and FinalizationWorker.

References
  1. Focus exclusively on identifying missing edge cases, potential race conditions, or logic that deviates from the PR's stated goals. (link)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant