flashblocks: track tx time-to-inclusion within a flashblock

[bitwiseguy]

Flashblocks feature

When --flashblocks-ws-url is provided, contender connects to a flashblocks WebSocket endpoint that auto-streams flashblocks. Each flashblock contains metadata.receipts keyed by tx hash — contender matches these against its pending tx cache to record when each tx first appeared in a flashblock.

A preflight check validates the ws endpoint on startup. Since flashblock diffs may arrive before the SentRunTx message is processed by the actor, marks are buffered in a pending_flashblock_marks map and applied retroactively.

Two new report charts are conditionally rendered when flashblock data is present:

• Flashblock Time to Inclusion — histogram of latency for tx inclusion within a flashblock (100ms buckets)
• Flashblock Index — distribution of which sub-block index (0, 1, 2, ...) txs are included in

Additional changes

• Upgrade timestamp precision from seconds to milliseconds across the DB schema, tx actor, and report charts
• Add optional --skip-tx-traces flag to speed up report generation by skipping debug_traceTransaction calls
• Improve setup reliability: detect reverted setup txs and poll for contract code visibility before proceeding
• Skip unnecessary eth_blobBaseFee RPC calls for non-EIP-4844 tx types
• Fix batch RPC start timestamp to be captured before the HTTP POSTs
• Improve spam loop's 1sec cadence reliability
  • Defer batch task collection to the next tick so tasks run during the interval wait
  • Change timed spammer missed-tick behavior from Burst to Delay to prevent cascading immediate ticks

Breaking Change

• DB schema v6 → v7: run_txs table adds flashblock_latency and flashblock_index columns; start_timestamp and end_timestamp now store milliseconds instead of seconds. Existing databases are incompatible.

Related Issue

• Closes https://github.com/ethereum-optimism/platforms-team/issues/1925

[scharissis]

Potential Issues:

1. pending_flashblock_marks unbounded growth — The 20s TTL eviction only runs during RemoveConfirmed flush events. If no blocks are being confirmed (e.g., chain stall), marks from foreign tx hashes could accumulate indefinitely. Consider also evicting in the main loop periodically, or capping the map size.
2. flashblock_latency_ms() uses .unwrap() on try_into() — At tx_actor.rs the u128 -> u64 conversion uses .try_into().unwrap(). While saturating_sub prevents the result from being huge in practice, if both timestamps are very large u128 values, the difference could theoretically exceed u64::MAX. A try_into().unwrap_or(u64::MAX) would be safer.
3. No reconnection in flashblocks_listener — The PR description mentions "automatic reconnection" but the implementation cancels the token and returns on connection loss. This means a transient WS disconnect kills flashblock tracking for the entire run. Consider a retry loop with backoff.
4. Burst → Delay missed-tick behavior — This is a meaningful behavioral change. Delay means that if a batch takes longer than the interval, subsequent ticks shift forward rather than catching up. This is probably the right call for this use case, but worth noting, perhaps in the changelog, since it changes throughput characteristics under load.
5. Batch start timestamp before HTTP POST — Good fix for accuracy. However, for non-batch (single tx) sends, I don't see the same treatment — the RuntimeTxInfo::default() still uses SystemTime::now() at construction. Worth verifying that single-tx sends also capture the timestamp before the RPC call.

Minor Suggestions

• FlashblockTimeToInclusionChart::echart_data() — Sparse bucket filtering removes empty interior buckets (e.g., if you have 0-100ms and 300-400ms, buckets 1-2 are dropped). This makes the histogram non-contiguous which could be confusing visually. Consider keeping empty buckets between min and max for a proper histogram shape.
• ws_message_to_text — Message::Text(t) => Some(t.to_string()) — t is already a String in tungstenite 0.26, so .to_string() is a no-op clone. Could just use Some(t) if the type matches.
• Heatmap chart with --skip-tx-traces — The tx_gas_used chart shows a "no data" message when empty, but the heatmap chart doesn't get the same treatment. Users with --skip-tx-traces will see an empty heatmap section with no explanation.

[bitwiseguy]

Potential Issues:

1. pending_flashblock_marks unbounded growth — The 20s TTL eviction only runs during RemoveConfirmed flush events. If no blocks are being confirmed (e.g., chain stall), marks from foreign tx hashes could accumulate indefinitely. Consider also evicting in the main loop periodically, or capping the map size.

Capped the map size at 1024.

2. flashblock_latency_ms() uses .unwrap() on try_into() — At tx_actor.rs the u128 -> u64 conversion uses .try_into().unwrap(). While saturating_sub prevents the result from being huge in practice, if both timestamps are very large u128 values, the difference could theoretically exceed u64::MAX. A try_into().unwrap_or(u64::MAX) would be safer.

Done.

3. No reconnection in flashblocks_listener — The PR description mentions "automatic reconnection" but the implementation cancels the token and returns on connection loss. This means a transient WS disconnect kills flashblock tracking for the entire run. Consider a retry loop with backoff.

The pr description was wrong based on the latest implementation. The code now fails and returns an error if the ws connection is broken because it would lose flashblocks and distort results.

4. Burst → Delay missed-tick behavior — This is a meaningful behavioral change. Delay means that if a batch takes longer than the interval, subsequent ticks shift forward rather than catching up. This is probably the right call for this use case, but worth noting, perhaps in the changelog, since it changes throughput characteristics under load.

Yes this behavioral change is intentional. Delay was chosen over Burst specifically because Burst sends missed ticks in a spiky, back-to-back fashion, which often causes nonce races and cascading failures in the mempool (eg mempool full if nonces arrive out of order). Delay maintains steady 1s pacing after any slowdown, ensures every batch fires (deterministic total tx count), and produces an honest average throughput measurement. The tradeoff is the run may take slightly longer than N seconds if a batch overruns, but that's useful signal that the system couldn't sustain the configured rate, or at least had momentary degraded performance, which would also surface via the average Mgas/s value across the run.

5. Batch start timestamp before HTTP POST — Good fix for accuracy. However, for non-batch (single tx) sends, I don't see the same treatment — the RuntimeTxInfo::default() still uses SystemTime::now() at construction. Worth verifying that single-tx sends also capture the timestamp before the RPC call.

Good catch. The single-tx path was relying on RuntimeTxInfo::default() happening to be constructed right before the send, which was correct but fragile.

Fixed by removing Default, new(), and with_start_timestamp() entirely. RuntimeTxInfo now has a single constructor: now(), which captures the system clock at call time. The single-tx path calls now() before the send; the batch path calls now() once before the HTTP POST and clones it for each tx. Both paths use the same API, and there's no way to construct a RuntimeTxInfo with an incorrect timestamp.

Minor Suggestions

• FlashblockTimeToInclusionChart::echart_data() — Sparse bucket filtering removes empty interior buckets (e.g., if you have 0-100ms and 300-400ms, buckets 1-2 are dropped). This makes the histogram non-contiguous which could be confusing visually. Consider keeping empty buckets between min and max for a proper histogram shape.

Done.

• ws_message_to_text — Message::Text(t) => Some(t.to_string()) — t is already a String in tungstenite 0.26, so .to_string() is a no-op clone. Could just use Some(t) if the type matches.

In tungstenite 0.26, Message::Text wraps Utf8Bytes, not String. The .to_string() is a necessary conversion, not a redundant clone.

• Heatmap chart with --skip-tx-traces — The tx_gas_used chart shows a "no data" message when empty, but the heatmap chart doesn't get the same treatment. Users with --skip-tx-traces will see an empty heatmap section with no explanation.

The heatmap already had a "no data" message, but it was misleading. Updated the message