Fix sliding-window chunked prefill in the gemma4-31B runner

[digantdesai]

The runner chunked prefill at get_max_prefill_chunk = 2sliding_window (2048).
A chunk larger than the window overflows the 2window ring KV cache across
chunk boundaries: after writing a 2048-token chunk the ring holds only the most
recent 2048 positions, so the first ~(chunk - window) queries of every chunk
after the first lose the tail of the previous chunk that is still inside their
1024 window. Those sliding-layer queries then attend over a truncated window,
which propagates into their hidden states and the global KV those positions
write, changing the output. The global flat-cache layers are unaffected.

Cap the prefill chunk at the sliding window: get_sliding_window from metadata
(now exported), else max_prefill/2 since the export sets max_prefill =
2*sliding_window. Decode is unaffected. Adds --max_prefill_chunk to override
the chunk size for testing.

Authored with assistance from Claude Code.

[digantdesai]

Stack from ghstack (oldest at bottom):

• Update tokenizers submodule to the linear-time HF encode path #20349
• Add --prompt_file flag to the eagle3 runner #20348
• Add chunked prefill to the eagle3 runner for prompts past the ring-cache cap #20347
• -> Fix sliding-window chunked prefill in the gemma4-31B runner #20346
• Make EAGLE-3 chain length K a runtime parameter #20345
• Add split-K mid-M SDPA Triton kernel for EAGLE-3 target_verify at long context #20344
• Add EAGLE-3 end-to-end speculative-decode test #20157
• Add the EAGLE-3 speculative-decoding runner (CUDA) #20156
• Add the EAGLE-3 speculator CUDA export #20155
• Add a weight-stationary INT4 GEMM for small M #20154
• Add the Eagle3Speculator module for speculative decoding #20153
• Add KV cache to the EAGLE-3 draft head #20152
• Add Gemma4 EAGLE-3 eager reference #20151
• Add Gemma4 EAGLE-3 hidden-state taps #20150
• Add EAGLE-3 draft head #20149

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20346

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 2 Active SEVs

There are 2 currently active SEVs. If your PR is affected, please view them below:

• [ROCm] MI350 CI runner label rename: rebase PRs using old linux.rocm.gpu.gfx950.* labels
• [Housekeeping] Rename uw1 cluster to meta-prod-aws-uw1

❌ 1 New Failure, 1 Unrelated Failure, 2 Unclassified Failures

As of commit 6415edb with merge base da9158b ():

NEW FAILURE - The following job has failed:

• pull / unittest-editable / windows / windows-job (gh)
  examples/models/test/test_export.py::ExportTest::test_efficient_sam_export_to_executorch

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Build Aarch64 Linux Wheels / pytorch/executorch / build-wheel-py3_10-cpu-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  /__w/executorch/executorch/pytorch/executorch/backends/apple/coreml/runtime/inmemoryfs/inmemory_filesystem.cpp:722:48: error: ‘inmemoryfs::InMemoryFileSystem::InMemoryNode::Kind’ has not been declared
• Build Aarch64 Linux Wheels / pytorch/executorch / upload / upload-wheel-py3_10-cpu-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_aarch64

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh) (trunk failure)

This comment was automatically generated by Dr. CI and updates every 15 minutes.