[OMNIML-4869] author_yaml

[ChenhanYu]

Draft PR opened by pensieve-intern for OMNIML-4869.

Stage author_yaml of Epic OMNIML-4868. The agent ran from the SPEC on the ticket description; review every change before marking ready.

Always-draft is enforced — the bot never auto-merges.

---

Agent's self-narration (stripped from PR diff; surfaced here for context):

VERIFICATION_COMMENT.txt:

OMNIML-4869 status: model not staged on cw_dfw.

Spec checks:
- HF Hub model exists: Qwen/Qwen3.5-4B (public, not gated; model_type=qwen3_5).
- Cluster stage check failed: /lustre/fsw/portfolios/coreai/projects/coreai_dlalgo_modelopt/hf-local/Qwen/Qwen3.5-4B not found.

Requested action on Epic: please stage the model via `/manage-assets cw_dfw get Qwen/Qwen3.5-4B`.

Notes:
- YAML already exists at tools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench_mtp_vllm.yaml with tp_size=1, gpus_per_node=1, and container vllm/vllm-openai:qwen3_5-cu130.
- Once staged, I can run the required dry-run: `uv run launch.py --yaml tools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench_mtp_vllm.yaml --dryrun --yes -v`.

Pollution-strip removed VERIFICATION_COMMENT.txt from this commit (sidecar narration and/or incidental lockfile regeneration are never part of the agent's intended deliverable).

Summary by CodeRabbit

• Chores
  • Added a benchmark configuration to evaluate speculative decoding (MTP) performance for the Qwen3.5-4B model, enabling separate speed and high-throughput (32k) runs with adjustable decoding and runtime parameters.
  • Configured execution settings for single-node GPU benchmarking and standardized container/runtime invocation for reproducible performance tests.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d7290bb9-4d26-462c-875c-c7d41eb2e59c

📥 Commits

Reviewing files that changed from the base of the PR and between 3e950a0 and cfc31b7.

📒 Files selected for processing (1)

• tools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench_mtp_vllm.yaml

🚧 Files skipped from review as they are similar to previous changes (1)

• tools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench_mtp_vllm.yaml

---

📝 Walkthrough

Walkthrough

This PR adds a new Slurm/vLLM YAML benchmark for Qwen3.5-4B using speculative decoding (MTP), defining a shared model checkpoint and two SPEED benchmark tasks (qualitative and throughput_32k) with per-task vLLM runtime args, environment variables, and single-node Slurm container settings.

Changes

Qwen3.5-4B SPECDEC Benchmark Configuration

Layer / File(s)	Summary
Benchmark specification with dual tasks and Slurm execution tools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench_mtp_vllm.yaml	Complete YAML configuration defining shared hf_model checkpoint, two benchmark tasks (SPEED qualitative and SPEED throughput_32k) with distinct runtime parameters and environment variables, vLLM engine arguments including speculative decoding (MTP) settings, and Slurm container execution configuration using single-node, single-task, single-GPU setup.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Suggested reviewers

• kevalmorabia97
• h-guo18

---

Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

• Ignore

❌ Failed checks (1 error, 1 inconclusive)

Check name	Status	Explanation	Resolution
Security Anti-Patterns	❌ Error	PR adds 19 # nosec comments to bypass Bandit checks, violating SECURITY.md rule 5. These comments are explicitly forbidden per security policy.	Remove all # nosec comments. For legitimate security patterns, add inline comments and request codeowners review with PR justification.
Title check	❓ Inconclusive	The PR title '[OMNIML-4869] author_yaml' is vague and does not clearly describe the actual change—adding a Qwen3.5-4B SPECDEC benchmark configuration file—making it unclear to someone reviewing PR history.	Use a more descriptive title that conveys the main change, such as 'Add Qwen3.5-4B speculative decoding benchmark configuration' or 'Add specdec_bench_mtp_vllm.yaml for Qwen3.5-4B'.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch pensieve-intern/OMNIML-4868/author-yaml

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[ChenhanYu]

/ok to test 3e950a0

[ChenhanYu]

/ok to test cfc31b7

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.82%. Comparing base (5eba879) to head (cfc31b7).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1574      +/-   ##
==========================================
- Coverage   76.88%   75.82%   -1.07%     
==========================================
  Files         478      478              
  Lines       52209    54286    +2077     
==========================================
+ Hits        40140    41161    +1021     
- Misses      12069    13125    +1056     

Flag	Coverage Δ
regression	15.18% <ø> (+<0.01%)	⬆️
unit	53.61% <ø> (+0.09%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ChenhanYu]

Review request: this is the agent-authored YAML for OMNIML-4868 (inaugural test of the specdec_bench pensieve-intern workflow). Single new file under tools/launcher/examples/Qwen/Qwen3.5-4B/, follows the same shape we just merged in #1531 / #1564. Once this lands, it unblocks the next workflow stage on the cluster.

FYI @cjluo-nv @kevalmorabia97 @h-guo18 @yeyu-nvidia — would appreciate a quick approval.

codecov/project failure is the recurring coverage-delta noise we hit on PR 1564 too (no tests cover a config YAML); it does not affect runtime.

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-05-31 06:23 UTC