dsv4-fp4-gb300-dynamo-trt: STP + MTP disagg trtllm recipes on GB300#1689
dsv4-fp4-gb300-dynamo-trt: STP + MTP disagg trtllm recipes on GB300#1689Ankur-singh wants to merge 2 commits into
Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
Summary
Adds disaggregated TRT-LLM inference benchmarks for DeepSeek-V4-Pro MXFP4 on
GB300 via the Dynamo frontend, covering both STP and MTP configurations.
Changes
.github/configs/nvidia-master.yaml— two new config keys:dsv4-fp4-gb300-dynamo-trt(STP): 27 scenarios, ISL 1024/OSL 1024 (14 conc points) + ISL 8192/OSL 1024 (13 conc points)dsv4-fp4-gb300-dynamo-trt-mtp(MTP): 27 scenarios, same sequence-length coveragenvcr.io#nvidia/ai-dynamo/tensorrtllm-runtime:1.3.0-deepseek-v4-dev.1NVIDIA/srt-slurmbranchsa-submission-q2-2026runners/launch_gb300-nv.sh— new branch forframework=dynamo-trt + model_prefix=dsv4; overridesSRT_SLURM_MODEL_PREFIXtodeepseek-ai/DeepSeek-V4-Proto match the recipe'smodel.path(HuggingFace-style), clonesNVIDIA/srt-slurm@sa-submission-q2-2026.perf-changelog.yaml— entry appended at end documenting both keys; PR link backfilled after open.Note
Low Risk
Benchmark and CI runner configuration only; no application runtime or auth logic changes.
Overview
Adds GB300 disaggregated DeepSeek-V4-Pro MXFP4 benchmark coverage for TensorRT-LLM via Dynamo, in both STP and MTP variants.
nvidia-master.yamlgainsdsv4-fp4-gb300-dynamo-trtanddsv4-fp4-gb300-dynamo-trt-mtp(54 scenarios total: 1k/1k and 8k/1k fixed-seq-len sweeps with prefill/decode parallelism andCONFIG_FILEpointers toNVIDIA/srt-slurmsa-submission-q2-2026recipes). Image:tensorrtllm-runtime:1.3.0-deepseek-v4-dev.1.launch_gb300-nv.shadds adynamo-trt+dsv4path that setsSRT_SLURM_MODEL_PREFIXto the HuggingFace model id and checks out that srt-slurm branch.perf-changelog.yamldocuments the new config keys.Reviewed by Cursor Bugbot for commit e2dd5ca. Bugbot is set up for automated code reviews on this repo. Configure here.