dsv4-fp4-b300-sglang: align env vars to GB300#1682
Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
1 similar comment
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — straightforward benchmark recipe alignment (image bump, env var consolidation, fp4-indexer flag).
Extended reasoning...
Overview
PR touches two files: a benchmark shell script (benchmarks/single_node/fixed_seq_len/dsv4_fp4_b300_sglang.sh) and a CI config yaml (.github/configs/nvidia-master.yaml). Changes are an image bump (nightly-dev-cu13-20260608-303757cc), addition of --enable-deepseek-v4-fp4-indexer to every concurrency profile, and a refactor that lifts shared DP-attention env vars (SGLANG_OPT_SWA_EVICT_DROP_PAGE_MARGIN, SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW, SGLANG_OPT_USE_ONLINE_COMPRESS, SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK, etc.) into a single block plus a few new vars aligned to GB300.
Security risks
None. This is benchmark/recipe code that runs in CI sweep contexts. No auth, crypto, permissions, network exposure, or user input handling is touched.
Level of scrutiny
Low. This is benchmark configuration aligning a single-node B300 SGLang recipe to its GB300 sibling. Worst case is a tuning regression on that one sweep, which is self-contained and discoverable from sweep results.
Other factors
The bug hunting system found nothing. The refactor is intentional: previously each per-CONC block redundantly set SWA_EVICT_DROP_PAGE_MARGIN, SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW, and DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK (with slightly different values 8320/8320/8256); the consolidated block uses 8192 across the board per GB300 alignment, which is the stated goal. The CONC=512 profile (flashinfer_mxfp4) now also picks up megamoe-specific env vars, but they're inert for non-megamoe backends and consistent with the GB300 reference recipe.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27128239626 |
5 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27128239626 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27128239626 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27128239626 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27128239626 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27128239626 |
yhyang201
added
the
non-canary-full-sweep-enabled
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6befeb8. Configure here.
| # Uses nightly image with megamoe backend for high-concurrency profiles. | ||
| dsv4-fp4-b300-sglang: | ||
| image: lmsysorg/sglang:nightly-dev-cu13-20260529-a8cfae0b | ||
| image: lmsysorg/sglang:nightly-dev-cu13-20260606-b3e4c204 |
There was a problem hiding this comment.
Changelog image mismatches config
Medium Severity
dsv4-fp4-b300-sglang runs lmsysorg/sglang:nightly-dev-cu13-20260606-b3e4c204, while perf-changelog.yaml and the PR summary document nightly-dev-cu13-20260608-303757cc. Benchmarks and release notes disagree on which SGLang build is in use.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 6befeb8. Configure here.
yhyang201
removed
the
non-canary-full-sweep-enabled
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27152512947 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27152636345 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27154740922 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27156930191 |
1 participant
Summary
--enable-deepseek-v4-fp4-indexerflag to all concurrency profilesnightly-dev-cu13-20260608-303757ccNote
Low Risk
Benchmark and launch-script tuning only; no application auth, data, or production serving paths are modified.
Overview
Aligns DeepSeek-V4 FP4 B300 single-node SGLang benchmark launch settings with the GB300 disaggregated recipes so perf sweeps use the same runtime tuning (excluding NCCL/comm).
In
dsv4_fp4_b300_sglang.sh, common env swapsSGLANG_JIT_DEEPGEMM_PRECOMPILE=0forFAST_WARMUP=1and addsRADIX_FORCE_MISS,DEFAULT_THINKING, andDSV4_REASONING_EFFORT=max. DP-attention profiles (CONCnot 1/32) now set a shared block of MegaMoE/SWA/logging env (including unifiedNUM_MAX_TOKENS_PER_RANK=8192) instead of repeating overrides per concurrency tier; high-conc blocks drop the old per-profile token-rank values.--enable-deepseek-v4-fp4-indexeris added to every concurrencyPARALLEL_ARGSprofile.nvidia-master.yamlbumps thedsv4-fp4-b300-sglangcontainer image to a newer nightly SGLang tag;perf-changelog.yamlrecords the config change.Reviewed by Cursor Bugbot for commit 80a03ba. Bugbot is set up for automated code reviews on this repo. Configure here.