[WIP] Initial work to add llm-d-vllm framework with H200 #1660

Open

ezrasilvera wants to merge 8 commits into

SemiAnalysisAI:mainfrom

ezrasilvera:llm-d-initial

Commits on Jun 4, 2026

[NV] llm-d: add llm-d-vllm framework with H200 wide-EP and simple 1P+1D recipes
ezrasilvera
committed
[NV] llm-d: address PR #1660 review (C1 sweep, C2 health waits, C3 NVSHMEM gate)
ezrasilvera
committed
[NV] llm-d: remove H200 wide-EP entry, keep simple 1P+1D
ezrasilvera
committed
[NV] llm-d: address PR #1660 review (C4 export slurm vars, C5 recipe path, C7 docker filter, C8 EPP wait)
ezrasilvera
committed
[NV] llm-d: address PR #1660 review (C9 marker-file teardown instead of in-container scancel)
ezrasilvera
committed
[NV] llm-d: address PR #1660 review (C10 auto-detect default NIC for NCCL/GLOO/NVSHMEM)
ezrasilvera
committed
[NV] llm-d: address PR #1660 review (C11 split MODEL into path and served-name)
ezrasilvera
committed
[NV] llm-d: have every rank wait for its own vLLM /health before falling through
ezrasilvera
committed