Skip to content

feat: Misc infra#970

Merged
bxyu-nvidia merged 153 commits intomainfrom
bxyu/mmlu-pro
Apr 6, 2026
Merged

feat: Misc infra#970
bxyu-nvidia merged 153 commits intomainfrom
bxyu/mmlu-pro

Conversation

@bxyu-nvidia
Copy link
Copy Markdown
Contributor

@bxyu-nvidia bxyu-nvidia commented Mar 27, 2026

  1. Multi-process benchmark preparation
  2. Various benchmark data preparation refactors
  3. Refactor Nemotron 3 Ultra to be easier to use
  4. Delete key directive
  5. Improve dummy model config handling
  6. Print config yaml when erroring on almost servers
  7. Try fix progress bar print with tqdm (not successful)
  8. Improve broken pipe print and handling behavior
  9. Adopt Cascade eval numpy.isclose for float comparison
  10. LocalVLLMModel accepts py_executable
  11. LocalVLLMModelProxy

Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot bot commented Mar 27, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: Brian Yu <bxyu@nvidia.com>
@bxyu-nvidia bxyu-nvidia marked this pull request as ready for review April 6, 2026 02:08
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot bot commented Apr 6, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Signed-off-by: Brian Yu <bxyu@nvidia.com>
@bxyu-nvidia bxyu-nvidia merged commit d8dd34e into main Apr 6, 2026
5 checks passed
@bxyu-nvidia bxyu-nvidia deleted the bxyu/mmlu-pro branch April 6, 2026 02:27
kajalj22 pushed a commit that referenced this pull request Apr 7, 2026
1. Multi-process benchmark preparation
2. Various benchmark data preparation refactors
3. Refactor Nemotron 3 Ultra to be easier to use
4. Delete key directive
5. Improve dummy model config handling
6. Print config yaml when erroring on almost servers
7. Try fix progress bar print with tqdm (not successful)
8. Improve broken pipe print and handling behavior
9. Adopt Cascade eval numpy.isclose for float comparison
10. LocalVLLMModel accepts py_executable
11. LocalVLLMModelProxy

---------

Signed-off-by: Brian Yu <bxyu@nvidia.com>
cmunley1 pushed a commit that referenced this pull request Apr 8, 2026
1. Multi-process benchmark preparation
2. Various benchmark data preparation refactors
3. Refactor Nemotron 3 Ultra to be easier to use
4. Delete key directive
5. Improve dummy model config handling
6. Print config yaml when erroring on almost servers
7. Try fix progress bar print with tqdm (not successful)
8. Improve broken pipe print and handling behavior
9. Adopt Cascade eval numpy.isclose for float comparison
10. LocalVLLMModel accepts py_executable
11. LocalVLLMModelProxy

---------

Signed-off-by: Brian Yu <bxyu@nvidia.com>
Signed-off-by: cmunley1 <cmunley@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Broken pipe retry message should provide more details about implications

1 participant