ci: remove nick-fields/retry wrapper and add shared sinfo-based GPU partition selection#1299

Merged

sbryngelson merged 29 commits intoMFlowCode:masterfrom

sbryngelson:retryci

Mar 11, 2026

Commits on Mar 10, 2026

ci: replace nick-fields/retry with plain run step; deprioritize v100 nodes
Spencer Bryngelson
committed
ci: remove unused RETRY_VALIDATE_CMD from retry_build
Spencer Bryngelson
committed
ci: shared sinfo-based GPU partition selection for tests and benchmarks
Spencer Bryngelson
committed
bench: update Phoenix tmpbuild path to project storage

Spencer Bryngelson
and
claude
committed
bench: require 2 idle/mix nodes for parallel benchmark GPU partition selection

Spencer Bryngelson
and
claude
committed
ci: restore RETRY_VALIDATE_CMD support in retry_build

Spencer Bryngelson
and
claude
committed
ci: exclude dead GPU node atl1-1-03-002-29-0 (cuInit error 999)

Spencer Bryngelson
and
claude
committed
ci: unify job submission, test, and bench scripts across clusters

Spencer Bryngelson
and
claude
committed
ci: remove dead rm -rf build, add gpu_partition default

Spencer Bryngelson
and
claude
committed
ci: use strict shell mode in common/test.sh and common/bench.sh

Spencer Bryngelson
and
claude
committed
ci: validate existing Phoenix build against node ISA before reuse

Spencer Bryngelson
and
claude
committed
ci: always nuke build/ on Phoenix to avoid stale ISA mismatches

Spencer Bryngelson
and
claude
committed
ci: nuke stale Phoenix bench builds; trap TMPDIR cleanup on exit

Spencer Bryngelson
and
claude
committed
ci: widen TMPDIR random range to reduce collision risk

Spencer Bryngelson
and
claude
committed
ci: remove no-op -j flag from mfc.sh bench invocations

Spencer Bryngelson
and
claude
committed
ci: add space after SBATCH -o for consistency with SLURM docs

Spencer Bryngelson
and
claude
committed
ci: fix bench master job failing to find common/bench.sh

sbryngelson
and
claude
committed
ci: drop gpu-rtx6000 from partition list (too slow for test time limit)

sbryngelson
and
claude
committed
ci: deprioritize gpu-v100 to last in partition selection

sbryngelson
and
claude
committed
ci: pass build_opts (GPU interface flag) to live test command

sbryngelson
and
claude
committed
ci: fix wait + set -e race that orphans parallel jobs

sbryngelson
and
claude
committed
ci: fix stale comments from incremental refactoring

sbryngelson
and
claude
committed
ci: fix duplicate --gpu flag on Frontier GPU test commands

sbryngelson
and
claude
committed
ci: remove stale .out file before SLURM submission

sbryngelson
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: remove nick-fields/retry wrapper and add shared sinfo-based GPU partition selection#1299

ci: remove nick-fields/retry wrapper and add shared sinfo-based GPU partition selection#1299
sbryngelson merged 29 commits intoMFlowCode:masterfrom
sbryngelson:retryci

Commits on Mar 10, 2026

ci: replace nick-fields/retry with plain run step; deprioritize v100 nodes

ci: remove unused RETRY_VALIDATE_CMD from retry_build

ci: shared sinfo-based GPU partition selection for tests and benchmarks

bench: update Phoenix tmpbuild path to project storage

bench: require 2 idle/mix nodes for parallel benchmark GPU partition selection

ci: restore RETRY_VALIDATE_CMD support in retry_build

ci: exclude dead GPU node atl1-1-03-002-29-0 (cuInit error 999)

ci: unify job submission, test, and bench scripts across clusters

ci: remove dead rm -rf build, add gpu_partition default

ci: use strict shell mode in common/test.sh and common/bench.sh

ci: validate existing Phoenix build against node ISA before reuse

ci: always nuke build/ on Phoenix to avoid stale ISA mismatches

ci: nuke stale Phoenix bench builds; trap TMPDIR cleanup on exit

ci: widen TMPDIR random range to reduce collision risk

ci: remove no-op -j flag from mfc.sh bench invocations

ci: add space after SBATCH -o for consistency with SLURM docs

ci: fix bench master job failing to find common/bench.sh

ci: drop gpu-rtx6000 from partition list (too slow for test time limit)

ci: deprioritize gpu-v100 to last in partition selection

ci: pass build_opts (GPU interface flag) to live test command

ci: fix wait + set -e race that orphans parallel jobs

ci: fix stale comments from incremental refactoring

ci: fix duplicate --gpu flag on Frontier GPU test commands

ci: remove stale .out file before SLURM submission

Commits on Mar 11, 2026

ci: cap case-optimization build jobs at 8 to match prebuild

ci: submit case-opt prebuild to CPU partition (no GPU needed for compilation)

ci: add CPU architecture to build cache key to prevent SIGILL

ci: remove build cache from GitHub runners (prevents SIGILL, negligible speedup)

ci: switch Frontier to service partition and develop QOS