Problem
When a before_all workspace hook fails for one eval file, the entire eval run aborts — including all other eval files that haven't started yet.
Reproduction
agentv eval run 'examples/**/*.eval.yaml' '!examples/showcase/multi-model-benchmark/**'
If copilot-log-eval's before_all hook (workspace-setup.mjs) fails, no subsequent evals run.
Expected behavior
A before_all failure should:
- Mark all tests in that eval file as errors
- Continue running the remaining eval files
- Report the failure in the summary
Actual behavior
Error: before_all script failed: Script failed: Process exited with code 1
The process exits immediately. No other evals run.
Workaround
Exclude the failing eval from glob patterns using negation:
agentv eval run 'examples/**/*.eval.yaml' '!examples/features/copilot-log-eval/**'