Skip to content

ci(workflows): make test jobs fail honestly on test failures#135

Merged
Andrei Kvapil (kvaps) merged 5 commits into
mainfrom
ci/pipefail-test-jobs
Jun 13, 2026
Merged

ci(workflows): make test jobs fail honestly on test failures#135
Andrei Kvapil (kvaps) merged 5 commits into
mainfrom
ci/pipefail-test-jobs

Conversation

@kvaps

@kvaps Andrei Kvapil (kvaps) commented Jun 11, 2026

Copy link
Copy Markdown
Member

What

The contract and integration test steps in contract.yml, integration.yml, and pull-request.yml pipe go test output through tee for log artifacts. GitHub's default bash -e {0} shell has no pipefail, so the step's exit code was tee's (always 0) — CI stayed green while the contract suite was actually failing.

How it is fixed

Enable set -o pipefail in all four affected run blocks so the jobs propagate go test's exit code. The tee-based log capture for failure artifacts is kept.

These were the only masked go test | … pipes in the workflows (audited all of .github/workflows/).

Note for reviewers

With masking removed, the contract job on this PR is expected to genuinely FAIL due to a known pre-existing corpus divergence (TestOracleTraceReplay layer_data), which is being fixed separately. A red contract check here is the new honest behavior, not a regression introduced by this change.

Summary by CodeRabbit

  • Bug Fixes
    • Updated CI/CD test workflows to correctly fail when tests fail, preventing false successes caused by log output handling masking test failures.

The contract and integration test steps pipe go test output through
tee for log artifacts. Without pipefail the step's exit code is
tee's (always 0), so CI stayed green while the contract suite was
actually failing. Enable pipefail in all four affected run blocks so
the jobs propagate go test's exit code while keeping the tee-based
log capture.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
@gemini-code-assist

Copy link
Copy Markdown

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e169c859-5883-4d98-88ae-60968f7d7738

📥 Commits

Reviewing files that changed from the base of the PR and between c349d79 and 8c01676.

📒 Files selected for processing (3)
  • .github/workflows/contract.yml
  • .github/workflows/integration.yml
  • .github/workflows/pull-request.yml

📝 Walkthrough

Walkthrough

This PR adds set -o pipefail to shell scripts in three GitHub Actions workflows (contract.yml, integration.yml, pull-request.yml) to prevent tee from masking go test exit codes when logging test output to files.

Changes

Pipeline exit code handling

Layer / File(s) Summary
Contract test pipefail
.github/workflows/contract.yml, .github/workflows/pull-request.yml
Contract test steps enable set -o pipefail before piping go test output to contract.log, ensuring failures in the test command propagate correctly.
Integration test pipefail
.github/workflows/integration.yml, .github/workflows/pull-request.yml
Integration test steps enable set -o pipefail before piping go test output to integration.log, ensuring failures in the test command propagate correctly.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

🐰 A pipeline's shell, once let failures hide,
Now pipefail runs with proper pride—
When tee would mask what go test shows,
This bash trick catches all the woes!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately describes the main change: enabling pipefail in CI workflows to ensure test jobs fail when go test fails, not when tee succeeds.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/pipefail-test-jobs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@kvaps Andrei Kvapil (kvaps) merged commit 7389580 into main Jun 13, 2026
15 checks passed
@kvaps Andrei Kvapil (kvaps) deleted the ci/pipefail-test-jobs branch June 13, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant