Skip to content

Store grouped-only regression baselines and run slow cases in update workflow#346

Merged
harryswift01 merged 2 commits into
mainfrom
345-general-limit-updated-regression-baselines-to-grouped-result-data
May 22, 2026
Merged

Store grouped-only regression baselines and run slow cases in update workflow#346
harryswift01 merged 2 commits into
mainfrom
345-general-limit-updated-regression-baselines-to-grouped-result-data

Conversation

@harryswift01
Copy link
Copy Markdown
Member

Summary

This PR updates the regression baseline workflow so that regenerated baselines only store the grouped result data used by the regression comparison.

It also updates the GitHub workflow for baseline regeneration to include --run-slow, ensuring slow regression cases are included when updating stored baselines.

Changes

Regression baseline output

  • Added a baseline payload helper to persist only the groups section of the CodeEntropy output.
  • Removed runtime arguments and provenance metadata from regenerated baseline files.
  • Keeps normal CodeEntropy output unchanged for users.
  • Keeps regression comparison focused on the grouped entropy values that are actually tested.

GitHub baseline update workflow

  • Added --run-slow to the baseline update command.
  • Ensures slow-marked regression tests are included when regenerating baselines in CI.
  • Avoids accidentally updating only the faster subset of regression baselines.

Impact

  • Regression baseline files are cleaner and more stable.
  • Machine-specific data such as temporary paths, Python versions, platform details, and Git SHAs will no longer be committed in updated baselines.
  • CI baseline updates now cover the full intended regression suite, including slow cases.
  • User-facing JSON output is unaffected.

@harryswift01 harryswift01 added this to the 2.3.0 milestone May 22, 2026
@harryswift01 harryswift01 requested a review from jimboid May 22, 2026 09:19
@harryswift01 harryswift01 self-assigned this May 22, 2026
@harryswift01 harryswift01 added the feature request New feature or request label May 22, 2026
Copy link
Copy Markdown
Member

@jimboid jimboid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, improves the testing base lines workflow.

@harryswift01 harryswift01 merged commit 6eb820c into main May 22, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[General]: Limit updated regression baselines to grouped result data

2 participants