Skip to content

Add CI job for Y-stream version skew e2e test#80315

Open
weliang1 wants to merge 2 commits into
openshift:mainfrom
weliang1:automate-ystream-version-skew
Open

Add CI job for Y-stream version skew e2e test#80315
weliang1 wants to merge 2 commits into
openshift:mainfrom
weliang1:automate-ystream-version-skew

Conversation

@weliang1

@weliang1 weliang1 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds a weekly periodic CI job to validate Y-stream version skew support in HyperShift hosted clusters.

Job Details

Job Name: e2e-aws-ystream-version-skew
Schedule: Weekly on Mondays at 6:00 AM UTC
Platform: AWS
Test: TestYStreamVersionSkew

Test Coverage:

  • Creates hosted cluster at OCP 4.21.x
  • Upgrades control plane to 4.22.x
  • Creates mixed-version data plane (some NodePools at 4.21, some at 4.22)
  • Validates cluster stability across version skew
  • Multi-AZ deployment testing

Configuration

- as: e2e-aws-ystream-version-skew
  cron: 0 6 * * 1
  steps:
    cluster_profile: hypershift-aws
    dependencies:
      OCP_IMAGE_LATEST: release:latest
      OCP_IMAGE_PREVIOUS: release:latest-4-20
    env:
      HYPERSHIFT_E2E_TEST: TestYStreamVersionSkew
      TEST_ARGS: --e2e.test-ystream-version-skew
      TEST_SUITE: hypershift
    test:
    - ref: hypershift-e2e-test
    workflow: hypershift-aws-e2e

Testing

This PR depends on test code from openshift/hypershift#8707

To test both changes together:

/payload-with-prs openshift/hypershift#8707
/test e2e-aws-ystream-version-skew

Related

Files Changed

  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.22__periodics.yaml - Added periodic job definition

Dependencies:

Summary by CodeRabbit

This PR updates the OpenShift CI configuration for the HyperShift repository by adding a new weekly periodic job to the 4.22 release periodics that validates Y-stream version-skew behavior for hosted clusters on AWS.

Practical effect:

  • Adds a periodic job named e2e-aws-ystream-version-skew (cron: 0 6 * * 1, weekly Mondays 06:00 UTC) to ci-operator config for the openshift/hypershift 4.22 periodics.
  • Targets AWS hosted-cluster e2e tests using the hypershift-aws-e2e-nested workflow.
  • Runs the Y-stream version-skew test flow (CI_TESTS_RUN/TestYStreamVersionSkew) which:
    • Creates a hosted cluster at OCP 4.21.x,
    • Upgrades the control plane to 4.22.x,
    • Provisions a mixed-version data plane (some NodePools at 4.21, some at 4.22),
    • Validates cluster stability, networking, and multi-AZ behavior across the skew.
  • Uses image pins OCP_IMAGE_LATEST: release:latest and OCP_IMAGE_PREVIOUS: release:latest-4-20.

Additional notes:

Impact:

  • Minimal config change (periodic job addition) that expands HyperShift CI coverage to automatically exercise control-plane/data-plane minor-version skew scenarios on AWS.

Adds periodic CI job to run Y-stream version skew testing for HyperShift.

Test configuration:
- Runs weekly (Monday 6:00 AM)
- Uses OCP 4.22 (latest) and 4.21 (previous)
- Executes TestYStreamVersionSkew from hypershift e2e suite
- Verifies control plane and data plane version skew support

This job validates CORENET-6787 (Y-stream skew support) by:
- Creating cluster at 4.21.15
- Upgrading control plane to 4.22.0-rc.4
- Gradually upgrading NodePools to create mixed versions
- Testing networking across version boundaries

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@weliang1

weliang1 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

/payload-with-prs openshift/hypershift#8707

@openshift-ci

openshift-ci Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

@weliang1: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

@weliang1

weliang1 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

/test e2e-aws-ystream-version-skew

@openshift-ci

openshift-ci Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

@weliang1: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test boskos-config
/test boskos-config-generation
/test check-gh-automation
/test check-gh-automation-tide
/test check-trigger-trusted-apps
/test ci-operator-config
/test ci-operator-config-metadata
/test ci-operator-registry
/test ci-secret-bootstrap-config-validation
/test ci-testgrid-allow-list
/test clusterimageset-validate
/test config
/test core-valid
/test generated-config
/test generated-dashboards
/test hyperfleet-risk-scorer-test
/test image-mirroring-config-validation
/test jira-lifecycle-config
/test labels
/test openshift-image-mirror-mappings
/test ordered-prow-config
/test owners
/test pr-reminder-config
/test prow-config
/test prow-config-filenames
/test prow-config-semantics
/test pylint
/test release-config
/test release-controller-config
/test rover-groups-config-validation
/test secret-generator-config-valid
/test services-valid
/test stackrox-stackrox-stackrox-stackrox-check
/test step-registry-metadata
/test step-registry-shellcheck
/test sync-rover-groups
/test verified-config
/test yamllint

The following commands are available to trigger optional jobs:

/test check-cluster-profiles-config

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-release-check-gh-automation
pull-ci-openshift-release-main-ci-operator-config
pull-ci-openshift-release-main-ci-operator-config-metadata
pull-ci-openshift-release-main-ci-operator-registry
pull-ci-openshift-release-main-core-valid
pull-ci-openshift-release-main-generated-config
pull-ci-openshift-release-main-ordered-prow-config
pull-ci-openshift-release-main-owners
pull-ci-openshift-release-main-prow-config-filenames
pull-ci-openshift-release-main-release-controller-config
pull-ci-openshift-release-openshift-image-mirror-mappings
pull-ci-openshift-release-yamllint
Details

In response to this:

/test e2e-aws-ystream-version-skew

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci openshift-ci Bot requested review from Nirshal and clebs June 9, 2026 20:44
@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jun 9, 2026
@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: c4513478-f3bd-4568-9966-b2ae5b7bb33f

📥 Commits

Reviewing files that changed from the base of the PR and between e52780f and 28c1c0a.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.22-periodics.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (1)
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.22__periodics.yaml

Walkthrough

Adds a new periodic hypershift e2e test entry e2e-aws-ystream-version-skew to the 4.22 periodics: scheduled at 0 6 * * 1, using hypershift-aws cluster profile, images release:latest and release:latest-4-20, CI_TESTS_RUN=TestYStreamVersionSkew, and workflow hypershift-aws-e2e-nested.

Changes

Hypershift periodic test configuration

Layer / File(s) Summary
e2e-aws-ystream-version-skew periodic test definition
ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.22__periodics.yaml
New periodic test entry with cron 0 6 * * 1, cluster_profile: hypershift-aws, OCP_IMAGE_LATEST: release:latest, OCP_IMAGE_PREVIOUS: release:latest-4-20, CI_TESTS_RUN: TestYStreamVersionSkew, and workflow: hypershift-aws-e2e-nested.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • openshift/release#80128: Related hypershift e2e periodic test configuration changes touching hypershift-aws-e2e-nested env and CI_TESTS_RUN settings.

Suggested reviewers

  • devguyio
  • csrwng
🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding a CI job for Y-stream version skew e2e testing in HyperShift, which aligns with the single new periodic job entry added to the configuration.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR only modifies YAML CI configuration files, not Ginkgo test code. The check for stable test names is not applicable as no test code changes are present.
Test Structure And Quality ✅ Passed Check is not applicable: PR adds only YAML CI configuration for a periodic test job, not Ginkgo test code. The check requires reviewing actual test implementation patterns.
Microshift Test Compatibility ✅ Passed PR only adds CI job configuration (YAML) referencing existing test; no new Ginkgo test code (It/Describe/Context) is added. Check applies only to test code additions.
Single Node Openshift (Sno) Test Compatibility ✅ Passed This PR adds only a YAML CI job configuration and does not add any new Ginkgo e2e test definitions. The SNO compatibility check applies only to new test code, not to CI configuration files.
Topology-Aware Scheduling Compatibility ✅ Passed PR adds only a CI periodic job config entry to a YAML file, not deployment manifests, operator code, or workload controllers. No scheduling constraints are introduced.
Ote Binary Stdout Contract ✅ Passed The PR only modifies CI job YAML configuration, not Go source code. OTE Binary Stdout Contract check applies to Go binaries only, making it inapplicable.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR adds only CI job configuration (YAML), not new Ginkgo e2e test code. The check for IPv6/disconnected network assumptions applies only to new test code, not CI configuration files.
No-Weak-Crypto ✅ Passed PR adds only YAML CI job configuration with no cryptographic code, weak crypto patterns, or secret/token comparisons.
Container-Privileges ✅ Passed File is CI/Prow job config with no container/K8s manifests; no privileged/hostPID/hostNetwork/hostIPC/SYS_ADMIN/allowPrivilegeEscalation settings found.
No-Sensitive-Data-In-Logs ✅ Passed The added job configuration contains only non-sensitive CI metadata: schedule, profile/workflow names, and release references. No passwords, tokens, API keys, PII, or credentials are present.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.22__periodics.yaml`:
- Around line 342-355: The new periodic job configuration for
e2e-aws-ystream-version-skew (env entries TestYStreamVersionSkew and flag
--e2e.test-ystream-version-skew) was added to the source config but the
generated prow job YAML wasn't updated/committed; run the repository generation
step (make update) to regenerate ci-operator job outputs so the periodic appears
in the final openshift-hypershift-release-4.22-periodics.yaml, verify the
generated job includes the e2e-aws-ystream-version-skew entry and the
TestYStreamVersionSkew/--e2e.test-ystream-version-skew occurrences, and commit
the generated YAML (or alternatively fix the source config so generation will
pick it up) before pushing the PR.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 0a0f2f99-ee1e-468c-bc69-982e22adca0e

📥 Commits

Reviewing files that changed from the base of the PR and between 7a0b697 and e52780f.

📒 Files selected for processing (1)
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.22__periodics.yaml

- Fixed workflow name: hypershift-aws-e2e-external -> hypershift-aws-e2e-nested
- Use CI_TESTS_RUN env var instead of HYPERSHIFT_E2E_TEST
- Use release:latest-4-20 instead of release:n1minor
- Regenerated Prow job configurations with make update
@openshift-ci

openshift-ci Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: weliang1
Once this PR has been reviewed and has the lgtm label, please assign csrwng for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot Bot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jun 9, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@weliang1: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
periodic-ci-openshift-hypershift-release-4.22-periodics-e2e-aws-ystream-version-skew N/A periodic Periodic changed
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@weliang1

weliang1 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

/test e2e-aws-ystream-version-skew

@openshift-ci

openshift-ci Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

@weliang1: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test boskos-config
/test boskos-config-generation
/test check-gh-automation
/test check-gh-automation-tide
/test check-trigger-trusted-apps
/test ci-operator-config
/test ci-operator-config-metadata
/test ci-operator-registry
/test ci-secret-bootstrap-config-validation
/test ci-testgrid-allow-list
/test clusterimageset-validate
/test config
/test core-valid
/test generated-config
/test generated-dashboards
/test hyperfleet-risk-scorer-test
/test image-mirroring-config-validation
/test jira-lifecycle-config
/test labels
/test openshift-image-mirror-mappings
/test ordered-prow-config
/test owners
/test pr-reminder-config
/test prow-config
/test prow-config-filenames
/test prow-config-semantics
/test pylint
/test release-config
/test release-controller-config
/test rover-groups-config-validation
/test secret-generator-config-valid
/test services-valid
/test stackrox-stackrox-stackrox-stackrox-check
/test step-registry-metadata
/test step-registry-shellcheck
/test sync-rover-groups
/test verified-config
/test yamllint

The following commands are available to trigger optional jobs:

/test check-cluster-profiles-config

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-release-check-gh-automation
pull-ci-openshift-release-main-ci-operator-config
pull-ci-openshift-release-main-ci-operator-config-metadata
pull-ci-openshift-release-main-ci-operator-registry
pull-ci-openshift-release-main-config
pull-ci-openshift-release-main-core-valid
pull-ci-openshift-release-main-generated-config
pull-ci-openshift-release-main-ordered-prow-config
pull-ci-openshift-release-main-owners
pull-ci-openshift-release-main-prow-config-filenames
pull-ci-openshift-release-main-prow-config-semantics
pull-ci-openshift-release-main-release-controller-config
pull-ci-openshift-release-openshift-image-mirror-mappings
pull-ci-openshift-release-yamllint
Details

In response to this:

/test e2e-aws-ystream-version-skew

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@weliang1

weliang1 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse periodic-ci-openshift-hypershift-release-4.22-periodics-e2e-aws-ystream-version-skew

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@weliang1: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci

openshift-ci Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

@weliang1: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant