Skip to content

USHIFT-7065: Support MicroShift V5.0 on RHEL 9.8 GA Implementation#6704

Open
agullon wants to merge 2 commits into
openshift:mainfrom
agullon:USHIFT-6499-main
Open

USHIFT-7065: Support MicroShift V5.0 on RHEL 9.8 GA Implementation#6704
agullon wants to merge 2 commits into
openshift:mainfrom
agullon:USHIFT-6499-main

Conversation

@agullon
Copy link
Copy Markdown
Contributor

@agullon agullon commented May 20, 2026

Summary

RHEL 9.8 is now GA. This PR removes pre-GA workarounds from the RHEL 9.8 bootc and ostree CI pipelines.

Bootc pipeline

  • Switch rhel98-test-agent base image from registry.stage.redhat.io to registry.redhat.io
  • Replace --disable-all + OCP mirror repo injection with --enable-eus (standard EUS repos are now available)
  • Switch rhel98-bootc.image-bootc from the localhost/rhel98-test-agent workaround to a direct GA registry reference
  • Fix rhel98-installer.image-installer from rhel-9.7 workaround to rhel-9.8
  • Delete rhel98-mirror.repo (no longer needed)

OStree pipeline

  • Re-enable GPG checks in rhocp-y, rhocp-y1, rhocp-y2, and fast-datapath package sources as indicated by existing TODO comments

configure-composer.sh

  • Remove enable_copr_repositories — RHEL 9.8 ships sufficient osbuild packages natively
  • Remove disable_gpg_check — no longer needed for GA repos on the host OS config
  • Remove enable_ocp_mirror_repositories — OCP mirror was only needed for pre-GA RHEL 9.8 packages
  • Add generation of rhel-9.8.json osbuild-composer config from the host OS template when the host is not yet on RHEL 9.8, so osbuild-composer can build RHEL 9.8 ostree images. GPG check is disabled in this generated config since the RHEL 9.6 host's rpmkeys cannot handle RHEL 9.8's post-quantum key 4 (tracked in USHIFT-7066)

Notes

  • RHEL 10.2 changes are intentionally excluded — the RHEL 9 Bootc Image Builder cannot handle RHEL 10's V6 post-quantum GPG keys. This will be addressed in a follow-up PR once a RHEL 10 BIB image is available.
  • The beta-fallback paths in rhocp-y and rhocp-y1 retain check_gpg = false since those still point to pre-GA mirrors.

Test plan

  • e2e-aws-tests-cache — validates bootc layer1-base and ostree layer1-base builds on x86_64
  • e2e-aws-tests-cache-arm — same on aarch64

🤖 Generated with Claude Code

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 20, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented May 20, 2026

@agullon: This pull request references USHIFT-6499 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

  • Switch bootc base images from registry.stage.redhat.io (staging) to registry.redhat.io (GA) for RHEL 9.8 and 10.2
  • Remove pre-GA workarounds: disabled repos, mirror repo injection, BIB localhost workaround
  • Delete no-longer-needed mirror repo templates (rhel98-mirror.repo, rhel102-mirror.repo)

Test plan

  • CI bootc builds pull RHEL 9.8/10.2 from registry.redhat.io
  • dnf install in containerfiles uses native GA repos
  • BIB ISO generation pulls GA images directly from registry
  • Downstream layers (presubmit, periodic, release) inherit correct repos

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 20, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Removes staging-specific test-agent bootc steps, switches bootc blueprints to upstream RHEL images, enables GPG checks in package-source TOMLs, updates RHEL version references to 9.8 in docs/scripts, and removes OCP-mirror/COPR composer configuration.

Changes

RHEL Bootc and Configuration Updates

Layer / File(s) Summary
Bootc Containerfile & blueprint edits
test/image-blueprints-bootc/el10/layer1-base/group1/rhel102-test-agent.containerfile, test/image-blueprints-bootc/el10/layer1-base/group2/rhel102-bootc.image-bootc, test/image-blueprints-bootc/el9/layer1-base/group1/rhel98-test-agent.containerfile, test/image-blueprints-bootc/el9/layer1-base/group2/rhel98-bootc.image-bootc, test/image-blueprints-bootc/el9/layer1-base/group2/rhel98-installer.image-installer
Executes rpm-repo-config.sh without --disable-all, removes staging-only mirror .repo handling/build args, and points bootc blueprints to upstream registry.redhat.io images.
Enable GPG verification in package sources
test/package-sources/fast-datapath-rhel9.toml, test/package-sources/rhocp-y.toml, test/package-sources/rhocp-y1.toml, test/package-sources/rhocp-y2.toml
Set check_gpg = true in multiple TOML package-source configurations and remove prior explanatory comments.
Docs and automation RHEL 9.8 bump
docs/contributor/devenv_cloud.md, docs/contributor/devenv_setup.md, docs/contributor/howto_quay_mirror.md, scripts/aws/manage_aws_stack.sh, scripts/devenv-builder/manage-vm.sh, test/image-blueprints-bootc/el9/layer1-base/group2/rhel98-installer.image-installer
Update ISO filenames, installer blueprint release, and default OS/version references from older RHEL 9.x values to RHEL 9.8.
Composer configuration simplification
scripts/devenv-builder/configure-composer.sh
Remove OCP-mirror and COPR enablement helpers, stop disabling GPG checks in composer JSON, drop --nobest from osbuild install, and simplify composer setup flow.

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 10 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The PR title references USHIFT-7065 but PR objectives indicate USHIFT-6499; title focuses on 'V5.0' while changes primarily upgrade from RHEL 9.4 to 9.8 and remove staging repos. Clarify whether title should reference USHIFT-6499 (per PR objectives) and verify 'V5.0' is the primary focus versus the RHEL version upgrade.
✅ Passed checks (10 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR contains no Ginkgo test files. Changes are limited to Containerfiles, image blueprints, repo configs, docs, scripts, and TOML files—no Go test definitions to evaluate.
Test Structure And Quality ✅ Passed PR contains no Ginkgo test code to review; changes are limited to Containerfiles, image blueprints, configs, scripts, and docs.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests are added in this PR. All changes (27 modifications, 3 deletions) are configuration, documentation, and script updates for RHEL 9.8/10.2 migration.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests (It(), Describe(), Context(), When()) found. PR changes are configuration, scripts, and documentation only.
Topology-Aware Scheduling Compatibility ✅ Passed PR contains only container build config, repository configs, and scripts—no Kubernetes manifests, operators, or controllers introducing scheduling constraints.
Ote Binary Stdout Contract ✅ Passed PR modifies only Containerfiles, configs, docs, and shell scripts—no Go test binaries or process-level code changed. OTE Binary Stdout Contract check is not applicable.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests are added in this PR. Changes are limited to containerfiles, image blueprints, configuration files, documentation, and shell scripts—no Go test code is present.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from copejon and pacevedom May 20, 2026 10:53
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 20, 2026
@agullon agullon marked this pull request as draft May 20, 2026 10:57
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 20, 2026
@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 20, 2026

/hold until RHEL9.8 and RHEL10.2 are GAed

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 20, 2026
@agullon agullon closed this May 20, 2026
@agullon agullon reopened this May 20, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/aws/manage_aws_stack.sh`:
- Around line 180-182: The help text shows a stale example 'rhel-9.3' that
contradicts the new default rhel-9.8; update the help string that contains
"[--os <os>]:            (create/ami only) specific version of RHEL" so the
example OS value matches the default (use 'rhel-9.8' instead of 'rhel-9.3') and
ensure any nearby documentation lines referencing the default are consistent
with the change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 9fd254aa-b4c3-41fa-8e65-cfbe769e0c92

📥 Commits

Reviewing files that changed from the base of the PR and between c8520cb and bdb616d.

📒 Files selected for processing (11)
  • docs/contributor/devenv_cloud.md
  • docs/contributor/devenv_setup.md
  • docs/contributor/howto_quay_mirror.md
  • scripts/aws/manage_aws_stack.sh
  • scripts/devenv-builder/configure-composer.sh
  • scripts/devenv-builder/manage-vm.sh
  • test/image-blueprints-bootc/el9/layer1-base/group2/rhel98-installer.image-installer
  • test/package-sources/fast-datapath-rhel9.toml
  • test/package-sources/rhocp-y.toml
  • test/package-sources/rhocp-y1.toml
  • test/package-sources/rhocp-y2.toml
💤 Files with no reviewable changes (1)
  • scripts/devenv-builder/configure-composer.sh
✅ Files skipped from review due to trivial changes (3)
  • test/image-blueprints-bootc/el9/layer1-base/group2/rhel98-installer.image-installer
  • docs/contributor/howto_quay_mirror.md
  • docs/contributor/devenv_setup.md

Comment thread scripts/aws/manage_aws_stack.sh
@agullon agullon force-pushed the USHIFT-6499-main branch 3 times, most recently from b987458 to fd7bb9d Compare May 20, 2026 13:31
Comment thread scripts/devenv-builder/configure-composer.sh
@agullon agullon force-pushed the USHIFT-6499-main branch 3 times, most recently from 7633cd7 to a87f79c Compare May 21, 2026 06:34
@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 21, 2026

/hold cancel

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 21, 2026
@agullon agullon marked this pull request as ready for review May 21, 2026 06:38
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 21, 2026
@agullon agullon self-assigned this May 21, 2026
@openshift-ci openshift-ci Bot requested a review from jogeo May 21, 2026 06:39
# Configure RHEL 9.8 repositories for building ostree images when the host
# OS is not yet running RHEL 9.8. Create the config from the host OS template
# with URLs rewritten to point to RHEL 9.8 production repos.
if [[ "${VERSION_ID}" != "9.8" ]]; then
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, create a story in the epic to undo this change when devenv is switched to RHEL 9.8

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created USHIFT-7066 to track this.

@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 21, 2026

/test e2e-aws-tests-cache
/test e2e-aws-tests-cache-arm

@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 25, 2026

/test e2e-aws-tests-cache
/test e2e-aws-tests-cache-arm

1 similar comment
@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 25, 2026

/test e2e-aws-tests-cache
/test e2e-aws-tests-cache-arm

@agullon agullon force-pushed the USHIFT-6499-main branch 2 times, most recently from b38921b to b80486a Compare May 25, 2026 08:55
@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 25, 2026

/test e2e-aws-tests-cache

@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 25, 2026

/test e2e-aws-tests-cache-arm

RHEL 9.8 is now GA. Replace staging registry reference with the
official registry.redhat.io image and remove pre-GA workarounds
(disabled repos, mirror repo injection, BIB localhost workaround).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

pre-commit.check-secrets: ENABLED
@agullon agullon force-pushed the USHIFT-6499-main branch 2 times, most recently from 3c36198 to 35eb920 Compare May 25, 2026 09:25
@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 25, 2026

/test e2e-aws-tests-cache

@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 25, 2026

/test e2e-aws-tests-cache-arm

@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 25, 2026

/test e2e-aws-tests-cache
/test e2e-aws-tests-cache-arm

@agullon agullon changed the title USHIFT-6499: Switch RHEL 9.8 and 10.2 bootc images to GA repos USHIFT-7065: Switch RHEL 9.8 and 10.2 bootc images to GA repos May 25, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented May 25, 2026

@agullon: This pull request references USHIFT-7065 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

RHEL 9.8 is now GA. This PR removes pre-GA workarounds from the bootc and ostree CI pipelines.

Bootc pipeline (test/image-blueprints-bootc/el9/)

  • Switch rhel98-test-agent base image from registry.stage.redhat.io to registry.redhat.io
  • Replace --disable-all + OCP mirror repo injection with --enable-eus (standard EUS repos)
  • Switch rhel98-bootc.image-bootc from localhost/rhel98-test-agent:latest workaround to direct registry reference
  • Fix rhel98-installer.image-installer from rhel-9.7 workaround to rhel-9.8
  • Delete rhel98-mirror.repo (no longer needed)

OStree pipeline (test/package-sources/)

  • Re-enable GPG checks in rhocp-y, rhocp-y1, rhocp-y2, and fast-datapath package sources, resolving existing TODO comments

DevEnv / composer setup (scripts/devenv-builder/configure-composer.sh)

  • Remove enable_copr_repositories — RHEL 9.8 ships sufficient osbuild packages natively
  • Remove disable_gpg_check — no longer needed for GA repos
  • Remove enable_ocp_mirror_repositories — OCP mirror was only needed for pre-GA RHEL 9.8 packages
  • Add rhel-9.8.json osbuild-composer config generation from the host OS template, so osbuild-composer on pre-9.8 hosts can build RHEL 9.8 ostree images. GPG check is disabled in this generated config because the RHEL 9.6 host's rpmkeys cannot handle RHEL 9.8's post-quantum key 4 until the host is upgraded.

Notes

  • RHEL 10.2 changes are intentionally excluded — the RHEL 9 Bootc Image Builder cannot handle RHEL 10's V6 post-quantum GPG keys. A separate PR will address 10.2 once a RHEL 10 BIB image is available.
  • The rhocp-y beta-fallback paths retain check_gpg = false since those still point to pre-GA mirrors.

Test plan

  • e2e-aws-tests-cache passes — validates bootc layer1-base and ostree layer1-base builds
  • e2e-aws-tests-cache-arm passes — same validation on aarch64

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@agullon agullon marked this pull request as ready for review May 25, 2026 12:33
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 25, 2026
@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 25, 2026

/hold cancel

Now that RHEL 9.8 is GA:
- Re-enable GPG checks in ostree package sources (rhocp-y, rhocp-y1,
  rhocp-y2, fast-datapath) as indicated by existing TODO comments
- Fix rhel98-installer to use rhel-9.8 instead of rhel-9.7 workaround
- Remove disable_gpg_check and enable_ocp_mirror_repositories from
  configure-composer.sh — both were pre-GA RHEL workarounds
- Create rhel-9.8.json osbuild-composer config from the host OS template
  on pre-9.8 hosts so osbuild-composer recognizes rhel-9.8 as a valid
  distro for building ostree images

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

pre-commit.check-secrets: ENABLED
@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 25, 2026
@agullon agullon force-pushed the USHIFT-6499-main branch from 35eb920 to b900600 Compare May 25, 2026 12:33
@openshift-ci openshift-ci Bot requested a review from vanhalenar May 25, 2026 12:35
@agullon agullon changed the title USHIFT-7065: Switch RHEL 9.8 and 10.2 bootc images to GA repos USHIFT-7065: Support MicroShift V5.0 on RHEL 9.8 GA Implementation May 25, 2026
@agullon agullon requested a review from ggiguash May 25, 2026 12:37
@agullon
Copy link
Copy Markdown
Contributor Author

agullon commented May 25, 2026

/test e2e-aws-tests-cache
/test e2e-aws-tests-cache-arm

Copy link
Copy Markdown
Contributor

@pacevedom pacevedom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@pacevedom
Copy link
Copy Markdown
Contributor

/hold
Need to check the AMI

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 25, 2026
@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 25, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 25, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: agullon, pacevedom

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants