Skip to content

USHIFT-6491: Improve gitops test #6116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
agullon wants to merge 1 commit into
base: main
Choose a base branch
from
+106 −31
Open

USHIFT-6491: Improve gitops test #6116

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 38 additions & 4 deletions test/bin/pyutils/generate_common_versions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import argparse
import logging
import pathlib
import time

sys.path.append(str(pathlib.Path(__file__).resolve().parent / '../../../scripts/pyutils'))
import gitutils # noqa: E402
Expand All @@ -24,9 +25,6 @@
# The version of systemd-logs image included in the sonobuoy release.
CNCF_SYSTEMD_LOGS_VERSION = "v0.4"

# The current version of the microshift-gitops package.
GITOPS_VERSION = "1.19"


logging.basicConfig(
level=logging.INFO,
Expand Down Expand Up @@ -209,6 +207,38 @@ def get_release_version_string(repo, var_name):
return None


def get_gitops_version(minor_version):
"""
Get the version of the microshift-gitops package.
Versions compatible with MicroShift: https://access.redhat.com/product-life-cycles?product=Red%20Hat%20OpenShift%20GitOps
"""
url = "https://access.redhat.com/product-life-cycles/api/v1/products"
params = {"name": "Red Hat OpenShift GitOps"}
resp = None
for attempt in range(1, 4):
try:
resp = requests.get(url, params=params, timeout=10)
resp.raise_for_status()
break
except Exception as e:
logging.warning(f"Attempt {attempt} failed with error: {e}. Retrying...")
time.sleep(2)
continue

if resp is None:
logging.error(f"Failed to fetch data from {url} after 3 attempts")
return ""
data = resp.json()
for current_microshift_minor_version in range(minor_version, minor_version - 4, -1):
for gitops_version_from_api_docs in data.get("data", [{}])[0].get("versions", []):
gitops_version_ocp_compatibility = gitops_version_from_api_docs.get("openshift_compatibility")
gitops_version_number = gitops_version_from_api_docs.get("name")
if f"4.{current_microshift_minor_version}" in gitops_version_ocp_compatibility:
logging.info(f"Latest GitOps version: {gitops_version_number} which is compatible with OCP {gitops_version_ocp_compatibility}")
return gitops_version_number
return ""


def generate_common_versions(minor_version):
previous_minor_version = minor_version - 1
yminus2_minor_version = minor_version - 2
Expand Down Expand Up @@ -256,6 +286,10 @@ def generate_common_versions(minor_version):
# The 'rhocp_minor_y2' should always be the y-2 minor version number.
rhocp_minor_y2 = yminus2_minor_version

# The current version of the microshift-gitops package.
logging.info("Getting GITOPS_VERSION")
gitops_version = get_gitops_version(minor_version)

template_path = pathlib.Path(__file__).resolve().parent / '../../assets/common_versions.sh.template'

with open(template_path, 'r') as f:
Expand All @@ -276,7 +310,7 @@ def generate_common_versions(minor_version):
rhocp_minor_y2=rhocp_minor_y2,
CNCF_SONOBUOY_VERSION=CNCF_SONOBUOY_VERSION,
CNCF_SYSTEMD_LOGS_VERSION=CNCF_SYSTEMD_LOGS_VERSION,
GITOPS_VERSION=GITOPS_VERSION,
GITOPS_VERSION=gitops_version,
ARCH=ARCH
)

Expand Down
41 changes: 33 additions & 8 deletions test/suites/gitops/gitops.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,15 @@ Documentation MicroShift GitOps tests
Resource ../../resources/microshift-process.resource

Suite Setup Setup Suite With Namespace
Suite Teardown Teardown Suite With Namespace
Suite Setup Setup Suite
Suite Teardown Teardown Suite


*** Variables ***
${APPLICATION_MANIFEST_PATH} ${CURDIR}/spring-petclinic-app.yaml
${APPLICATION_NAMESPACE} spring-petclinic
${APPLICATION_NAME} spring-petclinic
${GITOPS_NAMESPACE} openshift-gitops


*** Test Cases ***
Expand All @@ -14,10 +21,28 @@ Verify GitOps Pods Start Correctly
Wait Until Keyword Succeeds 2min 10s
... All Pods Should Be Running openshift-gitops

Verify Workload Deployed Correctly
[Documentation] Deploys workload and waits for ready status
Verify Application Deployed Correctly
[Documentation] Deploys an application and waits for it to be Healthy
... using the example from official docs: https://docs.redhat.com/en/documentation/red_hat_build_of_microshift/4.20/html/running_applications/microshift-gitops#microshift-gitops-adding-apps_microshift-gitops
[Setup] Setup Application Deployment

Wait Until Resource Exists applications ${APPLICATION_NAME} ${GITOPS_NAMESPACE} timeout=120s
Oc Wait
... -n ${GITOPS_NAMESPACE} application ${APPLICATION_NAME}
... --for=jsonpath='{.status.sync.status}'=Synced --timeout=300s
Oc Wait
... -n ${APPLICATION_NAMESPACE} pod --selector=app=${APPLICATION_NAME}
... --for=condition=Ready --timeout=300s
Comment on lines +29 to +35
Copy link

@coderabbitai coderabbitai bot Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# Check if the file exists and examine the relevant section
if [ -f "test/suites/gitops/gitops.robot" ]; then
  echo "=== File exists, showing lines 20-40 ==="
  cat -n test/suites/gitops/gitops.robot | sed -n '20,40p'
else
  echo "File not found at expected location"
  find . -name "gitops.robot" -type f 2>/dev/null | head -5
fi

Repository: openshift/microshift

Length of output: 1200

🏁 Script executed:

# Search for other uses of "Oc Wait" and "Synced" in the codebase to understand patterns
echo "=== Searching for Oc Wait usage patterns ==="
rg "Oc Wait" test/ --context 2 -A 2 -B 2 | head -50

echo -e "\n=== Searching for Application Synced patterns ==="
rg "Synced" test/ --context 1 | head -30

Repository: openshift/microshift

Length of output: 4388

🏁 Script executed:

# Look for Wait Until Keyword Succeeds usage in the test suite
echo "=== Checking Wait Until Keyword Succeeds patterns ==="
rg "Wait Until Keyword Succeeds" test/ --context 1 | head -20

Repository: openshift/microshift

Length of output: 1566

Pod readiness check is racy after Synced status.

Synced confirms Argo CD applied manifests but does not guarantee pods exist. When Oc Wait executes, the selector can still match zero pods, causing immediate failure instead of polling.

Wrap the pod wait with retry logic to handle the creation delay:

Suggested fix 
-    Oc Wait
-    ...    -n ${APPLICATION_NAMESPACE} pod --selector=app=${APPLICATION_NAME}
-    ...    --for=condition=Ready --timeout=300s
+    Wait Until Keyword Succeeds    5min    10s
+    ...    Oc Wait
+    ...    -n ${APPLICATION_NAMESPACE} pod --selector=app=${APPLICATION_NAME}
+    ...    --for=condition=Ready --timeout=30s
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@test/suites/gitops/gitops.robot` around lines 29 - 35, The pod readiness
check after Wait Until Resource Exists and the Oc Wait for application
${APPLICATION_NAME} is racy because Synced can occur before pods are created;
modify the test to first poll for the pod existence (use oc get pods
--selector=app=${APPLICATION_NAME} in ${APPLICATION_NAMESPACE} with a short
retry loop and timeout) and only when at least one pod is returned run the Oc
Wait ... pod --selector=app=${APPLICATION_NAME} --for=condition=Ready
--timeout=...; update the Robot steps around Wait Until Resource Exists / Oc
Wait to add this existence-check retry before the readiness-wait.


[Teardown] Teardown Application Deployment


*** Keywords ***
Setup Application Deployment
[Documentation] Setup the application deployment
Oc Apply -f ${APPLICATION_MANIFEST_PATH}

VAR ${manifest_path}= ${CURDIR}/test-deployment.yaml
Oc Apply -f ${manifest_path} -n ${NAMESPACE}
Wait Until Keyword Succeeds 5min 10s
... Named Deployment Should Be Available test-app
Teardown Application Deployment
[Documentation] Teardown the application deployment
Oc Delete -f ${APPLICATION_MANIFEST_PATH}
Oc Delete ns ${APPLICATION_NAMESPACE}
35 changes: 35 additions & 0 deletions test/suites/gitops/spring-petclinic-app.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
kind: AppProject
apiVersion: argoproj.io/v1alpha1
metadata:
name: default
namespace: openshift-gitops
spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
destinations:
- namespace: '*'
server: '*'
sourceRepos:
- '*'
---
kind: Application
apiVersion: argoproj.io/v1alpha1
metadata:
name: spring-petclinic
namespace: openshift-gitops
spec:
destination:
namespace: spring-petclinic
server: https://kubernetes.default.svc
project: default
source:
directory:
recurse: true
path: app
repoURL: https://github.com/siamaksade/openshift-gitops-getting-started
syncPolicy:
automated: {}
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
19 changes: 0 additions & 19 deletions test/suites/gitops/test-deployment.yaml
View file
Open in desktop

This file was deleted.