Skip to content

StretchCluster automated acceptance test + dev env setup#1289

Merged
hidalgopl merged 14 commits intork/k8s-751/implement-reconcilerfrom
pb/k8s-789/dev-env-for-stretch-cluster
Apr 2, 2026
Merged

StretchCluster automated acceptance test + dev env setup#1289
hidalgopl merged 14 commits intork/k8s-751/implement-reconcilerfrom
pb/k8s-789/dev-env-for-stretch-cluster

Conversation

@hidalgopl
Copy link
Copy Markdown
Contributor

…chCluster locally in k3s on vClusters

Dev workflow:

  1. set up REDPANDA_LICENSE_KEY env var to the contents of the license file (export REDPANDA_LICENSE_KEY=$(cat <license-file>)
  2. nix develop -c task setup-multicluster-dev-env <- sets up k3s, 3 vclusters, redpanda-operator inside each one
  3. after changing operator, teardown the env with k3d cluster delete harpoon and re-run the task command from 2)

@hidalgopl hidalgopl marked this pull request as draft March 10, 2026 11:51
@hidalgopl hidalgopl force-pushed the pb/k8s-789/dev-env-for-stretch-cluster branch from c721865 to 46b850a Compare March 13, 2026 16:18
@github-actions
Copy link
Copy Markdown

This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions bot added the stale label Mar 19, 2026
@RafalKorepta RafalKorepta force-pushed the rk/k8s-751/implement-reconciler branch 2 times, most recently from a9c4d25 to 2b70cf6 Compare March 20, 2026 15:08
@github-actions github-actions bot removed the stale label Mar 21, 2026
…luster locally in k3s on vClusters

* fix panics & ownership issues in multicluster controller
* fix initAdminClient

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
…nerated by vcluster

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
@hidalgopl hidalgopl force-pushed the pb/k8s-789/dev-env-for-stretch-cluster branch from 46b850a to 53aa109 Compare March 23, 2026 19:22
Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
@hidalgopl hidalgopl force-pushed the pb/k8s-789/dev-env-for-stretch-cluster branch from 2796303 to 5799cfe Compare March 24, 2026 11:25
@hidalgopl hidalgopl changed the title WIP: task dev:setup-multicluster-dev-env command for setting up Stret… StretchCluster automated acceptance test + dev env setup Mar 24, 2026
@hidalgopl hidalgopl marked this pull request as ready for review March 24, 2026 11:31
@hidalgopl hidalgopl force-pushed the pb/k8s-789/dev-env-for-stretch-cluster branch 3 times, most recently from ce63dc8 to 8073d21 Compare March 24, 2026 15:56
Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
@hidalgopl hidalgopl force-pushed the pb/k8s-789/dev-env-for-stretch-cluster branch from 8073d21 to 8ee48c5 Compare March 24, 2026 15:58
@github-actions
Copy link
Copy Markdown

This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions bot added the stale label Mar 30, 2026
Copy link
Copy Markdown
Contributor

@andrewstucki andrewstucki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there's still quite a bit to clean up here, but am fine merging this into the base branch for iteration -- I do think we'll need to disable the test by default for now until the license stuff is setup properly for CI

@operator:none
Feature: Multicluster Operator

@skip:gke @skip:aks @skip:eks @skip:k3d
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

think we'll need this until the license env injection works in CI

Copy link
Copy Markdown
Contributor Author

@hidalgopl hidalgopl Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it works now, the issue was not passing env to the docker container in the ci/scripts/run-in-nix-docker.sh

hidalgopl and others added 2 commits April 1, 2026 13:05
#1338)

* Detect StrechCluster.spec drift, halt reconcilation and set condition on all StrechClusters

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* enable nodepool controller for stretch cluster operator; update accep… (#1346)

* enable nodepool controller for stretch cluster operator; update acceptance test to assert on nodepools bound and deployed

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* Bootstrap admin for StretchCluster (#1348)

* distribute bootstrap admin credentials across stretch cluster k8s clusters

Add syncBootstrapUser syncer that runs before reconcilers to ensure the
bootstrap user secret exists consistently in all k8s clusters. Replace
hardcoded admin/admin credentials in AdminClientForStretch with credentials
read from the distributed secret.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* add BootstrapUserSynced condition and password mismatch detection

Scan all clusters for existing bootstrap user secrets and error with a
PasswordMismatch condition if different passwords are found. Set a
BootstrapUserSynced condition on StretchCluster status for all outcomes:
Synced (newly generated), ExistingReused (found in a cluster), or
PasswordMismatch (conflicting secrets require manual intervention).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* make sure we set canonical cluster name in the condition reason

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* fix per pod service, comment out nodepool watches temporarily

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* fix issue with nodepools with .spec.replicas > 1

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* Fix misc issues with multiple statefulset replicas

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* try to use v1 license in acceptance test

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* pass license env var to docker command in

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* operator: Distinguish in-cluster pools from all pools in RenderState

RenderState previously used a single `pools` slice for both rendering
local resources (StatefulSets, ConfigMaps, Secrets). The per Pod service
were not generated for remote NodePools. Those Kubernetes Services are
required to create DNS entries even if Pod does not exist in particular
Kubernetes cluster.

Split into two fields:
- `inClusterPools`: NodePools local to this K8s cluster, used for
  StatefulSets, ConfigMaps, Secrets, and pod name generation
- `pools`: all NodePools across clusters, used for seed server
  computation and per Pod Service

Move seed server generation from lifecycle into RenderState via
seedServersFromNodePools(), removing the now-unused
SeedServersFromNodePools() export. Add GetAllNodePools() to
StretchClusterWithPools so callers can pass the full pool list.

* regenerate golden files

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* add configuration of NodePool.Spec.Services that allows to modify spec/annotations for local and remote per-pod services

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

* operator: Move spec consistency check after finalizer removal

* operator: Fix seed server port type and propagate config errors

The seed server port was rendered as a string in redpanda.yaml instead of
an integer, which Redpanda rejects. Parse the port with strconv.ParseInt
and propagate the resulting error through redpandaConfigFile, configMaps,
and statefulSetChecksumAnnotation. Also add missing namespace to NodePool
test fixtures so seed server addresses render correctly.

---------

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Rafal Korepta <rafal.korepta@gmail.com>

* regenerate golden files

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>

---------

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Rafal Korepta <rafal.korepta@gmail.com>

---------

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Rafal Korepta <rafal.korepta@gmail.com>
Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 2, 2026

CLA assistant check
All committers have signed the CLA.

Signed-off-by: Paweł Bojanowski <pawel.bojanowski@redpanda.com>
@hidalgopl hidalgopl merged commit 4e3b56e into rk/k8s-751/implement-reconciler Apr 2, 2026
10 checks passed
@RafalKorepta RafalKorepta deleted the pb/k8s-789/dev-env-for-stretch-cluster branch April 8, 2026 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants