diff --git a/.gitignore b/.gitignore index 3c8dfaf..b21dd7c 100644 --- a/.gitignore +++ b/.gitignore @@ -6,4 +6,7 @@ *.iml ### Mac OS ### -**.DS_Store \ No newline at end of file +**.DS_Store + +# Hugo local preview site (generated by docs/preview.sh) +.docs-preview/ \ No newline at end of file diff --git a/README.md b/README.md index b173fdf..5628a1c 100644 --- a/README.md +++ b/README.md @@ -260,6 +260,28 @@ You can also provide an absolute path: LOCAL_DIR=/home/user/repos/developer-quickstart ./install.sh ``` +### Previewing Documentation Locally + +The `docs/` directory contains the project documentation. To preview it locally with Hugo: + +```shell +./docs/preview.sh +``` + +This starts a local server at [http://localhost:1313/docs/](http://localhost:1313/docs/) with live-reload. +Press `Ctrl+C` to stop. + +To generate static HTML instead: + +```shell +./docs/preview.sh build +``` + +The output is written to `.docs-preview/public/`. + +**Requirements:** `hugo`, `git`, and `go` must be installed. +The script handles theme fetching and site configuration automatically. + ## Testing ### CI Smoke Tests @@ -328,28 +350,3 @@ The scripts accept configuration via environment variables: | `PLATFORMS` | ComputeTestMatrix | `minikube kind` | Space-separated list of target platforms | | `LOG_TAIL_LINES` | Debug | `30` | Number of log lines to tail per pod | -## Repository Structure - -``` -components/ # Reusable Kustomize components -├── core/ # Core stack component -│ ├── base/ # Operators & CRDs -│ │ ├── strimzi-operator/ # Strimzi Kafka Operator -│ │ ├── apicurio-registry-operator/ # Apicurio Registry Operator -│ │ └── streamshub-console-operator/ # StreamsHub Console Operator -│ └── stack/ # Operands (Custom Resources) -│ ├── kafka/ # Single-node Kafka cluster -│ ├── apicurio-registry/ # In-memory registry instance -│ └── streamshub-console/ # Console instance -└── metrics/ # Prometheus metrics component - ├── base/ # Prometheus Operator - └── stack/ # Prometheus instance, PodMonitors, patches - -overlays/ # Deployable configurations -├── core/ # Default install (core only) -│ ├── base/ # Phase 1: Operators & CRDs -│ └── stack/ # Phase 2: Operands -└── metrics/ # Core + Prometheus metrics - ├── base/ # Phase 1: Operators & CRDs + Prometheus Operator - └── stack/ # Phase 2: Operands + Prometheus instance & monitors -``` diff --git a/docs/_index.md b/docs/_index.md new file mode 100644 index 0000000..083c137 --- /dev/null +++ b/docs/_index.md @@ -0,0 +1,47 @@ ++++ +title = 'Developer Quick-Start' +linkTitle = 'Developer Quick-Start' +weight = 0 +[[cascade]] + type = 'docs' ++++ + +# Developer Quick-Start + +Please make sure you have all the [prerequisites](prerequisites.md) covered first. +Then you can deploy a complete, open-source, event-streaming stack on your local Kubernetes cluster (e.g. MiniKube, KIND) with a single command: + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | bash +``` + +The script installs operators, waits for readiness, then deploys all services. +The full stack is typically ready in under five minutes. + +> **Note:** This is a **development-only** configuration. Resource limits, security settings, and storage are not suitable for production use. + +## What Gets Deployed + +| Component | Namespace | Description | +|-------------------------------|----------------------|-----------------------------------| +| Strimzi Kafka Operator | `strimzi` | Manages Kafka clusters via CRDs | +| Kafka cluster (`dev-cluster`) | `kafka` | Single-node Kafka for development | +| Apicurio Registry Operator | `apicurio-registry` | Manages schema registry instances | +| Apicurio Registry | `apicurio-registry` | In-memory schema registry | +| StreamsHub Console Operator | `streamshub-console` | Manages console instances | +| StreamsHub Console | `streamshub-console` | Web UI for Kafka management | + +Optional [overlays](overlays/_index.md) can extend the core stack with additional components such as Prometheus metrics. Install with an overlay by setting the `OVERLAY` variable: + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | OVERLAY= bash +``` + +## Next Steps + +- [Prerequisites](prerequisites.md) — what you need before installing. +- [Installation](installation.md) — all installation methods and options. +- [Accessing Services](accessing-services.md) — open the Console, connect to Kafka. +- [Overlays](overlays/) - look at the what additional components and features can be added with the provided overlays. +- [Uninstallation](uninstallation.md) — safe teardown on any cluster. +- [Troubleshooting](troubleshooting.md) - help with common issues. \ No newline at end of file diff --git a/docs/accessing-services.md b/docs/accessing-services.md new file mode 100644 index 0000000..0767f53 --- /dev/null +++ b/docs/accessing-services.md @@ -0,0 +1,119 @@ ++++ +title = 'Accessing Services' +weight = 3 ++++ + +# Accessing Services + +## StreamsHub Console + +### Ingress Access + +For the Console to startup correctly and be accessible, you need an Ingress controller running in your cluster. + +#### minikube + +Enable the ingress addon (if not already enabled) and start a tunnel: + +```shell +minikube addons enable ingress +minikube tunnel +``` + +You will need to leave the tunnel running in an open terminal. +Switch to a new terminal and use port-forwarding to access the console: + +```bash +kubectl port-forward -n streamshub-console svc/streamshub-console-console-service 8090:80 +``` + +Open [http://localhost:8090](http://localhost:8090) in your browser. + +#### KIND + +If you created your KIND cluster with the port mappings described in [Prerequisites](prerequisites.md), and deployed ingress-nginx: +Use port-forwarding to access the console: + +```bash +kubectl port-forward -n streamshub-console svc/streamshub-console-console-service 8090:80 +``` + +Open [http://localhost:8090](http://localhost:8090) in your browser. + + +## Kafka + +The Kafka cluster is accessible within the Kubernetes cluster at: + +``` +dev-cluster-kafka-bootstrap.kafka.svc.cluster.local:9092 +``` + +### Port-Forwarding + +To access Kafka from outside the cluster: + +```shell +kubectl port-forward -n kafka svc/dev-cluster-kafka-bootstrap 9092:9092 +``` + +Then connect your client to `localhost:9092`. + +### Producing and Consuming Messages + +From within the cluster, you can use the Kafka CLI tools: + +```shell +# Start a producer +kubectl run kafka-producer -it --rm --image=quay.io/strimzi/kafka:0.51.0-kafka-4.2.0 \ + --restart=Never -- bin/kafka-console-producer.sh \ + --bootstrap-server dev-cluster-kafka-bootstrap.kafka.svc.cluster.local:9092 \ + --topic test + +# Start a consumer (in a separate terminal) +kubectl run kafka-consumer -it --rm --image=quay.io/strimzi/kafka:0.51.0-kafka-4.2.0 \ + --restart=Never -- bin/kafka-console-consumer.sh \ + --bootstrap-server dev-cluster-kafka-bootstrap.kafka.svc.cluster.local:9092 \ + --topic test --from-beginning +``` + +## Apicurio Registry + +The registry has two services accessible within the cluster: + +- API: `apicurio-registry-app-service.apicurio-registry.svc.cluster.local:8080` +- UI: `apicurio-registry-ui-service.apicurio-registry.svc.cluster.local:8080` + +### Port-Forwarding + +The UI is a browser application that connects to the API backend at `localhost:8080` by default. +To use the UI, you must port-forward **both** the API and UI services. + +#### API + +The API service must be forwarded to port 8080 for the UI to function: + +```shell +kubectl port-forward -n apicurio-registry svc/apicurio-registry-app-service 8080:8080 +``` + +You can query the API directly: + +```shell +curl http://localhost:8080/apis/registry/v3/search/artifacts +``` + +#### UI + +In a separate terminal, forward the UI service: + +```shell +kubectl port-forward -n apicurio-registry svc/apicurio-registry-ui-service 8081:8080 +``` + +Open [http://localhost:8081](http://localhost:8081) in your browser. + +## Overlay Services + +If you installed with an overlay, it may deploy additional services. +See the specific overlay page under [Overlays](overlays/_index.md) for access instructions. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..21527ec --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,99 @@ ++++ +title = 'Architecture' +weight = 6 ++++ + +# Architecture + +## Two-Phase Deployment + +The event stack is deployed in two sequential phases: + +**Phase 1 - The Base - Operators and CRDs:** Deploys operator Deployments, RBAC resources, and Custom Resource Definitions. +The install script waits for each operator to become ready before proceeding. + +**Phase 2 - The Stack - Operands:** Deploys the actual workloads as Custom Resources (Kafka, ApicurioRegistry3, Console). +Operators must be running to process these resources. + +This separation exists for three reasons: + +1. **CRD registration** — Kubernetes must register CRDs before it can accept custom resources of that type +2. **Operator readiness** — operators must be running to reconcile their custom resources +3. **Safe teardown** — during uninstall, operands are deleted first while operators are still alive to process finalizers + +The install script uses `kubectl apply --server-side` for Phase 1 to handle large CRDs (such as those from the Prometheus Operator) that exceed the annotation size limit used by client-side apply. + +## Kustomize Structure + +The repository uses a component-based Kustomize architecture: + +``` +components/ # Reusable Kustomize components +├── core/ +│ ├── base/ # Component: operators & CRDs +│ └── stack/ # Component: operands +└── metrics/ # Optional metrics component + ├── base/ + └── stack/ +overlays/ # Deployable configurations +├── core/ # Default (no metrics) +│ ├── base/ # Phase 1: components/core/base +│ └── stack/ # Phase 2: components/core/stack +└── metrics/ # Core + Prometheus + ├── base/ # Phase 1: core/base + metrics/base + └── stack/ # Phase 2: core/stack + metrics/stack +``` + +**Components** are reusable building blocks (Kustomize `Component` kind). +They define operators, operands, and patches but are not directly deployable. + +**Overlays** compose components into deployable configurations. +Each overlay has a `base` (Phase 1) and `stack` (Phase 2) directory. +The `metrics` overlay includes everything from `core` plus the Prometheus components. + +## Resource Labeling + +Every resource deployed by the quick-start carries the label: + +```yaml +app.kubernetes.io/part-of: streamshub-developer-quickstart +``` + +This label is applied by the Kustomize `labels` transformer and serves two purposes: + +- Resource discovery — find all quick-start resources with a single label selector +- Shared-cluster safety — the uninstall script uses label selectors to distinguish quick-start resources from user-created ones, preventing accidental deletion of CRDs that other deployments depend on + +## Namespace Isolation + +Each component runs in its own namespace: + +| Namespace | Contents | +|----------------------|----------------------------------------------------| +| `strimzi` | Strimzi Kafka Operator | +| `kafka` | Kafka cluster (`dev-cluster`) | +| `apicurio-registry` | Registry Operator and instance | +| `streamshub-console` | Console Operator and instance | +| `monitoring` | Prometheus Operator and instance (metrics overlay) | + +## Updating Component Versions + +Use the `update-version.sh` script to manage operator versions: + +```shell +# List available versions for a component +./update-version.sh --list strimzi + +# Preview changes without modifying files +./update-version.sh --dry-run strimzi 0.52.0 + +# Check if a specific release exists +./update-version.sh --check apicurio-registry 3.2.0 + +# Apply the update +./update-version.sh strimzi 0.52.0 +``` + +Supported components: `strimzi`, `apicurio-registry`, `streamshub-console`, `prometheus-operator` + +The script updates the remote resource URLs in the relevant `kustomization.yaml` files to point to the new version's release artifacts. diff --git a/docs/installation.md b/docs/installation.md new file mode 100644 index 0000000..529ade0 --- /dev/null +++ b/docs/installation.md @@ -0,0 +1,122 @@ ++++ +title = 'Installation' +weight = 2 ++++ + +# Installation + +## Quick-Start Install + +Deploy the entire core stack with a single command: + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | bash +``` + +The script: + +1. Checks prerequisites (kubectl, IngressClass) +2. Installs operators and CRDs (Phase 1) +3. Waits for each operator to become ready +4. Deploys Kafka, Registry, and Console instances (Phase 2) + +### Install with an Overlay + +Overlays extend the core stack with optional components. To install with an overlay, set the `OVERLAY` variable: + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | OVERLAY= bash +``` + +See [Overlays](overlays/_index.md) for the list of available overlays and what each one adds. + +### Configuration + +The install script accepts the following environment variables: + +| Variable | Default | Description | +|-------------|-----------------------------------|-----------------------------------------------------------------------------| +| `REPO` | `streamshub/developer-quickstart` | GitHub repository path | +| `REF` | `main` | Git ref, branch, or tag | +| `OVERLAY` | *(empty)* | Overlay to apply (e.g. `metrics`) | +| `TIMEOUT` | `120s` | `kubectl wait` timeout (supports `s`, `m`, `h` suffixes) | +| `LOCAL_DIR` | *(empty)* | Use a local directory as the source of install files instead of GitHub URLs | + +**Examples:** + +```shell +# Install a specific version +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | REF=v1.0.0 bash + +# Increase timeout for slower clusters +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | TIMEOUT=300s bash +``` + +## Manual Install + +If you prefer step-by-step control, the stack is installed in two phases. + +### Phase 1 — Operators and CRDs + +```shell +kubectl apply -k 'https://github.com/streamshub/developer-quickstart//overlays/core/base?ref=main' +``` + +Optionally, you can wait for the operators to become ready using the commands below: + +```shell +kubectl wait --for=condition=Available deployment/strimzi-cluster-operator -n strimzi --timeout=120s +kubectl wait --for=condition=Available deployment/apicurio-registry-operator -n apicurio-registry --timeout=120s +kubectl wait --for=condition=Available deployment/streamshub-console-operator -n streamshub-console --timeout=120s +``` + +### Phase 2 — Operands + +```shell +kubectl apply -k 'https://github.com/streamshub/developer-quickstart//overlays/core/stack?ref=main' +``` + +### Manual Install with an Overlay + +For overlays, replace `overlays/core` with `overlays/` in the commands above. +Each overlay may add additional operators that need to be waited on during Phase 1. +See the specific overlay page under [Overlays](overlays/_index.md) for the full manual install commands. + +## Install from a Local Checkout + +When developing or testing changes to the kustomization files, point the scripts at a local directory: + +```shell +# Clone and install from local +git clone https://github.com/streamshub/developer-quickstart.git +cd developer-quickstart +LOCAL_DIR=. ./install.sh +``` + +When `LOCAL_DIR` is set, `REPO` and `REF` are ignored — all kustomization paths resolve relative to the given directory. +You can also pass an absolute path: + +```shell +LOCAL_DIR=/home/user/repos/developer-quickstart ./install.sh +``` + +## Verify the Installation + +After installation, confirm all components are running: + +```shell +# Check operators +kubectl get deployment -n strimzi strimzi-cluster-operator +kubectl get deployment -n apicurio-registry apicurio-registry-operator +kubectl get deployment -n streamshub-console streamshub-console-operator + +# Check operands +kubectl get kafka -n kafka +kubectl get apicurioregistry3 -n apicurio-registry +kubectl get console -n streamshub-console + +``` + +All operator deployments should show `READY 1/1`. Custom resources should reach `Ready` status. + +If you installed with an overlay, check its page under [Overlays](overlays/_index.md) for additional verification steps. diff --git a/docs/overlays/_index.md b/docs/overlays/_index.md new file mode 100644 index 0000000..b130733 --- /dev/null +++ b/docs/overlays/_index.md @@ -0,0 +1,27 @@ ++++ +title = 'Overlays' +weight = 4 ++++ + +# Overlays + +The developer quick-start uses [Kustomize overlays](architecture.md) to provide optional extensions on top of the core stack. +Each overlay adds components that integrate with the base deployment. + +To install with an overlay, set the `OVERLAY` environment variable: + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | OVERLAY= bash +``` + +To uninstall, use the same `OVERLAY` value: + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/uninstall.sh | OVERLAY= bash +``` + +## Available Overlays + +| Overlay | Description | +|-----------------------|--------------------------------------------------------------------------------------------------------------------------------------| +| [metrics](metrics.md) | Adds Prometheus Operator, a Prometheus instance, and Kafka metrics collection via PodMonitors. Wires the Console to display metrics. | diff --git a/docs/overlays/metrics.md b/docs/overlays/metrics.md new file mode 100644 index 0000000..fa7c460 --- /dev/null +++ b/docs/overlays/metrics.md @@ -0,0 +1,132 @@ ++++ +title = 'Metrics' +weight = 1 ++++ + +# Metrics Overlay + +The metrics overlay extends the core stack with Prometheus-based monitoring. + +## Quick-Start Install + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | OVERLAY=metrics bash +``` + +## Manual Install + +If you prefer step-by-step control, the metrics overlay uses `overlays/metrics` instead of `overlays/core`: + +```shell +# Phase 1 — Operators and CRDs (includes Prometheus Operator) +kubectl create -k 'https://github.com/streamshub/developer-quickstart//overlays/metrics/base?ref=main' + +# Optionally, wait for the operators to be ready +kubectl wait --for=condition=Available deployment/prometheus-operator -n monitoring --timeout=120s +kubectl wait --for=condition=Available deployment/strimzi-cluster-operator -n strimzi --timeout=120s +kubectl wait --for=condition=Available deployment/apicurio-registry-operator -n apicurio-registry --timeout=120s +kubectl wait --for=condition=Available deployment/streamshub-console-operator -n streamshub-console --timeout=120s + +# Phase 2 — Operands (includes Prometheus instance and monitors) +kubectl apply -k 'https://github.com/streamshub/developer-quickstart//overlays/metrics/stack?ref=main' +``` + +## Uninstall + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/uninstall.sh | OVERLAY=metrics bash +``` + +## What Gets Added + +On top of the components provided by the core overlay, the metrics overlay adds: + +| Component | Namespace | Description | +|---------------------|--------------|--------------------------------------------------------| +| Prometheus Operator | `monitoring` | Manages Prometheus instances and monitors | +| Prometheus instance | `monitoring` | Collects and stores metrics (400Mi memory, 1 replica) | +| PodMonitors | `monitoring` | Scrape targets for Kafka brokers and Strimzi operators | +| ServiceMonitor | `monitoring` | Scrape target for StreamsHub Console operator | + +The overlay also patches existing resources: + +- Kafka — enables the [Strimzi Metrics Reporter](https://strimzi.io/docs/operators/latest/deploying#proc-metrics-kafka-str) on the `dev-cluster`, exposing JMX metrics at `/metrics` +- Console — adds Prometheus as a metrics data source so the Console UI displays Kafka metrics + +## How Metrics Flow + +``` +Kafka Broker → /metrics (Strimzi Metrics Reporter) + ↓ +PodMonitor (kafka-resources-metrics) + ↓ +Prometheus (prometheus-operated:9090) + ↓ +StreamsHub Console (displays metrics in UI) +``` + +Additional scrape targets: + +- Strimzi Cluster Operator — scraped via `cluster-operator-metrics` PodMonitor +- Strimzi Entity Operator — scraped via `entity-operator-metrics` PodMonitor +- Console Operator — scraped via `streamshub-console-operator` ServiceMonitor +- Kubernetes cAdvisor — container-level resource metrics +- Kubernetes Kubelet — node-level metrics + +## Accessing Prometheus + +Port-forward to the Prometheus UI: + +```shell +kubectl port-forward -n monitoring svc/prometheus-operated 9090:9090 +``` + +Open [http://localhost:9090](http://localhost:9090) and navigate to **Status > Targets** to verify all scrape targets are up. + +## Verify the Installation + +Confirm the metrics components are running: + +```shell +kubectl get prometheus -n monitoring +kubectl get podmonitor -n monitoring +kubectl get servicemonitor -n monitoring +``` + +Prometheus should reach `Available` status. Check that scrape targets are healthy: + +```shell +kubectl port-forward -n monitoring svc/prometheus-operated 9090:9090 & +curl -s http://localhost:9090/api/v1/targets | grep -o '"health":"up"' | wc -l +``` + +Open the StreamsHub Console UI — Kafka cluster CPU and memory usage should show up straight away. +However, other metrics such as those for topics will only show once topics have been created and messages are flowing through them. + +## Troubleshooting + +### Metrics Not Appearing + +If the Console UI does not show metrics (metrics overlay): + +```shell +# Verify Prometheus is running +kubectl get prometheus -n monitoring + +# Check Prometheus scrape targets +kubectl port-forward -n monitoring svc/prometheus-operated 9090:9090 +# Visit http://localhost:9090/targets + +# Verify PodMonitors are created +kubectl get podmonitor -n monitoring + +# Verify Kafka has metrics enabled +kubectl get kafka/dev-cluster -n kafka -o jsonpath='{.spec.kafka.metricsConfig}' +``` + +**Common causes:** + +- Metrics overlay not installed — verify you used `OVERLAY=metrics` during installation +- PodMonitor label mismatch — Prometheus selects PodMonitors with `app: strimzi`; verify the label is present +- Kafka metrics not enabled — the metrics overlay patches the Kafka CR to add `metricsConfig`; check that it was applied + diff --git a/docs/prerequisites.md b/docs/prerequisites.md new file mode 100644 index 0000000..af50bc2 --- /dev/null +++ b/docs/prerequisites.md @@ -0,0 +1,70 @@ ++++ +title = 'Prerequisites' +weight = 1 ++++ + +# Prerequisites + +## Required + +- **kubectl v1.27 or later** — the install scripts use Kustomize v5.0 features (the `labels` transformer) which require kubectl 1.27+. + + Check your version: + + ```shell + kubectl version --client + ``` + +- **A running Kubernetes cluster** — any local or development cluster should work. Common options: + - [minikube](https://minikube.sigs.k8s.io/docs/start/) + - [KIND](https://kind.sigs.k8s.io/docs/user/quick-start/) + +## Recommended + +- **An Ingress controller** — required if you want to access the StreamsHub Console via its hostname (`console.streamshub.local`). Without one, you can still use port-forwarding to access Kafka. + +## Cluster Setup + +If you don't already have a cluster running, here are quick setup instructions for the most common options: + +### Minikube + +```shell +minikube start --cpus=4 --memory=8g --addons=ingress +``` + +### KIND + +KIND requires port mappings configured at cluster creation for Ingress to work: + +```shell +cat </dev/null; then + echo "Error: Required command '${cmd}' is not installed." + exit 1 + fi +done + +## Set up site structure ## + +mkdir -p "${SITE_DIR}" + +## Initialise Go module (required for Hugo mounts) ## + +if [ ! -f "${SITE_DIR}/go.mod" ]; then + (cd "${SITE_DIR}" && go mod init docs-preview) +fi + +## Fetch theme (cached across runs) ## + +if [ ! -d "${THEME_DIR}" ]; then + echo "Fetching hugo-book theme..." + git clone --depth 1 "${THEME_REPO}" "${THEME_DIR}" +else + echo "Using cached hugo-book theme." +fi + +## Generate hugo.toml ## + +# Mirrors the relevant settings from the streamshub-site hugo.toml. +# Uses Hugo module mounts to reference the docs directory directly. + +cat > "${SITE_DIR}/hugo.toml" << CONFIG +baseURL = 'http://localhost:1313/' +languageCode = 'en-us' +title = 'StreamsHub Developer Quick-Start — Local Preview' +theme = 'hugo-book' + +disablePathToLower = true + +[markup] + [markup.tableOfContents] + startLevel = 1 + [markup.highlight] + style = "catppuccin-macchiato" + [markup.goldmark] + [markup.goldmark.renderer] + unsafe = true + +[params] + BookTheme = 'dark' + BookSection = '/' + BookPortableLinks = true + +# Mount the docs directory as content/docs so Hugo reads it directly. +[[module.mounts]] + source = '${DOCS_DIR}' + target = 'content/docs' +CONFIG + +## Run Hugo ## + +cd "${SITE_DIR}" + +case "${1:-serve}" in + build) + echo "Building static site..." + hugo --gc --minify + echo "Output: ${SITE_DIR}/public/" + ;; + serve|*) + echo "" + echo "Starting local preview server..." + echo "Open http://localhost:1313/docs/ in your browser." + echo "Press Ctrl+C to stop." + echo "" + hugo server --buildDrafts --disableFastRender + ;; +esac diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md new file mode 100644 index 0000000..804617a --- /dev/null +++ b/docs/troubleshooting.md @@ -0,0 +1,98 @@ ++++ +title = 'Troubleshooting' +weight = 7 ++++ + +# Troubleshooting + +## Operator Not Becoming Ready + +If an operator deployment does not reach `Ready`/`Available` within the timeout: + +```shell +# Check the deployment status +kubectl describe deployment/strimzi-cluster-operator -n strimzi + +# Check operator pod logs +kubectl logs -n strimzi deployment/strimzi-cluster-operator + +# Check for pending pods (resource constraints) +kubectl get pods -n strimzi -o wide +``` + +**Common causes:** + +- Insufficient resources — the cluster may not have enough CPU or memory. This can often be an issue if you have selected an overlay which multiple components on top of the core. Try increasing resources assigned to your Kubernetes cluster: `minikube start --cpus=8 --memory=12g` +- Image pull errors — check that the cluster can reach container registries (quay.io, docker.io) +- Timeout too short — increase with `TIMEOUT=300s` + +## Custom Resource Not Reconciling + +If a Kafka, Registry, or Console resource is stuck: + +```shell +# Check CR status and conditions +kubectl describe kafka/dev-cluster -n kafka + +# Check events in the namespace +kubectl get events -n kafka --sort-by='.lastTimestamp' + +# Check the managing operator's logs +kubectl logs -n strimzi deployment/strimzi-cluster-operator --tail=100 +``` + +**Common causes:** + +- Operator not running — verify the operator deployment is ready before deploying operands +- CRDs not registered — if you skipped Phase 1 waits, CRDs may not be ready yet +- Namespace missing — the kustomization creates namespaces, but manual partial installs may miss them + +## Console Cannot Reach Kafka + +If the Console shows the Kafka cluster as unreachable: + +```shell +# Verify Console configuration +kubectl get console/streamshub-console -n streamshub-console -o yaml + +# Check the Console operator logs +kubectl logs -n streamshub-console deployment/streamshub-console-operator --tail=100 + +# Verify the Kafka bootstrap service exists +kubectl get svc/dev-cluster-kafka-bootstrap -n kafka +``` + +**Common causes:** + +- Kafka not ready — the Kafka cluster may still be starting. Check `kubectl get kafka -n kafka` +- Listener mismatch — the Console is configured to use the `plain` listener; verify the Kafka cluster exposes it +- Network policy — if your cluster has network policies, ensure cross-namespace traffic is allowed + +## CRD Conflicts on Shared Clusters + +If the cluster already has Strimzi, Apicurio, or Console CRDs from another deployment: + +- **Install** may show `already exists` warnings — this is usually safe. The quick-start will use the existing CRDs +- **Uninstall** will detect shared CRDs and retain them. The uninstall script reports which operator groups were retained and why + +To manually check for shared resources: + +```shell +kubectl get kafkas -A --selector='app.kubernetes.io/part-of!=streamshub-developer-quickstart' +kubectl get apicurioregistry3 -A --selector='app.kubernetes.io/part-of!=streamshub-developer-quickstart' +kubectl get console.console.streamshub.github.com -A --selector='app.kubernetes.io/part-of!=streamshub-developer-quickstart' +``` + +## Collecting Diagnostics + +For comprehensive diagnostic output, use the debug script: + +```shell +OVERLAY=core jbang .github/scripts/Debug.java +``` + +This dumps CR status, events, pod listings, and logs for all quick-start namespaces. With the metrics overlay: + +```shell +OVERLAY=metrics jbang .github/scripts/Debug.java +``` diff --git a/docs/uninstallation.md b/docs/uninstallation.md new file mode 100644 index 0000000..7d15a8c --- /dev/null +++ b/docs/uninstallation.md @@ -0,0 +1,95 @@ ++++ +title = 'Uninstallation' +weight = 5 ++++ + +# Uninstallation + +## Using the Uninstall Script + +The uninstall script provides safe teardown with shared-cluster awareness: + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/uninstall.sh | bash +``` + +If you installed with the metrics overlay: + +```shell +curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/uninstall.sh | OVERLAY=metrics bash +``` + +### Configuration + +| Variable | Default | Description | +|----------|---------|-------------| +| `REPO` | `streamshub/developer-quickstart` | GitHub repository path | +| `REF` | `main` | Git ref, branch, or tag | +| `OVERLAY` | *(empty)* | Overlay to uninstall (e.g. `metrics`) | +| `TIMEOUT` | `120s` | kubectl wait/poll timeout | +| `LOCAL_DIR` | *(empty)* | Use a local directory instead of GitHub | + +### What the Script Does + +The uninstall follows a 4-phase process designed to be safe on shared clusters: + +**Phase 1 — Interactive CR cleanup** (interactive mode only) + +Scans for Kafka, Registry, and Console resources that were *not* created by the quick-start (i.e., resources without the `app.kubernetes.io/part-of=streamshub-developer-quickstart` label). If found, you're prompted to delete them so their finalizers can be processed while operators are still running. + +**Phase 2 — Delete operands** + +Removes the quick-start custom resources (Kafka, Registry, Console instances) and waits for finalizers to complete. + +**Phase 3 — Shared-cluster safety checks** + +For each operator group (Strimzi, Apicurio, Console, Prometheus), the script checks whether non-quick-start custom resources exist on the cluster. This determines whether CRDs are shared with other deployments. + +**Phase 4 — Selective operator removal** + +- **Not shared** — the entire operator group is removed, including CRDs +- **Shared** — only the operator deployment is removed; CRDs are retained so other deployments continue to function + +The script reports which operator groups were retained and why. + +## Manual Teardown + +> **Warning:** On shared clusters, deleting CRDs will cascade-delete **all** custom resources of that type cluster-wide. Always check for non-quick-start resources first. + +### Phase 1 — Delete Operands + +```shell +kubectl delete -k 'https://github.com/streamshub/developer-quickstart//overlays/core/stack?ref=main' +``` + +Wait for all custom resources to be fully removed before continuing. + +### Phase 2 — Delete Operators and CRDs + +First, check for shared resources: + +```shell +kubectl get kafkas -A --selector='!app.kubernetes.io/part-of=streamshub-developer-quickstart' +kubectl get apicurioregistry3 -A --selector='!app.kubernetes.io/part-of=streamshub-developer-quickstart' +kubectl get console.console.streamshub.github.com -A --selector='!app.kubernetes.io/part-of=streamshub-developer-quickstart' +``` + +If no shared resources exist, delete the operators: + +```shell +kubectl delete -k 'https://github.com/streamshub/developer-quickstart//overlays/core/base?ref=main' +``` + +For the metrics overlay, use `overlays/metrics/base` and `overlays/metrics/stack` instead of `overlays/core`. + +## Finding Quick-Start Resources + +All resources deployed by the quick-start carry the label `app.kubernetes.io/part-of=streamshub-developer-quickstart`: + +```shell +# List all namespaced resources +kubectl get all -A -l app.kubernetes.io/part-of=streamshub-developer-quickstart + +# List cluster-scoped resources +kubectl get crds,clusterroles,clusterrolebindings -l app.kubernetes.io/part-of=streamshub-developer-quickstart +```