feat: kustomize for local & prod k8s

[SgtPooki]

Note: there are a lot of changes here, but there is a lot of documentation in /docs as well. check those out first to get a firm grasp of what's going on here.

Also note that due to the latest on FilOzone/tpm-utils#1, the filoz-infra expectations (ECR, AWS KMS, etc..) will change.

---

This pull request introduces a comprehensive local Kubernetes development workflow using Kind and Helm, adds robust CI/CD pipelines for Docker image builds and releases (including hotfixes), and establishes best practices for secret management and environment configuration. The changes streamline both local development and production release processes, ensuring consistency and security across environments.

Local development and Kubernetes workflow:

• Added DEVELOPMENT.md with detailed instructions for setting up and managing a local Kind cluster, handling secrets, deploying with Helm, and managing persistent storage for PostgreSQL.
• Introduced a Makefile with targets for building images, loading them into Kind, deploying via Helm, managing secrets, and orchestrating combined workflows for backend and web services.
• Added a sample .env.example for local secret management, clarifying which secrets are required and how they are consumed in local dev.
• Added a minimal apps/postgres/Dockerfile based on postgres:16-alpine for local database development.

CI/CD and release automation:

• Added .github/workflows/docker-build.yml to build and push backend and web Docker images to ECR, with change detection for efficient builds.
• Added .github/workflows/release-please.yml to automate versioning and retagging of Docker images after successful builds, using Release Please and ECR manifests.
• Added .github/workflows/hotfix-release.yml to support hotfix releases from hotfix/** branches, immediately building and pushing hotfix images upon release creation.
• Added .release-please-manifest.json to track and version backend and web applications.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
dealbot-web	Ready	Preview, Comment	Dec 19, 2025 7:36am

[SgtPooki]

Note: also this PR is [draft] and needs alignment and some cleanup to remove filoz-infra mentions (and cleanup of /docs) prior to merge. I wanted to get this out and get a pr review from copilot and will move this back to draft

[Copilot]

Pull request overview

This comprehensive pull request establishes a complete local-to-production Kubernetes deployment workflow for the dealbot application. The PR introduces Helm charts for local Kind cluster development, implements GitOps-based CI/CD pipelines with automated Docker image building and semantic versioning via release-please, and provides extensive documentation for both standard releases and emergency hotfixes.

Key changes:

• Replaces docker-compose with Helm charts and Kind for local development, providing production-parity K8s environments
• Implements automated CI/CD with path-based change detection, dual image tagging (SHA for staging, semver for production), and ECR manifest-based image promotion
• Adds comprehensive hotfix workflow supporting emergency releases directly from hotfix branches

Reviewed changes

Copilot reviewed 40 out of 41 changed files in this pull request and generated 20 comments.

Show a summary per file

File	Description
release-please-config.json	Configures release-please for independent versioning of backend and web apps with conventional commits
.release-please-manifest.json	Tracks current semantic versions for both applications starting at 0.0.1
kind-config.yaml	Defines local Kind cluster with port mappings for backend (8080) and web (3000) services
Makefile	Provides comprehensive targets for building images, loading to Kind, deploying Helm charts, and managing secrets
.env.example	Template for local development secrets (wallet keys, optional DB password)
DEVELOPMENT.md	Complete guide for local Kubernetes development with Kind, Helm, and secret management
docs/release-process.md	Documents the automated staging-to-production release flow with Flux CD integration
docs/release-please-flow.md	Explains conventional commit-based versioning and release-please PR workflow
docs/infra.md	Describes integration between local Helm charts and production Kustomize manifests in filoz-infra
docs/hotfix-and-edge-cases.md	Comprehensive guide for emergency hotfixes and edge case handling
charts/dealbot/Chart.yaml	Helm chart metadata for backend service
charts/dealbot/values.yaml	Production-ready values for backend with security and scaling options
charts/dealbot/values.local.yaml	Local development overrides including bundled PostgreSQL
charts/dealbot/values.local.override.example.yaml	Template for gitignored local customizations
charts/dealbot/templates/*.yaml	Kubernetes manifests for deployment, service, ingress, HPA, PostgreSQL, ConfigMap, and ServiceAccount
charts/dealbot-web/Chart.yaml	Helm chart metadata for web frontend
charts/dealbot-web/values.yaml	Production-ready values for web service
charts/dealbot-web/values.local.yaml	Local development overrides with NodePort service
charts/dealbot-web/values.local.override.example.yaml	Template for gitignored web customizations
charts/dealbot-web/templates/*.yaml	Kubernetes manifests for web deployment, service, ingress, and HPA
apps/web/Dockerfile	Multi-stage build with Caddy server, runtime config generation, and reverse proxy
apps/web/docker-entrypoint.sh	Generates /config.json from environment variables at container startup
apps/web/src/api/client.ts	Implements runtime config loading with fallback to build-time values for API base URL
apps/postgres/Dockerfile	Minimal PostgreSQL image for local development based on postgres:16-alpine
.github/workflows/docker-build.yml	Builds and pushes Docker images with SHA-based tags on main branch merges
.github/workflows/release-please.yml	Creates release PRs and retags images with semver after PR merge
.github/workflows/hotfix-release.yml	Handles emergency hotfix releases from hotfix/** branches
.gitignore	Adds entries for local Helm override files
docker-compose.dev.yml	Removed in favor of Kubernetes-based local development

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The WALLET_PRIVATE_KEY and WALLET_ADDRESS validation in the secret target doesn't verify that the values are non-empty, only that the keys exist. An empty value like 'WALLET_PRIVATE_KEY=' would pass validation but cause runtime failures. Consider adding validation to ensure these required secret values are not empty strings.

[Copilot]

The documentation states 'Do not skip step 6 or next release will reintroduce the bug!' but there's no automated check or safeguard to prevent this. Consider adding a GitHub Actions workflow or a pre-release check that verifies hotfix branches have been merged back to main before allowing new releases, or at least add a checklist item in the PR template to remind developers of this critical step.

[rjan90]

@BigLep and @rjan90 to review docs/infra.md

[SgtPooki]

Note that this is now deployed on my homelab k8s cluster and available at http://dealbot.bt.sgtpooki.com/

the diff is basically:

View diff needed for infra repo

diff --git a/docs/runbooks/dealbot.md b/docs/runbooks/dealbot.md
new file mode 100644
index 0000000..8cb6cbe
--- /dev/null
+++ b/docs/runbooks/dealbot.md
@@ -0,0 +1,36 @@
+# Dealbot (FilOzone) Deploy Notes
+
+This cluster deploys Dealbot from `https://github.com/FilOzone/dealbot.git` (branch `feat/helm-charts`) via a kustomize overlay in this repo.
+
+## 1) Create the required secret
+
+Dealbot backend expects a secret named `dealbot-secrets` in namespace `filoz-dealbot` that includes:
+
+- `DATABASE_PASSWORD`
+- `WALLET_ADDRESS`
+- `WALLET_PRIVATE_KEY`
+
+Create/update it from an env file:
+
+```bash
+kubectl -n filoz-dealbot create secret generic dealbot-secrets \
+  --from-env-file=/path/to/dealbot-secrets.env \
+  --dry-run=client -o yaml | kubectl apply -f -
+```
+
+Example `dealbot-secrets.env`:
+
+```text
+DATABASE_PASSWORD=change-me
+WALLET_ADDRESS=0x...
+WALLET_PRIVATE_KEY=...
+```
+
+## 2) GitOps sync
+
+The Argo CD `dealbot` application deploys:
+
+- Postgres (in-cluster, internal-only) as `svc/dealbot-postgres`
+- Backend as `svc/dealbot`
+- Web as `svc/dealbot-web` + public ingress `dealbot.bt.sgtpooki.com`
+
diff --git a/k8s/apps/dealbot/overlays/prod/dealbot-env.env b/k8s/apps/dealbot/overlays/prod/dealbot-env.env
new file mode 100644
index 0000000..164f99d
--- /dev/null
+++ b/k8s/apps/dealbot/overlays/prod/dealbot-env.env
@@ -0,0 +1,19 @@
+NODE_ENV=production
+DEALBOT_HOST=0.0.0.0
+DEALBOT_PORT=3130
+
+DATABASE_HOST=dealbot-postgres
+DATABASE_PORT=5432
+DATABASE_USER=dealbot
+DATABASE_NAME=filecoin_dealbot
+
+NETWORK=calibration
+
+# Scheduling (seconds)
+DEAL_INTERVAL_SECONDS=1800
+RETRIEVAL_INTERVAL_SECONDS=3600
+DEAL_START_OFFSET_SECONDS=0
+RETRIEVAL_START_OFFSET_SECONDS=300
+METRICS_START_OFFSET_SECONDS=600
+DEALBOT_ALLOWED_ORIGINS=http://dealbot.bt.sgtpooki.com,https://dealbot.bt.sgtpooki.com
+
diff --git a/k8s/apps/dealbot/overlays/prod/kustomization.yaml b/k8s/apps/dealbot/overlays/prod/kustomization.yaml
new file mode 100644
index 0000000..d24c73f
--- /dev/null
+++ b/k8s/apps/dealbot/overlays/prod/kustomization.yaml
@@ -0,0 +1,84 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+namespace: filoz-dealbot
+
+resources:
+  - namespace.yaml
+  - ../../../postgres/dealbot
+  - https://github.com/FilOzone/dealbot//kustomize/base/backend?ref=feat/helm-charts
+  - https://github.com/FilOzone/dealbot//kustomize/base/web?ref=feat/helm-charts
+
+configMapGenerator:
+  # Replace upstream configmap and use a hashed name so the Deployment rolls on config changes.
+  - name: dealbot-env
+    behavior: replace
+    envs:
+      - dealbot-env.env
+
+images:
+  - name: dealbot
+    newName: ghcr.io/filozone/dealbot-backend
+    newTag: sha-2596f4f0fc98dca7ac6cb9b14fe2bebed1fc2a63
+  - name: dealbot-web
+    newName: ghcr.io/filozone/dealbot-web
+    newTag: sha-2596f4f0fc98dca7ac6cb9b14fe2bebed1fc2a63
+
+patches:
+  # Upstream includes a Namespace; use this repo's namespace name instead.
+  - target:
+      kind: Namespace
+      name: dealbot
+    patch: |-
+      apiVersion: v1
+      kind: Namespace
+      metadata:
+        name: dealbot
+      $patch: delete
+
+  # Backend ingress is not needed; web container reverse-proxies /api to the backend service.
+  - target:
+      group: networking.k8s.io
+      version: v1
+      kind: Ingress
+      name: dealbot-api
+    patch: |-
+      apiVersion: networking.k8s.io/v1
+      kind: Ingress
+      metadata:
+        name: dealbot-api
+      $patch: delete
+
+  # Public hostname for the web UI.
+  - target:
+      group: networking.k8s.io
+      version: v1
+      kind: Ingress
+      name: dealbot-web
+    patch: |-
+      - op: replace
+        path: /spec/rules/0/host
+        value: dealbot.bt.sgtpooki.com
+      - op: add
+        path: /spec/ingressClassName
+        value: traefik
+
+  # Set the in-container reverse proxy target for /api.
+  - target:
+      group: apps
+      version: v1
+      kind: Deployment
+      name: dealbot-web
+    patch: |-
+      apiVersion: apps/v1
+      kind: Deployment
+      metadata:
+        name: dealbot-web
+      spec:
+        template:
+          spec:
+            containers:
+              - name: dealbot-web
+                env:
+                  - name: DEALBOT_API_UPSTREAM
+                    value: dealbot:3130
diff --git a/k8s/apps/dealbot/overlays/prod/namespace.yaml b/k8s/apps/dealbot/overlays/prod/namespace.yaml
new file mode 100644
index 0000000..788e5be
--- /dev/null
+++ b/k8s/apps/dealbot/overlays/prod/namespace.yaml
@@ -0,0 +1,5 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: filoz-dealbot
+
diff --git a/k8s/apps/postgres/dealbot/kustomization.yaml b/k8s/apps/postgres/dealbot/kustomization.yaml
new file mode 100644
index 0000000..b00ca25
--- /dev/null
+++ b/k8s/apps/postgres/dealbot/kustomization.yaml
@@ -0,0 +1,9 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+namespace: filoz-dealbot
+
+resources:
+  - service.yaml
+  - statefulset.yaml
+
diff --git a/k8s/apps/postgres/dealbot/service.yaml b/k8s/apps/postgres/dealbot/service.yaml
new file mode 100644
index 0000000..1990e55
--- /dev/null
+++ b/k8s/apps/postgres/dealbot/service.yaml
@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: dealbot-postgres
+spec:
+  type: ClusterIP
+  ports:
+    - name: postgres
+      port: 5432
+      targetPort: postgres
+  selector:
+    app.kubernetes.io/name: dealbot-postgres
+
diff --git a/k8s/apps/postgres/dealbot/statefulset.yaml b/k8s/apps/postgres/dealbot/statefulset.yaml
new file mode 100644
index 0000000..55adf97
--- /dev/null
+++ b/k8s/apps/postgres/dealbot/statefulset.yaml
@@ -0,0 +1,94 @@
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: dealbot-postgres
+  labels:
+    app.kubernetes.io/name: dealbot-postgres
+spec:
+  serviceName: dealbot-postgres
+  replicas: 1
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: dealbot-postgres
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: dealbot-postgres
+    spec:
+      initContainers:
+        - name: init-pgdata
+          image: busybox:1.36
+          command:
+            - sh
+            - -c
+            - |
+              set -eu
+              mkdir -p /var/lib/postgresql/data/pgdata
+              chown -R 70:70 /var/lib/postgresql/data/pgdata
+              chmod 700 /var/lib/postgresql/data/pgdata
+              mkdir -p /var/run/postgresql
+          securityContext:
+            runAsUser: 0
+            runAsGroup: 0
+            allowPrivilegeEscalation: false
+          volumeMounts:
+            - name: data
+              mountPath: /var/lib/postgresql/data
+            - name: run
+              mountPath: /var/run/postgresql
+      containers:
+        - name: postgres
+          image: postgres:16-alpine
+          ports:
+            - name: postgres
+              containerPort: 5432
+          env:
+            - name: PGDATA
+              value: /var/lib/postgresql/data/pgdata
+            - name: POSTGRES_USER
+              value: dealbot
+            - name: POSTGRES_DB
+              value: filecoin_dealbot
+            - name: POSTGRES_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: dealbot-secrets
+                  key: DATABASE_PASSWORD
+          securityContext:
+            allowPrivilegeEscalation: false
+          volumeMounts:
+            - name: data
+              mountPath: /var/lib/postgresql/data
+            - name: run
+              mountPath: /var/run/postgresql
+          startupProbe:
+            tcpSocket:
+              port: postgres
+            periodSeconds: 5
+            timeoutSeconds: 3
+            failureThreshold: 30
+          readinessProbe:
+            tcpSocket:
+              port: postgres
+            initialDelaySeconds: 5
+            periodSeconds: 5
+            timeoutSeconds: 3
+            failureThreshold: 6
+          livenessProbe:
+            tcpSocket:
+              port: postgres
+            initialDelaySeconds: 30
+            periodSeconds: 10
+            timeoutSeconds: 3
+            failureThreshold: 6
+      volumes:
+        - name: run
+          emptyDir: {}
+  volumeClaimTemplates:
+    - metadata:
+        name: data
+      spec:
+        accessModes: ["ReadWriteOnce"]
+        resources:
+          requests:
+            storage: 10Gi
diff --git a/k8s/clusters/prod/applications/dealbot.yaml b/k8s/clusters/prod/applications/dealbot.yaml
new file mode 100644
index 0000000..58138b8
--- /dev/null
+++ b/k8s/clusters/prod/applications/dealbot.yaml
@@ -0,0 +1,22 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: dealbot
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "3"
+spec:
+  project: default
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: filoz-dealbot
+  source:
+    repoURL: git@github.com:SgtPooki/homelab2.git
+    targetRevision: main
+    path: k8s/apps/dealbot/overlays/prod
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+      - CreateNamespace=true
diff --git a/k8s/clusters/prod/kustomization.yaml b/k8s/clusters/prod/kustomization.yaml
index 967ce48..9436d62 100644
--- a/k8s/clusters/prod/kustomization.yaml
+++ b/k8s/clusters/prod/kustomization.yaml
@@ -10,4 +10,4 @@ resources:
   - applications/homepage.yaml
   - applications/argocd-ingress.yaml
   - applications/whoami-public.yaml
-
+  - applications/dealbot.yaml

[BigLep]

2026-01-05 conversation between @BigLep and @SgtPooki :
Things that need to do:

• Update title since using kustomize and remove WIP
• Need to update to create images on "push to main" rather than pushes to this PR branch
• Mark as ready for review (no longer draft)
• Assign code reviewers
  • @BigLep as non blocking for reviewing documentation and understanding how parts fit together
  • @silent-cipher for understanding how this will deploy
  • @Kubuxu given he's been the partner in the overall infrastructure work

After this PR is done, there are FilOzone/infra changes. It still can be deployed to Coolify as is in the interim. @SgtPooki is going to make sure we have an issue(s) that lists the steps for getting us to a place where commits can be pushed to dealbot and they get deployed in a structured / reviewed way.