From a48890a8b46d69c54f04f8140d3d6151ebc26574 Mon Sep 17 00:00:00 2001 From: mikkeldamsgaard Date: Wed, 25 Feb 2026 14:01:04 +0100 Subject: [PATCH 1/2] feat(netbird): add PAT seeding via post-install/post-upgrade Job Add Personal Access Token (PAT) seeding capability to the NetBird Helm chart using Initium v1.0.1. When pat.enabled=true, a post-install/post-upgrade Job seeds a service user account and PAT into the database. Features: - New pat-seed-job.yaml: Helm hook Job that waits for the server and seeds PAT - New pat-seed-configmap.yaml: Initium seed specification - Supports all three database backends: SQLite, PostgreSQL, MySQL - Security contexts: root for SQLite (PVC access), non-root for external DBs - ConfigMap uses MiniJinja templates for sensitive token injection via env vars - Comprehensive unit tests (configmap and job) with 148 total tests passing Configuration (values.yaml): - pat.enabled: Enable/disable PAT seeding - pat.accountId/userId: IDs for the seeded account and service user - pat.name: PAT display name - pat.expirationDays: Token validity period (default 365 days) - pat.secret.secretName: K8s Secret containing the PAT token and hash E2E testing: - Updated e2e.sh with proper PAT token generation (nbp_ + 30-char secret + base62 CRC32 checksum) and curl-based authentication verification - All three backends (sqlite, postgres, mysql) tested and passing Dependencies: - Upgraded Initium to v1.0.1 (fixes PostgreSQL insert_row for text PKs) - Boolean values use 1/0 for MySQL tinyint compatibility - Datetime format uses space separator for MySQL compatibility --- CHANGELOG.md | 20 ++ charts/netbird/README.md | 82 ++++++ charts/netbird/templates/NOTES.txt | 8 + charts/netbird/templates/_helpers.tpl | 86 +++++++ .../netbird/templates/pat-seed-configmap.yaml | 23 ++ charts/netbird/templates/pat-seed-job.yaml | 114 +++++++++ .../tests/pat-seed-configmap_test.yaml | 139 ++++++++++ charts/netbird/tests/pat-seed-job_test.yaml | 240 ++++++++++++++++++ .../netbird/tests/server-deployment_test.yaml | 2 +- charts/netbird/values.yaml | 46 +++- ci/scripts/e2e.sh | 105 +++++++- 11 files changed, 856 insertions(+), 9 deletions(-) create mode 100644 charts/netbird/templates/pat-seed-configmap.yaml create mode 100644 charts/netbird/templates/pat-seed-job.yaml create mode 100644 charts/netbird/tests/pat-seed-configmap_test.yaml create mode 100644 charts/netbird/tests/pat-seed-job_test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a121d4..8286a2d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/). ## Unreleased +### Added + +- **PAT seeding**: Optional Personal Access Token seeding via `pat.*` values. + When `pat.enabled: true`, a post-install/post-upgrade Helm hook Job seeds + a service user account and PAT into the database using Initium's `seed` command. + The Job waits for the server to create its schema (GORM AutoMigrate), then + idempotently inserts account, user, and PAT records. +- PAT seed Job uses `wait-for` init container to wait for server TCP readiness + before seeding. +- PAT seed spec uses `wait_for` to wait for `accounts`, `users`, and + `personal_access_tokens` tables before inserting data. +- PAT seed data uses `unique_key` for idempotent inserts (safe on re-installs). +- PAT seed ConfigMap and Job are Helm hooks (post-install, post-upgrade) with + `before-hook-creation` delete policy. +- E2E tests extended to verify PAT authentication with `GET /api/groups` + across all three database backends (SQLite, PostgreSQL, MySQL). +- 28 new unit tests for PAT seed Job and ConfigMap templates. +- Upgraded Initium init container image to v1.0.1 (fixes PostgreSQL inserts + on tables with text primary keys). + ### Changed - **Breaking:** Replaced raw DSN secret (`server.secrets.storeDsn`) with structured diff --git a/charts/netbird/README.md b/charts/netbird/README.md index 2740067..af3689c 100644 --- a/charts/netbird/README.md +++ b/charts/netbird/README.md @@ -160,6 +160,75 @@ dashboard: - netbird.example.com ``` +## Personal Access Token (PAT) Seeding + +The chart can optionally seed the database with a Personal Access Token +after deployment. This enables immediate API access without manual token +creation — useful for automation, CI/CD, and GitOps workflows. + +### Generating a PAT + +NetBird PATs have the format `nbp_<30-char-secret><6-char-checksum>` (40 +chars total). The database stores a base64-encoded SHA256 hash. + +Generate a token and its hash: + +```bash +# Using Python +python3 -c " +import hashlib, base64, secrets, zlib +secret = secrets.token_urlsafe(22)[:30] +checksum = zlib.crc32(secret.encode()) & 0xffffffff +chars = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz' +cs = '' +v = checksum +while v > 0: cs = chars[v % 62] + cs; v //= 62 +token = 'nbp_' + secret + cs.rjust(6, '0') +hashed = base64.b64encode(hashlib.sha256(token.encode()).digest()).decode() +print(f'Token: {token}') +print(f'Hash: {hashed}') +" + +# Or using openssl +TOKEN="nbp_$(openssl rand -base64 32 | tr -dc 'a-zA-Z0-9' | head -c30)000000" +HASH=$(printf '%s' "$TOKEN" | openssl dgst -sha256 -binary | openssl base64 -A) +echo "Token: $TOKEN" +echo "Hash: $HASH" +``` + +### Creating the Secret + +```bash +kubectl create secret generic netbird-pat \ + --from-literal=token='nbp_...' \ + --from-literal=hashedToken='base64hash...' \ + -n netbird +``` + +### Enabling PAT Seeding + +```yaml +pat: + enabled: true + secret: + secretName: netbird-pat + name: "my-api-token" + expirationDays: 365 +``` + +The chart creates a post-install/post-upgrade Helm hook Job that: +1. Waits for the server to be reachable (TCP probe via Initium `wait-for`) +2. Waits for the `accounts`, `users`, and `personal_access_tokens` tables + (Initium seed `wait_for`) +3. Idempotently inserts a service user account and PAT + +### Using the PAT + +```bash +# Authenticate with the PAT +curl -H "Authorization: Token nbp_..." https://netbird.example.com/api/groups +``` + ## Values Reference ### Global @@ -186,6 +255,19 @@ dashboard: | `database.passwordSecret.secretKey` | string | `"password"` | Key in the Secret | | `database.sslMode` | string | `"disable"` | SSL mode for PostgreSQL (ignored for mysql/sqlite) | +### PAT (Personal Access Token) + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `pat.enabled` | bool | `false` | Enable PAT seeding via post-install Job | +| `pat.secret.secretName` | string | `""` | Kubernetes Secret containing token and hash | +| `pat.secret.tokenKey` | string | `"token"` | Key in Secret for the plaintext PAT | +| `pat.secret.hashedTokenKey` | string | `"hashedToken"` | Key in Secret for the base64-encoded SHA256 hash | +| `pat.name` | string | `"helm-seeded-token"` | Display name for the PAT | +| `pat.userId` | string | `"helm-seed-user"` | User ID for the service user | +| `pat.accountId` | string | `"helm-seed-account"` | Account ID for the service user | +| `pat.expirationDays` | int | `365` | PAT expiration in days from deployment | + ### Server | Key | Type | Default | Description | diff --git a/charts/netbird/templates/NOTES.txt b/charts/netbird/templates/NOTES.txt index b125ca8..3fd6365 100644 --- a/charts/netbird/templates/NOTES.txt +++ b/charts/netbird/templates/NOTES.txt @@ -34,3 +34,11 @@ Components: {{- end }} Database: {{ .Values.database.type }} +{{- if .Values.pat.enabled }} + + PAT Seeding: ENABLED + A post-install Job will seed a Personal Access Token into the database. + Secret: {{ .Values.pat.secret.secretName }} + The plaintext PAT can be retrieved with: + kubectl get secret {{ .Values.pat.secret.secretName }} -n {{ .Release.Namespace }} -o jsonpath='{.data.{{ .Values.pat.secret.tokenKey }}}' | base64 -d +{{- end }} diff --git a/charts/netbird/templates/_helpers.tpl b/charts/netbird/templates/_helpers.tpl index eb6b613..1f7cc73 100644 --- a/charts/netbird/templates/_helpers.tpl +++ b/charts/netbird/templates/_helpers.tpl @@ -236,3 +236,89 @@ phases: database: {{ .Values.database.name }} create_if_missing: true {{- end }} +{{/* +netbird.database.patDatabaseUrl — constructs the database URL for PAT seeding. +This URL connects to the target database (not the system database). +For sqlite, it points to the database file. +The spec is a MiniJinja template — {{ env.DB_PASSWORD }} is resolved +by Initium at runtime from the DB_PASSWORD environment variable. +*/}} +{{- define "netbird.database.patDatabaseUrl" -}} +{{- if eq .Values.database.type "sqlite" -}} +/var/lib/netbird/store.db +{{- else if eq .Values.database.type "postgresql" -}} +postgres://{{ .Values.database.user }}:{{ "{{ env.DB_PASSWORD }}" }}@{{ .Values.database.host }}:{{ include "netbird.database.port" . }}/{{ .Values.database.name }}?sslmode={{ .Values.database.sslMode }} +{{- else if eq .Values.database.type "mysql" -}} +mysql://{{ .Values.database.user }}:{{ "{{ env.DB_PASSWORD }}" }}@{{ .Values.database.host }}:{{ include "netbird.database.port" . }}/{{ .Values.database.name }} +{{- end -}} +{{- end }} +{{/* +netbird.pat.seedSpec — renders the Initium seed spec YAML for +inserting a Personal Access Token into the database. +The seed waits for the personal_access_tokens table (created by NetBird +on startup via GORM AutoMigrate), then idempotently inserts the +account, user, and PAT records. +MiniJinja placeholders: + {{ env.PAT_HASHED_TOKEN }} — base64-encoded SHA256 hash of the PAT +*/}} +{{- define "netbird.pat.seedSpec" -}} +database: + driver: {{ include "netbird.database.engine" . }} + url: "{{ include "netbird.database.patDatabaseUrl" . }}" +phases: + - name: seed-pat + order: 1 + wait_for: + - type: table + name: personal_access_tokens + timeout: 120s + - type: table + name: users + timeout: 120s + - type: table + name: accounts + timeout: 120s + seed_sets: + - name: pat-account + order: 1 + tables: + - table: accounts + unique_key: [id] + rows: + - id: {{ .Values.pat.accountId | quote }} + created_by: "helm-seed" + domain: "netbird.selfhosted" + domain_category: "private" + is_domain_primary_account: 1 + network_identifier: "seed-network" + network_net: "100.64.0.0/10" + network_dns: "" + network_serial: 0 + - name: pat-user + order: 2 + tables: + - table: users + unique_key: [id] + rows: + - id: {{ .Values.pat.userId | quote }} + account_id: {{ .Values.pat.accountId | quote }} + role: "admin" + is_service_user: 1 + service_user_name: "helm-seed-service-user" + non_deletable: 0 + blocked: 0 + issued: "api" + - name: pat-token + order: 3 + tables: + - table: personal_access_tokens + unique_key: [id] + rows: + - id: "helm-seeded-pat" + user_id: {{ .Values.pat.userId | quote }} + name: {{ .Values.pat.name | quote }} + hashed_token: "{{ "{{ env.PAT_HASHED_TOKEN }}" }}" + expiration_date: {{ now | dateModify (printf "+%dh" (mul .Values.pat.expirationDays 24)) | date "2006-01-02 15:04:05" | quote }} + created_by: {{ .Values.pat.userId | quote }} + created_at: {{ now | date "2006-01-02 15:04:05" | quote }} +{{- end }} diff --git a/charts/netbird/templates/pat-seed-configmap.yaml b/charts/netbird/templates/pat-seed-configmap.yaml new file mode 100644 index 0000000..00108fd --- /dev/null +++ b/charts/netbird/templates/pat-seed-configmap.yaml @@ -0,0 +1,23 @@ +{{/* +ConfigMap for the PAT seed spec. Only rendered when pat.enabled is true. +Contains the Initium seed spec that inserts account, user, and PAT records. +*/}} +{{- if .Values.pat.enabled }} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "netbird.server.fullname" . }}-pat-seed + namespace: {{ .Release.Namespace }} + labels: + {{- include "netbird.labels" . | nindent 4 }} + app.kubernetes.io/name: {{ include "netbird.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: pat-seed + annotations: + helm.sh/hook: post-install,post-upgrade + helm.sh/hook-weight: "0" + helm.sh/hook-delete-policy: before-hook-creation +data: + pat-seed.yaml: | + {{- include "netbird.pat.seedSpec" . | nindent 4 }} +{{- end }} diff --git a/charts/netbird/templates/pat-seed-job.yaml b/charts/netbird/templates/pat-seed-job.yaml new file mode 100644 index 0000000..a48b71d --- /dev/null +++ b/charts/netbird/templates/pat-seed-job.yaml @@ -0,0 +1,114 @@ +{{/* +Post-install/post-upgrade Job - seeds a Personal Access Token into the +NetBird database via Initium seed command. Only rendered when +pat.enabled is true. +*/}} +{{- if .Values.pat.enabled }} +{{- $isExternal := eq (include "netbird.database.isExternal" .) "true" }} +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ include "netbird.server.fullname" . }}-pat-seed + namespace: {{ .Release.Namespace }} + labels: + {{- include "netbird.labels" . | nindent 4 }} + app.kubernetes.io/name: {{ include "netbird.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: pat-seed + annotations: + helm.sh/hook: post-install,post-upgrade + helm.sh/hook-weight: "5" + helm.sh/hook-delete-policy: before-hook-creation +spec: + backoffLimit: 3 + ttlSecondsAfterFinished: 300 + template: + metadata: + labels: + app.kubernetes.io/name: {{ include "netbird.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: pat-seed + spec: + restartPolicy: OnFailure + serviceAccountName: {{ include "netbird.serviceAccountName" . }} + {{- with (coalesce .Values.server.imagePullSecrets .Values.imagePullSecrets) }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + initContainers: + - name: wait-server + image: "{{ .Values.server.initImage.repository }}:{{ .Values.server.initImage.tag }}" + args: + - wait-for + - --target + - "tcp://{{ include "netbird.server.fullname" . }}:{{ .Values.server.service.port }}" + - --timeout + - "180s" + securityContext: + runAsNonRoot: true + runAsUser: 65534 + readOnlyRootFilesystem: true + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + containers: + - name: pat-seed + image: "{{ .Values.server.initImage.repository }}:{{ .Values.server.initImage.tag }}" + args: + - seed + - --spec + - /spec/pat-seed.yaml + env: + - name: PAT_HASHED_TOKEN + valueFrom: + secretKeyRef: + name: {{ .Values.pat.secret.secretName }} + key: {{ .Values.pat.secret.hashedTokenKey }} + {{- if $isExternal }} + - name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: {{ .Values.database.passwordSecret.secretName }} + key: {{ .Values.database.passwordSecret.secretKey }} + {{- end }} + securityContext: + {{- if eq .Values.database.type "sqlite" }} + runAsUser: 0 + {{- else }} + runAsNonRoot: true + runAsUser: 65534 + {{- end }} + readOnlyRootFilesystem: true + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + volumeMounts: + - name: config + mountPath: /spec + readOnly: true + {{- if eq .Values.database.type "sqlite" }} + - name: data + mountPath: /var/lib/netbird + {{- end }} + volumes: + - name: config + configMap: + name: {{ include "netbird.server.fullname" . }}-pat-seed + {{- if eq .Values.database.type "sqlite" }} + - name: data + {{- if .Values.server.persistentVolume.enabled }} + persistentVolumeClaim: + claimName: {{ include "netbird.server.fullname" . }} + {{- else }} + emptyDir: {} + {{- end }} + {{- end }} + {{- with .Values.server.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.server.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/charts/netbird/tests/pat-seed-configmap_test.yaml b/charts/netbird/tests/pat-seed-configmap_test.yaml new file mode 100644 index 0000000..7f3f73c --- /dev/null +++ b/charts/netbird/tests/pat-seed-configmap_test.yaml @@ -0,0 +1,139 @@ +suite: PAT seed configmap tests +templates: + - templates/pat-seed-configmap.yaml +tests: + - it: should not render when pat.enabled is false + asserts: + - hasDocuments: + count: 0 + + - it: should render when pat.enabled is true + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - hasDocuments: + count: 1 + - isKind: + of: ConfigMap + + - it: should have helm hook annotations + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - equal: + path: metadata.annotations["helm.sh/hook"] + value: "post-install,post-upgrade" + - equal: + path: metadata.annotations["helm.sh/hook-weight"] + value: "0" + + - it: should contain pat-seed.yaml key + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - exists: + path: data["pat-seed.yaml"] + + - it: should match seed spec snapshot for sqlite + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + pat.accountId: test-account + pat.userId: test-user + pat.name: test-pat-name + asserts: + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "driver: sqlite" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "url: \"/var/lib/netbird/store.db\"" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "id: \"test-account\"" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "id: \"test-user\"" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "name: \"test-pat-name\"" + + - it: should use postgresql database URL when database.type is postgresql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.port: 5432 + database.user: netbird + database.name: netbird + database.sslMode: disable + database.passwordSecret.secretName: pg-secret + asserts: + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "driver: postgres" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "postgres://netbird.*pg\\.example\\.com:5432/netbird" + + - it: should use mysql database URL when database.type is mysql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: mysql + database.host: mysql.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: mysql-secret + asserts: + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "driver: mysql" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "mysql://netbird.*mysql\\.example\\.com:3306/netbird" + + - it: should wait for required tables + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "name: personal_access_tokens" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "name: users" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "name: accounts" + + - it: should use PAT_HASHED_TOKEN MiniJinja placeholder + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "env\\.PAT_HASHED_TOKEN" + + - it: should seed account user and pat tables + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "table: accounts" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "table: users" + - matchRegex: + path: data["pat-seed.yaml"] + pattern: "table: personal_access_tokens" + diff --git a/charts/netbird/tests/pat-seed-job_test.yaml b/charts/netbird/tests/pat-seed-job_test.yaml new file mode 100644 index 0000000..4cc1fcf --- /dev/null +++ b/charts/netbird/tests/pat-seed-job_test.yaml @@ -0,0 +1,240 @@ +suite: PAT seed job tests +templates: + - templates/pat-seed-job.yaml +tests: + # ── Not rendered when disabled ─────────────────────────────────────── + + - it: should not render when pat.enabled is false + asserts: + - hasDocuments: + count: 0 + + - it: should not render when pat is not configured + set: + pat.enabled: false + asserts: + - hasDocuments: + count: 0 + + # ── Basic rendering ───────────────────────────────────────────────── + + - it: should render a Job when pat.enabled is true + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - hasDocuments: + count: 1 + - isKind: + of: Job + + - it: should have correct helm hook annotations + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - equal: + path: metadata.annotations["helm.sh/hook"] + value: "post-install,post-upgrade" + - equal: + path: metadata.annotations["helm.sh/hook-weight"] + value: "5" + - equal: + path: metadata.annotations["helm.sh/hook-delete-policy"] + value: "before-hook-creation" + + - it: should include pat-seed component label + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - equal: + path: metadata.labels["app.kubernetes.io/component"] + value: pat-seed + + # ── Init container: wait-server ────────────────────────────────────── + + - it: should have wait-server init container using initium + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - equal: + path: spec.template.spec.initContainers[0].name + value: wait-server + - equal: + path: spec.template.spec.initContainers[0].image + value: "ghcr.io/kitstream/initium:1.0.1" + - contains: + path: spec.template.spec.initContainers[0].args + content: "tcp://RELEASE-NAME-netbird-server:80" + + - it: should set hardened securityContext on wait-server + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - equal: + path: spec.template.spec.initContainers[0].securityContext.runAsNonRoot + value: true + - equal: + path: spec.template.spec.initContainers[0].securityContext.readOnlyRootFilesystem + value: true + + # ── Main container: pat-seed ───────────────────────────────────────── + + - it: should have pat-seed container using initium + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - equal: + path: spec.template.spec.containers[0].name + value: pat-seed + - equal: + path: spec.template.spec.containers[0].image + value: "ghcr.io/kitstream/initium:1.0.1" + + - it: should inject PAT_HASHED_TOKEN env var from secret + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + pat.secret.hashedTokenKey: myHashKey + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: PAT_HASHED_TOKEN + valueFrom: + secretKeyRef: + name: my-pat-secret + key: myHashKey + + - it: should not inject DB_PASSWORD for sqlite + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - notContains: + path: spec.template.spec.containers[0].env + content: + name: DB_PASSWORD + any: true + + - it: should inject DB_PASSWORD for postgresql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret + database.passwordSecret.secretKey: dbpass + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: pg-secret + key: dbpass + + - it: should inject DB_PASSWORD for mysql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: mysql + database.host: mysql.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: mysql-secret + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: mysql-secret + key: password + + - it: should mount config volume + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: config + mountPath: /spec + readOnly: true + + - it: should mount data volume for sqlite + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: data + mountPath: /var/lib/netbird + + - it: should not mount data volume for postgresql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret + asserts: + - notContains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: data + mountPath: /var/lib/netbird + + # ── Volumes ────────────────────────────────────────────────────────── + + - it: should have config volume from configmap + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - contains: + path: spec.template.spec.volumes + content: + name: config + configMap: + name: RELEASE-NAME-netbird-server-pat-seed + + # ── Job spec ───────────────────────────────────────────────────────── + + - it: should have correct backoffLimit and ttl + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - equal: + path: spec.backoffLimit + value: 3 + - equal: + path: spec.ttlSecondsAfterFinished + value: 300 + + - it: should set restartPolicy to OnFailure + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + asserts: + - equal: + path: spec.template.spec.restartPolicy + value: OnFailure + diff --git a/charts/netbird/tests/server-deployment_test.yaml b/charts/netbird/tests/server-deployment_test.yaml index 3561dd0..0571e81 100644 --- a/charts/netbird/tests/server-deployment_test.yaml +++ b/charts/netbird/tests/server-deployment_test.yaml @@ -144,7 +144,7 @@ tests: value: config-init - equal: path: spec.template.spec.initContainers[0].image - value: "ghcr.io/kitstream/initium:1.0.0" + value: "ghcr.io/kitstream/initium:1.0.1" - it: should have only config-init for sqlite asserts: diff --git a/charts/netbird/values.yaml b/charts/netbird/values.yaml index 77045d5..59b90b7 100644 --- a/charts/netbird/values.yaml +++ b/charts/netbird/values.yaml @@ -77,6 +77,50 @@ database: sslMode: "disable" # ============================================================================= +# ============================================================================= +# PAT (Personal Access Token) Seeding +# ============================================================================= +# Optionally seed the NetBird database with a Personal Access Token after +# deployment. This enables immediate API access without manual token creation. +# +# Prerequisites: +# 1. Generate a PAT and its hash (see README for instructions) +# 2. Create a Kubernetes Secret with both values: +# +# kubectl create secret generic netbird-pat \ +# --from-literal=token='nbp_...' \ +# --from-literal=hashedToken='base64hash...' \ +# -n netbird +# ============================================================================= +pat: + # -- Enable PAT seeding. When true, a post-install Job inserts a + # Personal Access Token into the database. + enabled: false + + # -- Kubernetes Secret containing the PAT credentials. + secret: + # -- Name of the existing Secret with token and hashed token. + secretName: "" + # -- Key in the Secret holding the plaintext PAT (e.g. "nbp_..."). + tokenKey: "token" + # -- Key in the Secret holding the base64-encoded SHA256 hash. + hashedTokenKey: "hashedToken" + + # -- Display name for the seeded PAT. + name: "helm-seeded-token" + + # -- User ID to associate the PAT with. A service user with this ID + # will be created if it does not exist. + userId: "helm-seed-user" + + # -- Account ID for the seeded user. An account with this ID will be + # created if it does not exist. + accountId: "helm-seed-account" + + # -- PAT expiration in days from the time of deployment. + expirationDays: 365 + + # Combined NetBird Server (Management + Signal + Relay + STUN) # ============================================================================= server: @@ -106,7 +150,7 @@ server: # -- Init container image repository. repository: ghcr.io/kitstream/initium # -- Init container image tag. - tag: "1.0.0" + tag: "1.0.1" # -- Component-level image pull secrets. imagePullSecrets: [] diff --git a/ci/scripts/e2e.sh b/ci/scripts/e2e.sh index 11a7423..ad30d5d 100755 --- a/ci/scripts/e2e.sh +++ b/ci/scripts/e2e.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # -# E2E test runner for the netbird Helm chart. +# E2E test runner for the netbird Helm chart (with PAT seeding). # # Usage: # ci/scripts/e2e.sh @@ -16,7 +16,7 @@ BACKEND="${1:-sqlite}" RELEASE="netbird-e2e" NAMESPACE="netbird-e2e" CHART="charts/netbird" -TIMEOUT="5m" +TIMEOUT="10m" log() { echo "==> $*"; } fail() { echo "FAIL: $*" >&2; exit 1; } @@ -158,6 +158,33 @@ EOF --from-literal=password="testpassword" } +# ── Generate PAT for testing ─────────────────────────────────────────── +# NetBird PAT format: nbp_ (4) + secret (30) + base62(CRC32(secret)) (6) = 40 chars +generate_pat_secret() { + log "Generating PAT secret for testing..." + read -r PAT_TOKEN PAT_HASH < <(python3 -c " +import hashlib, base64, binascii + +BASE62 = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz' +def base62_encode(num, length=6): + r = [] + while num > 0: + r.append(BASE62[num % 62]) + num //= 62 + return ''.join(reversed(r)).rjust(length, '0') + +secret = 'TestSecretValue1234567890ABCDE' # exactly 30 chars +crc = binascii.crc32(secret.encode()) & 0xFFFFFFFF +token = 'nbp_' + secret + base62_encode(crc) +h = base64.b64encode(hashlib.sha256(token.encode()).digest()).decode() +print(token, h) +") + log "Test PAT token: $PAT_TOKEN (length=${#PAT_TOKEN})" + log "Test PAT hash: $PAT_HASH" + kubectl -n "$NAMESPACE" create secret generic netbird-pat \ + --from-literal=token="$PAT_TOKEN" \ + --from-literal=hashedToken="$PAT_HASH" +} case "$BACKEND" in sqlite) log "Using SQLite — no external database needed" @@ -176,16 +203,31 @@ case "$BACKEND" in ;; esac +# ── Create PAT secret ───────────────────────────────────────────────── +generate_pat_secret + # ── Install netbird chart ───────────────────────────────────────────── -log "Installing netbird chart with $BACKEND backend..." -helm install "$RELEASE" "$CHART" \ +log "Installing netbird chart with $BACKEND backend and PAT seeding..." +EXTRA_SETS=() +if [ "$BACKEND" = "sqlite" ]; then + EXTRA_SETS+=(--set server.persistentVolume.enabled=true) +fi +if ! helm install "$RELEASE" "$CHART" \ -n "$NAMESPACE" \ -f "$VALUES_FILE" \ - --wait --timeout "$TIMEOUT" + --set pat.enabled=true \ + --set pat.secret.secretName=netbird-pat \ + "${EXTRA_SETS[@]}" \ + --timeout "$TIMEOUT"; then + log "Helm install failed — dumping PAT seed job logs..." + kubectl -n "$NAMESPACE" logs job/"$RELEASE"-server-pat-seed --all-containers 2>/dev/null || true + kubectl -n "$NAMESPACE" describe job/"$RELEASE"-server-pat-seed 2>/dev/null || true + fail "Helm install failed" +fi # ── Verify rollout ──────────────────────────────────────────────────── log "Verifying deployments..." -kubectl -n "$NAMESPACE" rollout status deployment/"$RELEASE"-server --timeout=120s +kubectl -n "$NAMESPACE" rollout status deployment/"$RELEASE"-server --timeout=300s kubectl -n "$NAMESPACE" rollout status deployment/"$RELEASE"-dashboard --timeout=120s log "Pod status:" @@ -195,4 +237,53 @@ kubectl -n "$NAMESPACE" get pods -o wide log "Running helm test..." helm test "$RELEASE" -n "$NAMESPACE" --timeout 2m -log "E2E test with $BACKEND backend PASSED!" +# ── Wait for PAT seed job to complete ───────────────────────────────── +log "Waiting for PAT seed job to complete..." +kubectl -n "$NAMESPACE" wait --for=condition=complete \ + job/"$RELEASE"-server-pat-seed --timeout=180s || { + log "PAT seed job logs:" + kubectl -n "$NAMESPACE" logs job/"$RELEASE"-server-pat-seed --all-containers || true + fail "PAT seed job did not complete" +} +# ── Verify PAT authentication ───────────────────────────────────────── +log "Verifying PAT authentication against API..." +# Re-derive the PAT_TOKEN (same deterministic generation as above) +PAT_TOKEN=$(python3 -c " +import binascii +BASE62='0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz' +def b62(n,l=6): + r=[] + while n>0: r.append(BASE62[n%62]); n//=62 + return ''.join(reversed(r)).rjust(l,'0') +s='TestSecretValue1234567890ABCDE' +print('nbp_'+s+b62(binascii.crc32(s.encode())&0xFFFFFFFF)) +") +SVC_URL="http://$RELEASE-server.$NAMESPACE.svc.cluster.local:80" +kubectl -n "$NAMESPACE" run pat-test --image=alpine:3.20 --restart=Never \ + --command -- sh -c " + apk add --no-cache curl >/dev/null 2>&1 + sleep 3 + echo '==> Testing PAT auth on /api/groups...' + HTTP_CODE=\$(curl -s -o /tmp/body -w '%{http_code}' \ + -H 'Authorization: Token $PAT_TOKEN' \ + '$SVC_URL/api/groups') + echo \"HTTP status: \$HTTP_CODE\" + echo \"Body: \$(cat /tmp/body | head -c 500)\" + if [ \"\$HTTP_CODE\" = '200' ]; then + echo 'PASS: PAT authentication accepted (200 OK)' + exit 0 + else + echo \"FAIL: Expected HTTP 200, got \$HTTP_CODE\" + exit 1 + fi + " +log "Waiting for PAT test pod..." +kubectl -n "$NAMESPACE" wait --for=condition=Ready pod/pat-test --timeout=60s 2>/dev/null || true +kubectl -n "$NAMESPACE" wait --for=jsonpath='{.status.phase}'=Succeeded pod/pat-test --timeout=60s || { + log "PAT test pod logs:" + kubectl -n "$NAMESPACE" logs pat-test || true + fail "PAT authentication test failed" +} +log "PAT test pod logs:" +kubectl -n "$NAMESPACE" logs pat-test || true +log "E2E test with $BACKEND backend PASSED (including PAT seeding)!" From e5d3f1f768ea0fcb6bed52057d68b08df10dbc6f Mon Sep 17 00:00:00 2001 From: mikkeldamsgaard Date: Wed, 25 Feb 2026 21:29:35 +0100 Subject: [PATCH 2/2] fix(netbird): use native sidecar for SQLite PAT seeding The PAT seed Job cannot mount a ReadWriteOnce PVC that is already attached to the server pod. Replace the Job with a Kubernetes native sidecar (init container with restartPolicy: Always, K8s 1.28+) that shares the data volume inside the same pod. Changes: - server-deployment.yaml: add pat-seed native sidecar with --sidecar flag (Initium v1.0.4) to stay alive after seeding, maintaining full pod readiness (2/2 Running) - pat-seed-job.yaml: only render for external databases (PostgreSQL/MySQL) - pat-seed-configmap.yaml: regular resource for SQLite (needed by Deployment), Helm hook for external DBs (needed by hook Job) - _helpers.tpl: remove gob-encoded network_* columns from accounts seed to prevent migratePreAuto decode errors on server restart - values.yaml: bump Initium from v1.0.1 to v1.0.4 - e2e.sh: check sidecar logs for completion instead of container termination (sidecar stays alive with --sidecar flag) - 167 unit tests passing, SQLite e2e test passing with PAT auth Co-Authored-By: Claude Opus 4.6 --- CHANGELOG.md | 28 ++- charts/netbird/README.md | 27 ++- charts/netbird/templates/NOTES.txt | 6 + charts/netbird/templates/_helpers.tpl | 4 - .../netbird/templates/pat-seed-configmap.yaml | 7 + charts/netbird/templates/pat-seed-job.yaml | 29 +-- .../netbird/templates/server-deployment.yaml | 43 ++++ .../tests/pat-seed-configmap_test.yaml | 35 ++- charts/netbird/tests/pat-seed-job_test.yaml | 135 ++++++++--- .../netbird/tests/server-deployment_test.yaml | 210 +++++++++++++++++- charts/netbird/values.yaml | 2 +- ci/scripts/e2e.sh | 51 ++++- 12 files changed, 490 insertions(+), 87 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8286a2d..35e0cec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,22 +10,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/). ### Added - **PAT seeding**: Optional Personal Access Token seeding via `pat.*` values. - When `pat.enabled: true`, a post-install/post-upgrade Helm hook Job seeds - a service user account and PAT into the database using Initium's `seed` command. - The Job waits for the server to create its schema (GORM AutoMigrate), then - idempotently inserts account, user, and PAT records. -- PAT seed Job uses `wait-for` init container to wait for server TCP readiness - before seeding. + When `pat.enabled: true`, a service user account and PAT are seeded into + the database using Initium's `seed` command. The seed waits for the server + to create its schema (GORM AutoMigrate), then idempotently inserts account, + user, and PAT records. +- **SQLite**: PAT seed runs as a Kubernetes native sidecar (init container with + `restartPolicy: Always`, K8s 1.28+) in the server Deployment. The sidecar + uses Initium's `--sidecar` flag to stay alive after seeding, maintaining + full pod readiness (`2/2 Running`). This avoids ReadWriteOnce PVC + multi-attach issues that prevent a separate Job from mounting the PVC. +- **PostgreSQL/MySQL**: PAT seed runs as a post-install/post-upgrade Helm hook + Job with a `wait-for` init container for server TCP readiness. - PAT seed spec uses `wait_for` to wait for `accounts`, `users`, and `personal_access_tokens` tables before inserting data. - PAT seed data uses `unique_key` for idempotent inserts (safe on re-installs). -- PAT seed ConfigMap and Job are Helm hooks (post-install, post-upgrade) with - `before-hook-creation` delete policy. +- PAT seed ConfigMap is a regular release resource for SQLite and a Helm hook + for external databases. - E2E tests extended to verify PAT authentication with `GET /api/groups` across all three database backends (SQLite, PostgreSQL, MySQL). -- 28 new unit tests for PAT seed Job and ConfigMap templates. -- Upgraded Initium init container image to v1.0.1 (fixes PostgreSQL inserts - on tables with text primary keys). +- Unit tests for PAT seed Job, ConfigMap, and sidecar templates. +- Upgraded Initium init container image to v1.0.4 (adds `--sidecar` flag for + keeping the process alive after task completion, SHA256/base64 template + filters, PostgreSQL text primary key fix). ### Changed diff --git a/charts/netbird/README.md b/charts/netbird/README.md index af3689c..c24867e 100644 --- a/charts/netbird/README.md +++ b/charts/netbird/README.md @@ -20,7 +20,7 @@ For external databases (PostgreSQL, MySQL), the chart automatically: ## Prerequisites -- Kubernetes 1.24+ +- Kubernetes 1.24+ (1.28+ required for SQLite PAT seeding with native sidecars) - Helm 3.x - An OAuth2 / OIDC identity provider (Auth0, Keycloak, Authentik, Zitadel, etc.) - An Ingress controller (nginx recommended) with TLS termination @@ -216,11 +216,24 @@ pat: expirationDays: 365 ``` -The chart creates a post-install/post-upgrade Helm hook Job that: -1. Waits for the server to be reachable (TCP probe via Initium `wait-for`) -2. Waits for the `accounts`, `users`, and `personal_access_tokens` tables - (Initium seed `wait_for`) -3. Idempotently inserts a service user account and PAT +The seeding mechanism depends on the database type: + +- **SQLite**: The seed runs as a **native sidecar** (Kubernetes 1.28+) in the + server Deployment. It is declared as an init container with + `restartPolicy: Always` and uses the `--sidecar` flag to stay alive after + seeding. This is required because SQLite uses a local file and + ReadWriteOnce PVCs cannot be mounted by multiple pods simultaneously. +- **PostgreSQL / MySQL**: The seed runs as a post-install/post-upgrade Helm + hook Job that connects to the database over the network. + +In both cases, the seed: +1. Waits for the `accounts`, `users`, and `personal_access_tokens` tables + to exist (created by the server via GORM AutoMigrate) +2. Idempotently inserts a service user account and PAT + +> **Note:** The SQLite PAT sidecar requires **Kubernetes 1.28+** for native +> sidecar support. The sidecar stays alive after completing the seed +> (via Initium's `--sidecar` flag), so the pod shows `2/2 Running`. ### Using the PAT @@ -277,7 +290,7 @@ curl -H "Authorization: Token nbp_..." https://netbird.example.com/api/groups | `server.image.tag` | string | `""` (appVersion) | Server image tag | | `server.image.pullPolicy` | string | `"IfNotPresent"` | Image pull policy | | `server.initImage.repository` | string | `"ghcr.io/kitstream/initium"` | Init container image ([Initium](https://github.com/KitStream/initium)) | -| `server.initImage.tag` | string | `"1.0.0"` | Init container image tag | +| `server.initImage.tag` | string | `"1.0.4"` | Init container image tag | | `server.imagePullSecrets` | list | `[]` | Component-level pull secrets | #### Server Configuration diff --git a/charts/netbird/templates/NOTES.txt b/charts/netbird/templates/NOTES.txt index 3fd6365..16cc957 100644 --- a/charts/netbird/templates/NOTES.txt +++ b/charts/netbird/templates/NOTES.txt @@ -37,7 +37,13 @@ Components: {{- if .Values.pat.enabled }} PAT Seeding: ENABLED + {{- if eq .Values.database.type "sqlite" }} + A sidecar container in the server pod will seed a Personal Access Token + into the SQLite database. The sidecar shares the data volume with the + server (required for ReadWriteOnce PVCs). + {{- else }} A post-install Job will seed a Personal Access Token into the database. + {{- end }} Secret: {{ .Values.pat.secret.secretName }} The plaintext PAT can be retrieved with: kubectl get secret {{ .Values.pat.secret.secretName }} -n {{ .Release.Namespace }} -o jsonpath='{.data.{{ .Values.pat.secret.tokenKey }}}' | base64 -d diff --git a/charts/netbird/templates/_helpers.tpl b/charts/netbird/templates/_helpers.tpl index 1f7cc73..7022e1c 100644 --- a/charts/netbird/templates/_helpers.tpl +++ b/charts/netbird/templates/_helpers.tpl @@ -290,10 +290,6 @@ phases: domain: "netbird.selfhosted" domain_category: "private" is_domain_primary_account: 1 - network_identifier: "seed-network" - network_net: "100.64.0.0/10" - network_dns: "" - network_serial: 0 - name: pat-user order: 2 tables: diff --git a/charts/netbird/templates/pat-seed-configmap.yaml b/charts/netbird/templates/pat-seed-configmap.yaml index 00108fd..0fe4768 100644 --- a/charts/netbird/templates/pat-seed-configmap.yaml +++ b/charts/netbird/templates/pat-seed-configmap.yaml @@ -1,8 +1,13 @@ {{/* ConfigMap for the PAT seed spec. Only rendered when pat.enabled is true. Contains the Initium seed spec that inserts account, user, and PAT records. + +For SQLite this is a regular release resource (consumed by the sidecar in the +Deployment). For external databases it remains a Helm hook so the seed Job +can reference it during the post-install/post-upgrade phase. */}} {{- if .Values.pat.enabled }} +{{- $isExternal := eq (include "netbird.database.isExternal" .) "true" }} apiVersion: v1 kind: ConfigMap metadata: @@ -13,10 +18,12 @@ metadata: app.kubernetes.io/name: {{ include "netbird.name" . }} app.kubernetes.io/instance: {{ .Release.Name }} app.kubernetes.io/component: pat-seed + {{- if $isExternal }} annotations: helm.sh/hook: post-install,post-upgrade helm.sh/hook-weight: "0" helm.sh/hook-delete-policy: before-hook-creation + {{- end }} data: pat-seed.yaml: | {{- include "netbird.pat.seedSpec" . | nindent 4 }} diff --git a/charts/netbird/templates/pat-seed-job.yaml b/charts/netbird/templates/pat-seed-job.yaml index a48b71d..f43d299 100644 --- a/charts/netbird/templates/pat-seed-job.yaml +++ b/charts/netbird/templates/pat-seed-job.yaml @@ -1,10 +1,15 @@ {{/* Post-install/post-upgrade Job - seeds a Personal Access Token into the -NetBird database via Initium seed command. Only rendered when -pat.enabled is true. +NetBird database via Initium seed command. + +Only rendered for external databases (PostgreSQL/MySQL) where the seed +connects over the network. For SQLite the seed runs as a sidecar container +in the server Deployment to share the same PVC (ReadWriteOnce PVCs do not +support multi-pod attach). */}} {{- if .Values.pat.enabled }} {{- $isExternal := eq (include "netbird.database.isExternal" .) "true" }} +{{- if $isExternal }} apiVersion: batch/v1 kind: Job metadata: @@ -64,20 +69,14 @@ spec: secretKeyRef: name: {{ .Values.pat.secret.secretName }} key: {{ .Values.pat.secret.hashedTokenKey }} - {{- if $isExternal }} - name: DB_PASSWORD valueFrom: secretKeyRef: name: {{ .Values.database.passwordSecret.secretName }} key: {{ .Values.database.passwordSecret.secretKey }} - {{- end }} securityContext: - {{- if eq .Values.database.type "sqlite" }} - runAsUser: 0 - {{- else }} runAsNonRoot: true runAsUser: 65534 - {{- end }} readOnlyRootFilesystem: true allowPrivilegeEscalation: false capabilities: @@ -86,23 +85,10 @@ spec: - name: config mountPath: /spec readOnly: true - {{- if eq .Values.database.type "sqlite" }} - - name: data - mountPath: /var/lib/netbird - {{- end }} volumes: - name: config configMap: name: {{ include "netbird.server.fullname" . }}-pat-seed - {{- if eq .Values.database.type "sqlite" }} - - name: data - {{- if .Values.server.persistentVolume.enabled }} - persistentVolumeClaim: - claimName: {{ include "netbird.server.fullname" . }} - {{- else }} - emptyDir: {} - {{- end }} - {{- end }} {{- with .Values.server.nodeSelector }} nodeSelector: {{- toYaml . | nindent 8 }} @@ -112,3 +98,4 @@ spec: {{- toYaml . | nindent 8 }} {{- end }} {{- end }} +{{- end }} diff --git a/charts/netbird/templates/server-deployment.yaml b/charts/netbird/templates/server-deployment.yaml index 2a9fe25..4f3de3d 100644 --- a/charts/netbird/templates/server-deployment.yaml +++ b/charts/netbird/templates/server-deployment.yaml @@ -14,6 +14,9 @@ spec: metadata: annotations: checksum/config: {{ include "netbird.server.configTemplate" . | sha256sum }} + {{- if and .Values.pat.enabled (eq .Values.database.type "sqlite") }} + checksum/pat-seed: {{ include "netbird.pat.seedSpec" . | sha256sum }} + {{- end }} {{- with .Values.server.podAnnotations }} {{- toYaml . | nindent 8 }} {{- end }} @@ -36,6 +39,7 @@ spec: {{- $authSecretName := include "netbird.server.resolveSecretName" (dict "ref" .Values.server.secrets.authSecret "generated" $generated) }} {{- $encKeySecretName := include "netbird.server.resolveSecretName" (dict "ref" .Values.server.secrets.storeEncryptionKey "generated" $generated) }} {{- $isExternal := eq (include "netbird.database.isExternal" .) "true" }} + {{- $patSidecar := and .Values.pat.enabled (not $isExternal) }} initContainers: {{- if $isExternal }} # ── Wait for database to be reachable ────────────────────────── @@ -133,6 +137,40 @@ spec: readOnly: true - name: config-generated mountPath: /out + {{- if $patSidecar }} + # ── PAT seed native sidecar (SQLite only) ──────────────────────── + # Declared as an init container with restartPolicy: Always, making + # it a Kubernetes native sidecar (K8s 1.28+). It starts alongside + # the server and stays alive after seeding (--sidecar flag). + # Required because ReadWriteOnce PVCs do not support multi-pod + # attach, so a separate Job cannot mount the PVC. + - name: pat-seed + restartPolicy: Always + image: "{{ .Values.server.initImage.repository }}:{{ .Values.server.initImage.tag }}" + args: + - --sidecar + - seed + - --spec + - /spec/pat-seed.yaml + env: + - name: PAT_HASHED_TOKEN + valueFrom: + secretKeyRef: + name: {{ .Values.pat.secret.secretName }} + key: {{ .Values.pat.secret.hashedTokenKey }} + securityContext: + runAsUser: 0 + readOnlyRootFilesystem: true + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + volumeMounts: + - name: pat-seed-config + mountPath: /spec + readOnly: true + - name: data + mountPath: /var/lib/netbird + {{- end }} containers: - name: server image: "{{ .Values.server.image.repository }}:{{ .Values.server.image.tag | default .Chart.AppVersion }}" @@ -182,6 +220,11 @@ spec: name: {{ include "netbird.server.fullname" . }} - name: config-generated emptyDir: {} + {{- if $patSidecar }} + - name: pat-seed-config + configMap: + name: {{ include "netbird.server.fullname" . }}-pat-seed + {{- end }} - name: data {{- if .Values.server.persistentVolume.enabled }} persistentVolumeClaim: diff --git a/charts/netbird/tests/pat-seed-configmap_test.yaml b/charts/netbird/tests/pat-seed-configmap_test.yaml index 7f3f73c..b721a30 100644 --- a/charts/netbird/tests/pat-seed-configmap_test.yaml +++ b/charts/netbird/tests/pat-seed-configmap_test.yaml @@ -17,10 +17,26 @@ tests: - isKind: of: ConfigMap - - it: should have helm hook annotations + # ── Hook annotations: only for external databases ────────────────── + + - it: should not have helm hook annotations for sqlite + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - notExists: + path: metadata.annotations["helm.sh/hook"] + + - it: should have helm hook annotations for postgresql set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - equal: path: metadata.annotations["helm.sh/hook"] @@ -29,6 +45,22 @@ tests: path: metadata.annotations["helm.sh/hook-weight"] value: "0" + - it: should have helm hook annotations for mysql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: mysql + database.host: mysql.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: mysql-secret + asserts: + - equal: + path: metadata.annotations["helm.sh/hook"] + value: "post-install,post-upgrade" + + # ── Seed spec content ────────────────────────────────────────────── + - it: should contain pat-seed.yaml key set: pat.enabled: true @@ -136,4 +168,3 @@ tests: - matchRegex: path: data["pat-seed.yaml"] pattern: "table: personal_access_tokens" - diff --git a/charts/netbird/tests/pat-seed-job_test.yaml b/charts/netbird/tests/pat-seed-job_test.yaml index 4cc1fcf..7fd9a40 100644 --- a/charts/netbird/tests/pat-seed-job_test.yaml +++ b/charts/netbird/tests/pat-seed-job_test.yaml @@ -16,12 +16,53 @@ tests: - hasDocuments: count: 0 - # ── Basic rendering ───────────────────────────────────────────────── + # ── Not rendered for SQLite (uses sidecar instead) ───────────────── - - it: should render a Job when pat.enabled is true + - it: should not render for sqlite even when pat.enabled is true set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - hasDocuments: + count: 0 + + - it: should not render for sqlite with PV enabled + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + server.persistentVolume.enabled: true + asserts: + - hasDocuments: + count: 0 + + # ── Basic rendering (external DB only) ───────────────────────────── + + - it: should render a Job for postgresql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret + asserts: + - hasDocuments: + count: 1 + - isKind: + of: Job + + - it: should render a Job for mysql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: mysql + database.host: mysql.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: mysql-secret asserts: - hasDocuments: count: 1 @@ -32,6 +73,11 @@ tests: set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - equal: path: metadata.annotations["helm.sh/hook"] @@ -47,6 +93,11 @@ tests: set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - equal: path: metadata.labels["app.kubernetes.io/component"] @@ -58,13 +109,18 @@ tests: set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - equal: path: spec.template.spec.initContainers[0].name value: wait-server - equal: path: spec.template.spec.initContainers[0].image - value: "ghcr.io/kitstream/initium:1.0.1" + value: "ghcr.io/kitstream/initium:1.0.4" - contains: path: spec.template.spec.initContainers[0].args content: "tcp://RELEASE-NAME-netbird-server:80" @@ -73,6 +129,11 @@ tests: set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - equal: path: spec.template.spec.initContainers[0].securityContext.runAsNonRoot @@ -87,19 +148,29 @@ tests: set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - equal: path: spec.template.spec.containers[0].name value: pat-seed - equal: path: spec.template.spec.containers[0].image - value: "ghcr.io/kitstream/initium:1.0.1" + value: "ghcr.io/kitstream/initium:1.0.4" - it: should inject PAT_HASHED_TOKEN env var from secret set: pat.enabled: true pat.secret.secretName: my-pat-secret pat.secret.hashedTokenKey: myHashKey + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - contains: path: spec.template.spec.containers[0].env @@ -110,18 +181,6 @@ tests: name: my-pat-secret key: myHashKey - - it: should not inject DB_PASSWORD for sqlite - set: - pat.enabled: true - pat.secret.secretName: my-pat-secret - database.type: sqlite - asserts: - - notContains: - path: spec.template.spec.containers[0].env - content: - name: DB_PASSWORD - any: true - - it: should inject DB_PASSWORD for postgresql set: pat.enabled: true @@ -165,6 +224,11 @@ tests: set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - contains: path: spec.template.spec.containers[0].volumeMounts @@ -173,19 +237,23 @@ tests: mountPath: /spec readOnly: true - - it: should mount data volume for sqlite + - it: should not mount data volume for postgresql set: pat.enabled: true pat.secret.secretName: my-pat-secret - database.type: sqlite + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - - contains: + - notContains: path: spec.template.spec.containers[0].volumeMounts content: name: data mountPath: /var/lib/netbird - - it: should not mount data volume for postgresql + - it: should run non-root for external databases set: pat.enabled: true pat.secret.secretName: my-pat-secret @@ -195,11 +263,12 @@ tests: database.name: netbird database.passwordSecret.secretName: pg-secret asserts: - - notContains: - path: spec.template.spec.containers[0].volumeMounts - content: - name: data - mountPath: /var/lib/netbird + - equal: + path: spec.template.spec.containers[0].securityContext.runAsNonRoot + value: true + - equal: + path: spec.template.spec.containers[0].securityContext.runAsUser + value: 65534 # ── Volumes ────────────────────────────────────────────────────────── @@ -207,6 +276,11 @@ tests: set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - contains: path: spec.template.spec.volumes @@ -221,6 +295,11 @@ tests: set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - equal: path: spec.backoffLimit @@ -233,8 +312,12 @@ tests: set: pat.enabled: true pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret asserts: - equal: path: spec.template.spec.restartPolicy value: OnFailure - diff --git a/charts/netbird/tests/server-deployment_test.yaml b/charts/netbird/tests/server-deployment_test.yaml index 0571e81..5cb9e74 100644 --- a/charts/netbird/tests/server-deployment_test.yaml +++ b/charts/netbird/tests/server-deployment_test.yaml @@ -144,9 +144,9 @@ tests: value: config-init - equal: path: spec.template.spec.initContainers[0].image - value: "ghcr.io/kitstream/initium:1.0.1" + value: "ghcr.io/kitstream/initium:1.0.4" - - it: should have only config-init for sqlite + - it: should have only config-init for sqlite without pat asserts: - lengthEqual: path: spec.template.spec.initContainers @@ -329,6 +329,210 @@ tests: path: spec.template.spec.initContainers[2].securityContext.runAsNonRoot value: true + # ── PAT seed native sidecar (SQLite only) ────────────────────────── + + - it: should have only one container (server) when pat is disabled + set: + pat.enabled: false + asserts: + - lengthEqual: + path: spec.template.spec.containers + count: 1 + + - it: should have only one container (server) even with pat enabled for sqlite + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - lengthEqual: + path: spec.template.spec.containers + count: 1 + + - it: should add pat-seed native sidecar as init container for sqlite + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - lengthEqual: + path: spec.template.spec.initContainers + count: 2 + - equal: + path: spec.template.spec.initContainers[0].name + value: config-init + - equal: + path: spec.template.spec.initContainers[1].name + value: pat-seed + + - it: should set restartPolicy Always on pat-seed native sidecar + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - equal: + path: spec.template.spec.initContainers[1].restartPolicy + value: Always + + - it: should use initium image for pat-seed sidecar + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - equal: + path: spec.template.spec.initContainers[1].image + value: "ghcr.io/kitstream/initium:1.0.4" + + - it: should run seed command with --sidecar flag in pat-seed sidecar + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - equal: + path: spec.template.spec.initContainers[1].args + value: + - --sidecar + - seed + - --spec + - /spec/pat-seed.yaml + + - it: should inject PAT_HASHED_TOKEN into sidecar + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + pat.secret.hashedTokenKey: myHash + database.type: sqlite + asserts: + - contains: + path: spec.template.spec.initContainers[1].env + content: + name: PAT_HASHED_TOKEN + valueFrom: + secretKeyRef: + name: my-pat-secret + key: myHash + + - it: should run sidecar as root for SQLite PVC access + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - equal: + path: spec.template.spec.initContainers[1].securityContext.runAsUser + value: 0 + - equal: + path: spec.template.spec.initContainers[1].securityContext.readOnlyRootFilesystem + value: true + - equal: + path: spec.template.spec.initContainers[1].securityContext.allowPrivilegeEscalation + value: false + + - it: should mount pat-seed-config and data volumes on sidecar + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - contains: + path: spec.template.spec.initContainers[1].volumeMounts + content: + name: pat-seed-config + mountPath: /spec + readOnly: true + - contains: + path: spec.template.spec.initContainers[1].volumeMounts + content: + name: data + mountPath: /var/lib/netbird + + - it: should not add pat-seed sidecar for postgresql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret + asserts: + - lengthEqual: + path: spec.template.spec.initContainers + count: 3 + - equal: + path: spec.template.spec.initContainers[0].name + value: db-wait + - equal: + path: spec.template.spec.initContainers[1].name + value: db-seed + - equal: + path: spec.template.spec.initContainers[2].name + value: config-init + + - it: should add pat-seed-config volume for sqlite pat sidecar + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - contains: + path: spec.template.spec.volumes + content: + name: pat-seed-config + configMap: + name: RELEASE-NAME-netbird-server-pat-seed + + - it: should not add pat-seed-config volume when pat is disabled + set: + pat.enabled: false + asserts: + - notContains: + path: spec.template.spec.volumes + content: + name: pat-seed-config + any: true + + - it: should not add pat-seed-config volume for postgresql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret + asserts: + - notContains: + path: spec.template.spec.volumes + content: + name: pat-seed-config + any: true + + - it: should include pat-seed checksum annotation for sqlite + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: sqlite + asserts: + - exists: + path: spec.template.metadata.annotations.checksum/pat-seed + + - it: should not include pat-seed checksum for postgresql + set: + pat.enabled: true + pat.secret.secretName: my-pat-secret + database.type: postgresql + database.host: pg.example.com + database.user: netbird + database.name: netbird + database.passwordSecret.secretName: pg-secret + asserts: + - notExists: + path: spec.template.metadata.annotations.checksum/pat-seed + # ── Other deployment settings ──────────────────────────────────────── - it: should set resource limits when specified @@ -398,5 +602,3 @@ tests: path: spec.template.spec.containers[0].securityContext value: readOnlyRootFilesystem: true - - diff --git a/charts/netbird/values.yaml b/charts/netbird/values.yaml index 59b90b7..016e08d 100644 --- a/charts/netbird/values.yaml +++ b/charts/netbird/values.yaml @@ -150,7 +150,7 @@ server: # -- Init container image repository. repository: ghcr.io/kitstream/initium # -- Init container image tag. - tag: "1.0.1" + tag: "1.0.4" # -- Component-level image pull secrets. imagePullSecrets: [] diff --git a/ci/scripts/e2e.sh b/ci/scripts/e2e.sh index ad30d5d..4ae045f 100755 --- a/ci/scripts/e2e.sh +++ b/ci/scripts/e2e.sh @@ -219,9 +219,15 @@ if ! helm install "$RELEASE" "$CHART" \ --set pat.secret.secretName=netbird-pat \ "${EXTRA_SETS[@]}" \ --timeout "$TIMEOUT"; then - log "Helm install failed — dumping PAT seed job logs..." - kubectl -n "$NAMESPACE" logs job/"$RELEASE"-server-pat-seed --all-containers 2>/dev/null || true - kubectl -n "$NAMESPACE" describe job/"$RELEASE"-server-pat-seed 2>/dev/null || true + log "Helm install failed — dumping logs..." + if [ "$BACKEND" = "sqlite" ]; then + log "PAT seed sidecar logs:" + kubectl -n "$NAMESPACE" logs deployment/"$RELEASE"-server -c pat-seed 2>/dev/null || true + else + log "PAT seed job logs:" + kubectl -n "$NAMESPACE" logs job/"$RELEASE"-server-pat-seed --all-containers 2>/dev/null || true + kubectl -n "$NAMESPACE" describe job/"$RELEASE"-server-pat-seed 2>/dev/null || true + fi fail "Helm install failed" fi @@ -237,14 +243,37 @@ kubectl -n "$NAMESPACE" get pods -o wide log "Running helm test..." helm test "$RELEASE" -n "$NAMESPACE" --timeout 2m -# ── Wait for PAT seed job to complete ───────────────────────────────── -log "Waiting for PAT seed job to complete..." -kubectl -n "$NAMESPACE" wait --for=condition=complete \ - job/"$RELEASE"-server-pat-seed --timeout=180s || { - log "PAT seed job logs:" - kubectl -n "$NAMESPACE" logs job/"$RELEASE"-server-pat-seed --all-containers || true - fail "PAT seed job did not complete" -} +# ── Wait for PAT seed to complete ───────────────────────────────────── +if [ "$BACKEND" = "sqlite" ]; then + # SQLite: PAT seed runs as a native sidecar (init container with + # restartPolicy: Always) in the server pod. The sidecar stays alive + # after seeding (--sidecar flag), so we check its logs for the + # "seed execution completed" message rather than waiting for + # container termination. + log "Waiting for PAT seed native sidecar to complete seeding..." + for i in $(seq 1 60); do + LOGS=$(kubectl -n "$NAMESPACE" logs deployment/"$RELEASE"-server -c pat-seed 2>/dev/null || echo "") + if echo "$LOGS" | grep -q "seed execution completed"; then + log "PAT seed sidecar completed seeding successfully" + break + fi + if [ "$i" -eq 60 ]; then + log "PAT seed sidecar logs:" + echo "$LOGS" + fail "PAT seed sidecar did not complete seeding within timeout" + fi + sleep 3 + done +else + # External DB: PAT seed runs as a separate hook Job. + log "Waiting for PAT seed job to complete..." + kubectl -n "$NAMESPACE" wait --for=condition=complete \ + job/"$RELEASE"-server-pat-seed --timeout=180s || { + log "PAT seed job logs:" + kubectl -n "$NAMESPACE" logs job/"$RELEASE"-server-pat-seed --all-containers || true + fail "PAT seed job did not complete" + } +fi # ── Verify PAT authentication ───────────────────────────────────────── log "Verifying PAT authentication against API..." # Re-derive the PAT_TOKEN (same deterministic generation as above)