+
+## Frequently asked questions
+
+
+Can I package any software or are there any prerequisites to be a Controller?
+
+We define a *Controller* as a software that has at least one Custom Resource Definition (CRD) and a Kubernetes controller for that CRD. This is the minimum requirement to be a *Controller*. We have some checks to enforce this at packaging time.
+
+
+
+
+How can I package my software as a Controller?
+
+Currently, we support Helm charts as the underlying package format for *Controllers*. As long as you have a Helm chart, you can package it as a *Controller*.
+
+If you don't have a Helm chart, you can't deploy the software. We only support Helm charts as the underlying package format for *Controllers*. We may extend this to support other packaging formats like Kustomize in the future.
+
+
+
+
+Can I package Crossplane XRDs/Compositions as a Helm chart to deploy as a Controller?
+
+This is not recommended. For packaging Crossplane XRDs/ and Compositions, we recommend using the `Configuration` package format. A helm chart only with Crossplane XRDs/Compositions does not qualify as a *Controller*.
+
+
+
+
+How can I override the Helm values when deploying a Controller?
+
+Overriding the Helm values is possible at two levels:
+- During packaging time, in the package manifest file.
+- At runtime, using a `ControllerRuntimeConfig` resource (similar to Crossplane `DeploymentRuntimeConfig`).
+
+
+
+
+How can I configure the helm release name and namespace for the controller?
+
+Right now, it is not possible to configure this at runtime. The package author configures release name and namespace during packaging, so it is hardcoded inside the package. Unlike a regular application that is deployed by a Helm chart, *Controllers* can only be deployed once in a given control plane, so, we hope it should be ok to rely on predefined release names and namespaces. We may consider exposing these in `ControllerRuntimeConfig` later, but, we would like to keep it opinionated unless there are strong reasons to do so.
+
+
+
+
+Can I deploy more than one instance of a Controller package?
+
+No, this is not possible. Remember, a *Controller* package introduces CRDs which are cluster-scoped objects. Just like one cannot deploy more than one instance of the same Crossplane Provider package today, it is not possible to deploy more than one instance of a *Controller*.
+
+
+
+
+Do I need a specific Crossplane version to run Controllers?
+
+Yes, you need to use Crossplane v1.19.0 or later to use *Controllers*. This is because of the changes in the Crossplane codebase to support third-party package formats in dependencies.
+
+Spaces `v1.12.0` supports Crossplane `v1.19` in the *Rapid* release channel.
+
+
+
+
+Can I deploy Controllers outside of an Upbound control plane? With UXP?
+
+No, *Controllers* are a proprietary package format and are only available for control planes running in Spaces hosting environments in Upbound.
+
+
+
+
+[cli]: /manuals/uxp/overview
+
diff --git a/spaces_versioned_docs/version-1.14/howtos/ctp-audit-logs.md b/spaces_versioned_docs/version-1.14/howtos/ctp-audit-logs.md
new file mode 100644
index 000000000..e387b2873
--- /dev/null
+++ b/spaces_versioned_docs/version-1.14/howtos/ctp-audit-logs.md
@@ -0,0 +1,544 @@
+---
+title: Control plane audit logging
+---
+
+This guide explains how to enable and configure audit logging for control planes
+in Self-Hosted Upbound Spaces.
+
+Starting in Spaces `v1.14.0`, each control plane contains an API server that
+supports audit log collection. You can use audit logging to track creation,
+updates, and deletions of Crossplane resources. Control plane audit logs
+use observability features to collect audit logs with `SharedTelemetryConfig` and
+send logs to an OpenTelemetry (`OTEL`) collector.
+
+
+## Prerequisites
+
+Before you begin, make sure you have:
+
+* Spaces `v1.14.0` or greater
+* Admin access to your Spaces host cluster
+* `kubectl` configured to access the host cluster
+* `helm` installed
+* `yq` installed
+* `up` CLI installed and logged in to your organization
+
+## Enable observability
+
+
+Observability graduated to General Available in `v1.14.0` but is disabled by
+default.
+
+
+
+
+
+### Before `v1.14`
+To enable the GA Observability feature, upgrade your Spaces installation to `v1.14.0`
+or later and update your installation setting to the new flag:
+
+```diff
+helm upgrade spaces upbound/spaces -n upbound-system \
+- --set "features.alpha.observability.enabled=true"
++ --set "observability.enabled=true"
+```
+
+
+
+### After `v1.14`
+
+To enable the GA Observability feature for `v1.14.0` and later, pass the feature
+flag:
+
+```sh
+helm upgrade spaces upbound/spaces -n upbound-system \
+ --set "observability.enabled=true"
+
+```
+
+
+
+
+To confirm Observability is enabled, run the `helm get values` command:
+
+
+```shell
+helm get values --namespace upbound-system spaces | yq .observability
+```
+
+Your output should return:
+
+```shell-noCopy
+ enabled: true
+```
+
+## Install an observability backend
+
+:::note
+If you already have an observability backend in your environment, skip to the
+next section.
+:::
+
+
+For this guide, you'll use Grafana's `docker-otel-lgtm` bundle to validate audit log
+generation. production environments, configure a dedicated observability
+backend like Datadog, Splunk, or an enterprise-grade Grafana stack.
+
+
+
+First, make sure your `kubectl` context points to your Spaces host cluster:
+
+```shell
+kubectl config current-context
+```
+
+The output should return your cluster name.
+
+Next, install `docker-otel-lgtm` as a deployment using port-forwarding to
+connect to Grafana. Create a manifest file and paste the
+following configuration:
+
+```yaml title="otel-lgtm.yaml"
+apiVersion: v1
+kind: Namespace
+metadata:
+ name: observability
+---
+apiVersion: v1
+kind: Service
+metadata:
+ labels:
+ app: otel-lgtm
+ name: otel-lgtm
+ namespace: observability
+spec:
+ ports:
+ - name: grpc
+ port: 4317
+ protocol: TCP
+ targetPort: 4317
+ - name: http
+ port: 4318
+ protocol: TCP
+ targetPort: 4318
+ - name: grafana
+ port: 3000
+ protocol: TCP
+ targetPort: 3000
+ selector:
+ app: otel-lgtm
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: otel-lgtm
+ labels:
+ app: otel-lgtm
+ namespace: observability
+spec:
+ replicas: 1
+ selector:
+ matchLabels:
+ app: otel-lgtm
+ template:
+ metadata:
+ labels:
+ app: otel-lgtm
+ spec:
+ containers:
+ - name: otel-lgtm
+ image: grafana/otel-lgtm
+ ports:
+ - containerPort: 4317
+ - containerPort: 4318
+ - containerPort: 3000
+```
+
+Next, apply the manifest:
+
+```shell
+kubectl apply --filename otel-lgtm.yaml
+```
+
+Your output should return the resources:
+
+```shell
+namespace/observability created
+ service/otel-lgtm created
+ deployment.apps/otel-lgtm created
+```
+
+To verify your resources deployed, use `kubectl get` to display resources with
+an `ACTIVE` or `READY` status.
+
+Next, forward the Grafana port:
+
+```shell
+kubectl port-forward svc/otel-lgtm --namespace observability 3000:3000
+```
+
+Now you can access the Grafana UI at http://localhost:3000.
+
+
+## Create an audit-enabled control plane
+
+To enable audit logging for a control plane, you need to label it so the
+`SharedTelemetryConfig` can identify and apply audit settings. This section
+creates a new control plane with the `audit-enabled: "true"` label. The
+`audit-enabled: "true"` label marks this control plane for audit logging. The
+`SharedTelemetryConfig` (created in the next section) finds control planes with
+this label and enables audit logging on them.
+
+Create a new manifest file and paste the configuration below:
+
+
+```yaml title="ctp-audit.yaml"
+apiVersion: v1
+kind: Namespace
+metadata:
+ name: audit-test
+---
+apiVersion: spaces.upbound.io/v1beta1
+kind: ControlPlane
+metadata:
+ labels:
+ audit-enabled: "true"
+ name: ctp1
+ namespace: audit-test
+spec:
+ writeConnectionSecretToRef:
+ name: kubeconfig-ctp1
+ namespace: audit-test
+```
+
+
+The `metadata.labels` section contains the `audit-enabled` setting.
+
+Apply the manifest:
+
+```shell
+kubectl apply --filename ctp-audit.yaml
+```
+
+Confirm your control plane reaches the `READY` status:
+
+```shell
+kubectl get --filename ctp-audit.yaml
+```
+
+## Create a `SharedTelemetryConfig`
+
+The `SharedTelemetryConfig` applies to all control plane objects in a namespace
+and enables audit logging and routes logs to your `OTEL` endpoint.
+
+Create a `SharedTelemetryConfig` manifest file and paste the configuration
+below:
+
+
+```yaml title="sharedtelemetryconfig.yaml"
+apiVersion: observability.spaces.upbound.io/v1alpha1
+kind: SharedTelemetryConfig
+metadata:
+ name: apiserver-audit
+ namespace: audit-test
+spec:
+ apiServer:
+ audit:
+ enabled: true
+ exporters:
+ otlphttp:
+ endpoint: http://otel-lgtm.observability:4318
+ exportPipeline:
+ logs: [otlphttp]
+ controlPlaneSelector:
+ labelSelectors:
+ - matchLabels:
+ audit-enabled: "true"
+```
+
+
+This configuration:
+
+* Sets `apiServer.audit.enabled` to `true`
+* Configures the `otlphttp` exporter to point to the `docker-otel-lgtm` service
+* Uses `controlPlaneSelector` to match any control plane in the namespace with the `audit-enabled` label set to `true`
+
+:::note
+You can configure the `SharedTelemetryConfig` to select control planes in
+several ways. more information on control plane selection, see the [control
+plane selection][ctp-selection] documentation.
+:::
+
+Apply the `SharedTelemetryConfig`:
+
+```shell
+kubectl apply --filename sharedtelemetryconfig.yaml
+```
+
+Confirm the configuration selected the control plane:
+
+```shell
+kubectl get --filename sharedtelemetryconfig.yaml
+```
+
+The output should return `SELECTED` as `1` and `VALIDATED` as `TRUE`.
+
+For more detailed status information, use `kubectl get`:
+
+```shell
+kubectl get --filename sharedtelemetryconfig.yaml --output yaml | yq .status
+```
+
+## Generate and monitor audit events
+
+You enabled telemetry on your new control plane and can now generate events to
+test the audit logging. This guide uses the `nop-provider` to simulate resource
+operations.
+
+Switch your `up` context to the new control plane:
+
+```shell
+up ctx ///
+```
+
+Create a new Provider manifest:
+
+```yaml title="provider-nop.yaml"
+apiVersion: pkg.crossplane.io/v1
+ kind: Provider
+ metadata:
+ name: crossplane-contrib-provider-nop
+ spec:
+ package: xpkg.upbound.io/crossplane-contrib/provider-nop:v0.4.0
+```
+
+Apply the provider manifest:
+
+```shell
+kubectl apply --filename provider-nop.yaml
+```
+
+Verify the provider installed and returns `HEALTHY` status as `TRUE`.
+
+Apply an example resource to kick off event generation:
+
+
+```shell
+kubectl apply --filename https://raw.githubusercontent.com/crossplane-contrib/provider-nop/refs/heads/main/examples/nopresource.yaml
+```
+
+In your Grafana dashboard, navigate to **Drilldown** > **Logs** under the
+Grafana menu.
+
+
+Filter for `controlplane-audit` log messages.
+
+Create a query to find `create` events on `nopresources` by filtering:
+
+* The `verb` field for `create` events
+* The `objectRef_resource` field to match the Kind `nopresources`
+
+Review the audit log results. The log stream displays:
+
+*The client applying the create operation
+* The resource kind
+* Client details
+* The response code
+
+Expand the example below for an audit log entry:
+
+
+ Audit log entry
+
+```json
+{
+ "level": "Metadata",
+ "auditID": "51bbe609-14ad-4874-be78-1289c10d506a",
+ "stage": "ResponseComplete",
+ "requestURI": "/apis/nop.crossplane.io/v1alpha1/nopresources?fieldManager=kubectl-client-side-apply&fieldValidation=Strict",
+ "verb": "create",
+ "user": {
+ "username": "kubernetes-admin",
+ "groups": ["system:masters", "system:authenticated"]
+ },
+ "impersonatedUser": {
+ "username": "upbound:spaces:host:masterclient",
+ "groups": [
+ "system:authenticated",
+ "upbound:controlplane:admin",
+ "upbound:spaces:host:system:masters"
+ ]
+ },
+ "sourceIPs": ["10.244.0.135", "127.0.0.1"],
+ "userAgent": "kubectl/v1.32.2 (darwin/arm64) kubernetes/67a30c0",
+ "objectRef": {
+ "resource": "nopresources",
+ "name": "example",
+ "apiGroup": "nop.crossplane.io",
+ "apiVersion": "v1alpha1"
+ },
+ "responseStatus": { "metadata": {}, "code": 201 },
+ "requestReceivedTimestamp": "2025-09-19T23:03:24.540067Z",
+ "stageTimestamp": "2025-09-19T23:03:24.557583Z",
+ "annotations": {
+ "authorization.k8s.io/decision": "allow",
+ "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"controlplane-admin\" of ClusterRole \"controlplane-admin\" to Group \"upbound:controlplane:admin\""
+ }
+ }
+```
+
+
+## Customize the audit policy
+
+Spaces `v1.14.0` includes a default audit policy. You can customize this policy
+by creating a configuration file and passing the values to
+`observability.collectors.apiServer.auditPolicy` in the helm values file.
+
+An example custom audit policy:
+
+```yaml
+observability:
+ controlPlanes:
+ apiServer:
+ auditPolicy: |
+ apiVersion: audit.k8s.io/v1
+ kind: Policy
+ rules:
+ # ============================================================================
+ # RULE 1: Exclude health check and version endpoints
+ # ============================================================================
+ - level: None
+ nonResourceURLs:
+ - '/healthz*'
+ - '/readyz*'
+ - /version
+ # ============================================================================
+ # RULE 2: ConfigMaps - Write operations only
+ # ============================================================================
+ - level: Metadata
+ resources:
+ - group: ""
+ resources:
+ - configmaps
+ verbs:
+ - create
+ - update
+ - patch
+ - delete
+ omitStages:
+ - RequestReceived
+ - ResponseStarted
+ # ============================================================================
+ # RULE 3: Secrets - ALL operations
+ # ============================================================================
+ - level: Metadata
+ resources:
+ - group: ""
+ resources:
+ - secrets
+ verbs:
+ - get
+ - list
+ - watch
+ - create
+ - update
+ - patch
+ - delete
+ omitStages:
+ - RequestReceived
+ - ResponseStarted
+ # ============================================================================
+ # RULE 4: Global exclusion of read-only operations
+ # ============================================================================
+ - level: None
+ verbs:
+ - get
+ - list
+ - watch
+ # ==========================================================================
+ # RULE 5: Exclude standard Kubernetes resources from write operation logging
+ # ==========================================================================
+ - level: None
+ resources:
+ - group: ""
+ - group: "apps"
+ - group: "networking.k8s.io"
+ - group: "policy"
+ - group: "rbac.authorization.k8s.io"
+ - group: "storage.k8s.io"
+ - group: "batch"
+ - group: "autoscaling"
+ - group: "metrics.k8s.io"
+ - group: "node.k8s.io"
+ - group: "scheduling.k8s.io"
+ - group: "coordination.k8s.io"
+ - group: "discovery.k8s.io"
+ - group: "events.k8s.io"
+ - group: "flowcontrol.apiserver.k8s.io"
+ - group: "internal.apiserver.k8s.io"
+ - group: "authentication.k8s.io"
+ - group: "authorization.k8s.io"
+ - group: "admissionregistration.k8s.io"
+ verbs:
+ - create
+ - update
+ - patch
+ - delete
+ # ============================================================================
+ # RULE 6: Catch-all for ALL custom resources and any missed resources
+ # ============================================================================
+ - level: Metadata
+ verbs:
+ - create
+ - update
+ - patch
+ - delete
+ omitStages:
+ - RequestReceived
+ - ResponseStarted
+ # ============================================================================
+ # RULE 7: Final catch-all - exclude everything else
+ # ============================================================================
+ - level: None
+ omitStages:
+ - RequestReceived
+ - ResponseStarted
+```
+You can apply this policy during Spaces installation or upgrade using the helm values file.
+
+Audit policies use rules evaluated in order from top to bottom where the first
+matching rule applies. Control plane audit policies follow Kubernetes conventions and use the
+following logging levels:
+
+* **None** - Don't log events matching this rule
+* **Metadata** - Log request metadata (user, timestamp, resource, verb) but not request or response bodies
+* **Request** - Log metadata and request body but not response body
+* **RequestResponse** - Log metadata, request body, and response body
+
+For more information, review the Kubernetes [Auditing] documentation.
+
+## Disable audit logging
+
+You can disable audit logging on a control plane by removing it from the
+`SharedTelemetryConfig` selector or by deleting the `SharedTelemetryConfig`.
+
+### Disable for specific control planes
+
+Remove the `audit-enabled` label from control planes that should stop sending audit logs:
+
+```bash
+kubectl label controlplane --namespace audit-enabled-
+```
+
+The `SharedTelemetryConfig` no longer selects this control plane, and audit log collection stops.
+
+### Disable for all control planes
+
+Delete the `SharedTelemetryConfig` to stop audit logging for all control planes it manages:
+
+```bash
+kubectl delete sharedtelemetryconfig --namespace