Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 13 additions & 7 deletions .github/release-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,13 +96,19 @@ repositories or docs.]
## Breaking changes / Migration notes

- [Delete this section if there are no breaking changes.]
- **Upgrading from a pre-PR #523 cluster**: PR #523 relocated six `bedag/raw`
helmfile releases into the `base` chart. Existing clusters must run
`bash hack/migrate-bedag-raw-to-base.sh` once before `obol stack up` to
transfer Helm ownership annotations; otherwise `helm upgrade base` fails
with `invalid ownership metadata`. See
[`docs/upgrade-from-pre-pr-523.md`](../docs/upgrade-from-pre-pr-523.md).
Fresh installs are unaffected.
- **Pre-release tester warning**: If you ran an unreleased marketplace or
chart-consolidation branch before this release, `obol stack up` may fail
with Helm `invalid ownership metadata` errors for resources or namespaces
that moved into the `base` chart. This is not a supported production
migration path. Back up anything you need from the local test stack, then
recreate it:

```bash
obol stack down
obol stack purge --force
obol stack init
obol stack up
```

## Known issues

Expand Down
32 changes: 31 additions & 1 deletion .github/workflows/helm-template-smoke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,37 @@ jobs:
helm template base "$workdir/base" \
--set dataDir=/data \
--set network=mainnet \
> /dev/null
> "$workdir/base-rendered.yaml"

# Kubernetes object identity must be unique within one rendered
# chart. Helm will happily render duplicate apiVersion/kind/name
# tuples and leave the actual outcome to manifest ordering; this
# caught the duplicated obol-frontend ClusterRole/Binding review bug.
awk '
function flush() {
if (api && kind && name) {
key = api "/" kind "/" ns "/" name
count[key]++
}
api = kind = name = ns = ""; inmeta = 0
}
/^---/ { flush(); next }
/^apiVersion:/ { api = $2; next }
/^kind:/ { kind = $2; next }
/^metadata:/ { inmeta = 1; next }
inmeta && /^ name:/ { name = $2; next }
inmeta && /^ namespace:/ { ns = $2; next }
/^[^ ]/ && $0 !~ /^(apiVersion|kind|metadata):/ { inmeta = 0 }
END {
flush()
for (k in count) {
if (count[k] > 1) {
print count[k] " " k
dup = 1
}
}
exit dup
}' "$workdir/base-rendered.yaml"

- name: helm template ./cloudflared
run: |
Expand Down
8 changes: 8 additions & 0 deletions cmd/serviceoffer-controller/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ func runWithLeaderElection(ctx context.Context, cfg *rest.Config, controller *se
log.Printf("serviceoffer-controller: became leader %s", podName)
if err := controller.Run(ctx, workers); err != nil {
log.Printf("controller run: %v", err)
os.Exit(controllerRunExitCode(err))
}
},
OnStoppedLeading: func() {
Expand All @@ -110,6 +111,13 @@ func runWithLeaderElection(ctx context.Context, cfg *rest.Config, controller *se
})
}

func controllerRunExitCode(err error) int {
if err != nil {
return 1
}
return 0
}

func loadConfig(kubeconfig string) (*rest.Config, error) {
if kubeconfig != "" {
return clientcmd.BuildConfigFromFlags("", kubeconfig)
Expand Down
10 changes: 10 additions & 0 deletions cmd/serviceoffer-controller/main_test.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package main

import (
"errors"
"os"
"path/filepath"
"testing"
Expand Down Expand Up @@ -61,6 +62,15 @@ func TestLeaderElectionDefaults(t *testing.T) {
}
}

func TestControllerRunExitCode(t *testing.T) {
if got := controllerRunExitCode(nil); got != 0 {
t.Fatalf("controllerRunExitCode(nil) = %d, want 0", got)
}
if got := controllerRunExitCode(errors.New("informer died")); got != 1 {
t.Fatalf("controllerRunExitCode(error) = %d, want 1", got)
}
}

const minimalKubeconfig = `apiVersion: v1
kind: Config
clusters:
Expand Down
22 changes: 11 additions & 11 deletions docs/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ PR #530 swapped it to `increase()` over an explicit window.
+------------------------------+
| x402-verifier (stateless) |
| - in-memory counters |
| - labels: route, |
| - labels: |
| offer_namespace, |
| offer_name, chain, |
| asset_symbol |
Expand All @@ -77,8 +77,8 @@ PR #530 swapped it to `increase()` over an explicit window.
v
+------------------------------+
| Pre-aggregated series |
| offer:x402_revenue:7d_by_offer
| offer:x402_paid_requests:7d_by_offer
| x402:revenue:7d_by_offer
| x402:revenue:7d_by_offer_chain
+---------------+--------------+
|
| PromQL queries
Expand Down Expand Up @@ -191,20 +191,21 @@ Naming follows the standard Prometheus pattern:

Examples we ship:

- `offer:x402_revenue:7d_by_offer` — revenue aggregated to the `offer` level,
base metric is `x402_revenue`, operation is `increase` over `7d` grouped
`by_offer`.
- `offer:x402_paid_requests:7d_by_offer` — same shape for paid request count.
- `x402:revenue:7d_by_offer` — paid request count aggregated to the offer
level over the last 7d. The frontend multiplies this by the ServiceOffer
price table to display revenue.
- `x402:revenue:7d_by_offer_chain_asset_symbol` — same window, retaining
chain and settlement-token facets for per-token and per-chain views.

Rules:

1. **Name the window in the rule.** `7d_by_offer` is honest; `lifetime_by_offer`
is a lie (Prometheus has no "lifetime"). The window in the name must match
the window in the expression.
2. **Use `increase()` over an explicit range, not `sum()` of the raw counter.**
See PR #530 — the original rule did `sum(by offer) (x402_revenue_total)` and
silently zeroed every time the verifier pod restarted. The fixed rule is
`sum by (offer_namespace, offer_name) (increase(x402_revenue_total[7d]))`.
See PR #530 — the original rule did `sum(by offer) (charged_requests_total)`
and silently zeroed every time the verifier pod restarted. The fixed rule is
`sum by (offer_namespace, offer_name) (increase(obol_x402_verifier_charged_requests_total[7d]))`.
3. **Keep the window aligned with retention.** Recording a `30d` rule with 8d
retention is a footgun: the rule sees nulls and silently produces nothing.

Expand All @@ -226,7 +227,6 @@ Concrete examples:

| Label | Source | Why include it |
|-------------------|------------------|-------------------------------------------|
| `route` | offer CR pattern | Direct query facet, bounded by # offers |
| `offer_namespace` | offer CR meta | Tenancy facet |
| `offer_name` | offer CR meta | Per-offer breakdown |
| `chain` | offer CR payment | "Revenue by chain" is a real question |
Expand Down
82 changes: 0 additions & 82 deletions docs/upgrade-from-pre-pr-523.md

This file was deleted.

82 changes: 0 additions & 82 deletions hack/migrate-bedag-raw-to-base.sh

This file was deleted.

This file was deleted.

Loading