feat(planner): re-apply config.toml as part of buildNodeUpdatePlan

[bdchatham]

Problem

buildNodeUpdatePlan (the image-drift plan) progression was:

ApplyStatefulSet → ApplyService → ReplacePod → ObserveImage → MarkReady

No TaskConfigApply. Result: any change to fields the controller owns via commonOverrides — most notably Spec.ExternalAddress from the publishable-P2P feature — never reached the running pod's config.toml. Inbound NLB→pod worked; outbound advertising (NodeInfo / PEX) didn't, because seid had external-address = ''.

Caught during the arctic-1 canary rollout (platform#764–771). With AWS LBC + harbor SEI_NLB_TARGET_TYPE=instance now in place, the NLB path is fully working, but seid still doesn't tell other peers how to reach it.

Fix

Add TaskConfigApply + TaskConfigValidate to the update progression, between ApplyService and ReplacePod:

ApplyStatefulSet → ApplyService → TaskConfigApply → TaskConfigValidate → ReplacePod → ObserveImage → MarkReady

Conceptual reframe: "deployment" stops being "swap image" and becomes "rebuild the running state from current spec." commonOverrides already produces the correct config from current Spec.ExternalAddress; we just weren't re-running TaskConfigApply outside init.

Two correctness fixes from pre-implementation expert review

Both Platform and K8s specialists independently flagged these before I touched a keyboard:

1. Real ConfigIntent (not nil). Old buildNodeUpdatePlan called paramsForTaskType(node, taskType, nil, nil). With configIntent=nil, the params builder returned an empty ConfigIntent{} — empty Mode — and sei-config's ResolveIntent rejects empty Mode. TaskConfigApply would have failed at runtime on the first image bump.

Fix: extracted NodePlanner.BuildConfigIntent(node) from each mode planner's BuildPlan. buildNodeUpdatePlan calls it via plannerForMode(node).BuildConfigIntent(node). Same source of truth for mode + overrides on init and day-2.

2. Incremental: true on the day-2 intent. The init path uses applyFull — regenerates config.toml from mode defaults + overrides. Running that on a Running node would wipe:

• persistent-peers (set by the sidecar's writePeersToConfig)
• State-sync trust point
• Genesis-peers
• Operator-managed TOML keys outside the controller's overrides set

applyIncremental reads on-disk config and patches only the overrides we own. Day-2 must use it.

Tests

• TestBuildRunningPlan_ImageDrift_ReturnsNodeUpdatePlan — updated to expect the new 7-task progression.
• TestBuildRunningPlan_ImageDriftWinsOverSidecar — same.
• TestBuildNodeUpdatePlan_ConfigApplyHasIntentWithMode — asserts Mode is set + Overrides[KeyP2PExternalAddress] propagates from Spec.ExternalAddress.
• TestBuildNodeUpdatePlan_ConfigApplyIsIncremental — pins Incremental: true.
• TestBuildNodeUpdatePlan_ConfigApplyIntentMatchesModePlanner — table-driven coverage across full/archive/validator modes.

Existing unit + envtest suites: green.

Progression ordering — why between ApplyService and ReplacePod

• ConfigApply must run before ReplacePod: the OLD pod's sidecar writes config.toml to the PVC; the new pod mounts the PVC and reads the fresh file on seid start.
• ConfigApply after ReplacePod would race against new-pod sidecar liveness and waste a restart cycle.
• external-address has no hot-reload path in seid today (seictl/sidecar/tasks/config_reload.go literally has a TODO: signal seid to re-read config), so the pod restart is the trigger.
• Matches the init baseProgression order.

classifyPlan unchanged

classifyPlan (planner.go:217) identifies "node-update" plans by TaskTypeObserveImage presence — still in the progression, classification still works.

Out of scope

Autonomous Spec.ExternalAddress drift detection on Running nodes. With this change, ExternalAddress propagates on the next trigger of buildNodeUpdatePlan (image bump or other image drift). Operators currently trigger via an image SHA pin. If autonomous propagation becomes operationally necessary (NLB hostname rotation, Spec.Overrides drift, publishable-P2P opt-out), file a follow-up to add drift detection in buildRunningPlan that routes to the same buildNodeUpdatePlan.

Test plan

• Unit + envtest green on CI.
• After merge + ECR build + harbor controller image bump: pinning arctic-1 syncer image to the same digest (no-op semantically) triggers buildNodeUpdatePlan with the new progression. config.toml on the new pod has external-address = "syncer-0-0-p2p.arctic-1.harbor.platform.sei.io:26656". seid logs show PEX advertising the NLB hostname.

🤖 Generated with Claude Code

[cursor]

PR Summary

Medium Risk
Changes the running-node rollout path and PVC config writes; mitigated by incremental apply and strong unit tests, but mis-ordering or full apply on day-2 could still disrupt P2P/state on live nodes.

Overview
Running-node image-drift plans now re-apply config.toml before pod replacement, so controller-owned settings (e.g. Spec.ExternalAddress / P2P external address) reach the live node instead of only being applied at init.

buildNodeUpdatePlan inserts TaskConfigApply and TaskConfigValidate after apply Service and before ReplacePod (7 tasks total). Config is written to the PVC on the old pod’s sidecar, then the new pod reads it on start.

NodePlanner.BuildConfigIntent is added and implemented on full, archive, validator, and replayer planners so init and day-2 share the same mode + merged overrides. Day-2 sets Incremental: true so sei-config patches only owned keys and does not wipe on-disk peers, state-sync trust, or other operator TOML. Init remains non-incremental.

Tests lock task order, non-empty Mode, ExternalAddress in overrides, incremental vs init behavior, and per-mode intent consistency.

^{Reviewed by Cursor Bugbot for commit 43a0018. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 1d4dc7d. Configure here.}

[cursor]

Node update plan missing key validation readiness gates

High Severity

buildNodeUpdatePlan includes TaskTypeReplacePod (cycles the pod) but omits the TaskTypeValidateSigningKey, TaskTypeValidateNodeKey, and TaskTypeValidateOperatorKeyring readiness gates that both buildBasePlan and buildBootstrapPlan conditionally include before pod creation. For validator nodes with key secrets mounted as volumes, an image-drift rollout would recreate the pod without first verifying those secrets exist, risking a kubelet volume-mount failure instead of a clear controller-side validation error.

Additional Locations (1)

• internal/planner/planner.go#L531-L542

 

^{Triggered by learned rule: Planner readiness gates must apply to all pod-cycling plans}

^{Reviewed by Cursor Bugbot for commit 1d4dc7d. Configure here.}