feat(supervisor): compute workload manager (#3114)

Adds the `ComputeWorkloadManager` for routing task execution through the
compute gateway, including full checkpoint/restore support, OTel trace
integration, and template pre-warming.

## Changes

**Compute workload manager**
(`apps/supervisor/src/workloadManager/compute.ts`)
- Routes instance create, snapshot, delete, and restore through the
compute gateway API
- Wide event logging on create with full timing and context
- Configurable gateway timeout, auth token, image digest stripping

**Compute snapshot service**
(`apps/supervisor/src/services/computeSnapshotService.ts`)
- Timer wheel for delayed snapshot dispatch (avoids wasted work on
short-lived waitpoints)
- Configurable dispatch concurrency limit
(`COMPUTE_SNAPSHOT_DISPATCH_LIMIT`)
- Snapshot-complete callback handler with suspend completion reporting
- Trace context management and OTel span emission for snapshot
operations

**OTel trace service**
(`apps/supervisor/src/services/otlpTraceService.ts`)
- Fire-and-forget OTLP span emission for compute operations (provision,
restore, snapshot)
- BigInt nanosecond conversion preserving sub-ms precision for span
ordering

**Template creation**
(`apps/webapp/app/v3/services/computeTemplateCreation.server.ts`)
- Three-mode rollout: required (MICROVM projects), shadow (feature flag
/ percentage), skip
- Integrated into deploy finalize flow

**Shared compute package** (`internal-packages/compute/`)
- Gateway client with namespace-based API (instances, templates,
snapshots)
- Zod schemas for all gateway request/response types

**Database**
- `COMPUTE` variant added to `TaskRunCheckpointType` enum
- `WorkloadType` enum and column on `WorkerInstanceGroup`
- `hasComputeAccess` feature flag

**Env / config**
- Compute gateway URL, auth token, timeout
- Snapshot enable flag, delay, dispatch limit
- Dedicated OTLP endpoint for compute spans
(`COMPUTE_TRACE_OTLP_ENDPOINT`)

Author: nicktrn

.changeset/fix-local-build-load.md

@@ -0,0 +1,5 @@
+---
+"trigger.dev": patch
+---
+
+Fix `--load` flag being silently ignored on local/self-hosted builds.

.claude/skills/span-timeline-events/SKILL.md

@@ -0,0 +1,78 @@
+---
+name: span-timeline-events
+description: Use when adding, modifying, or debugging OTel span timeline events in the trace view. Covers event structure, ClickHouse storage constraints, rendering in SpanTimeline component, admin visibility, and the step-by-step process for adding new events.
+allowed-tools: Read, Write, Edit, Glob, Grep, Bash
+---
+
+# Span Timeline Events
+
+The trace view's right panel shows a timeline of events for the selected span. These are OTel span events rendered by `app/utils/timelineSpanEvents.ts` and the `SpanTimeline` component.
+
+## How They Work
+
+1. **Span events** in OTel are attached to a parent span. In ClickHouse, they're stored as separate rows with `kind: "SPAN_EVENT"` sharing the parent span's `span_id`. The `#mergeRecordsIntoSpanDetail` method reassembles them into the span's `events` array at query time.
+2. The timeline only renders events whose `name` starts with `trigger.dev/` - all others are silently filtered out.
+3. The **display name** comes from `properties.event` (not the span event name), mapped through `getFriendlyNameForEvent()`.
+4. Events are shown on the **span they belong to** - events on one span don't appear in another span's timeline.
+
+## ClickHouse Storage Constraint
+
+When events are written to ClickHouse, `spanEventsToTaskEventV1Input()` filters out events whose `start_time` is not greater than the parent span's `startTime`. Events at or before the span start are silently dropped. This means span events must have timestamps strictly after the span's own `startTimeUnixNano`.
+
+## Timeline Rendering (SpanTimeline component)
+
+The `SpanTimeline` component in `app/components/run/RunTimeline.tsx` renders:
+
+1. **Events** (thin 1px line with hollow dots) - all events from `createTimelineSpanEventsFromSpanEvents()`
+2. **"Started"** marker (thick cap) - at the span's `startTime`
+3. **Duration bar** (thick 7px line) - from "Started" to "Finished"
+4. **"Finished"** marker (thick cap) - at `startTime + duration`
+
+The thin line before "Started" only appears when there are events with timestamps between the span start and the first child span. For the Attempt span this works well (Dequeued -> Pod scheduled -> Launched -> etc. all happen before execution starts). Events all get `lineVariant: "light"` (thin) while the execution bar gets `variant: "normal"` (thick).
+
+## Trace View Sort Order
+
+Sibling spans (same parent) are sorted by `start_time ASC` from the ClickHouse query. The `createTreeFromFlatItems` function preserves this order. Event timestamps don't affect sort order - only the span's own `start_time`.
+
+## Event Structure
+
+```typescript
+// OTel span event format
+{
+  name: "trigger.dev/run",        // Must start with "trigger.dev/" to render
+  timeUnixNano: "1711200000000000000",
+  attributes: [
+    { key: "event", value: { stringValue: "dequeue" } },  // The actual event type
+    { key: "duration", value: { intValue: 150 } },         // Optional: duration in ms
+  ]
+}
+```
+
+## Admin-Only Events
+
+`getAdminOnlyForEvent()` controls visibility. Events default to **admin-only** (`true`).
+
+| Event | Admin-only | Friendly name |
+|-------|-----------|---------------|
+| `dequeue` | No | Dequeued |
+| `fork` | No | Launched |
+| `import` | No (if no fork event) | Importing task file |
+| `create_attempt` | Yes | Attempt created |
+| `lazy_payload` | Yes | Lazy attempt initialized |
+| `pod_scheduled` | Yes | Pod scheduled |
+| (default) | Yes | (raw event name) |
+
+## Adding New Timeline Events
+
+1. Add OTLP span event with `name: "trigger.dev/<scope>"` and `properties.event: "<type>"`
+2. Event timestamp must be strictly after the parent span's `startTimeUnixNano` (ClickHouse drops earlier events)
+3. Add friendly name in `getFriendlyNameForEvent()` in `app/utils/timelineSpanEvents.ts`
+4. Set admin visibility in `getAdminOnlyForEvent()`
+5. Optionally add help text in `getHelpTextForEvent()`
+
+## Key Files
+
+- `app/utils/timelineSpanEvents.ts` - filtering, naming, admin logic
+- `app/components/run/RunTimeline.tsx` - `SpanTimeline` component (thin line + thick bar rendering)
+- `app/presenters/v3/SpanPresenter.server.ts` - loads span data including events
+- `app/v3/eventRepository/clickhouseEventRepository.server.ts` - `spanEventsToTaskEventV1Input()` (storage filter), `#mergeRecordsIntoSpanDetail` (reassembly)

.server-changes/compute-template-shadow-mode.md

@@ -0,0 +1,6 @@
+---
+area: webapp
+type: feature
+---
+
+Pre-warm compute templates on deploy for orgs with compute access. Required for projects using a compute region, background-only for others.

CLAUDE.md

@@ -80,6 +80,10 @@ pnpm run changeset:add
 
 When modifying only server components (`apps/webapp/`, `apps/supervisor/`, etc.) with no package changes, add a `.server-changes/` file instead. See `.server-changes/README.md` for format and documentation.
 
+## Dependency Pinning
+
+Zod is pinned to a single version across the entire monorepo (currently `3.25.76`). When adding zod to a new or existing package, use the **exact same version** as the rest of the repo - never a different version or a range. Mismatched zod versions cause runtime type incompatibilities (e.g., schemas from one package can't be used as body validators in another).
+
 ## Architecture Overview
 
 ### Request Flow

apps/supervisor/package.json

@@ -14,9 +14,11 @@
   },
   "dependencies": {
     "@aws-sdk/client-ecr": "^3.839.0",
+    "@internal/compute": "workspace:*",
     "@kubernetes/client-node": "^1.0.0",
     "@trigger.dev/core": "workspace:*",
     "dockerode": "^4.0.6",
+    "p-limit": "^6.2.0",
     "prom-client": "^15.1.0",
     "socket.io": "4.7.4",
     "std-env": "^3.8.0",