Skip to content

Commit c017aa2

Browse files
ericallamericallam
committed
code sandbox and database patterns
1 parent bd345bb commit c017aa2

File tree

2 files changed

+252
-0
lines changed

2 files changed

+252
-0
lines changed

‎docs/ai-chat/patterns/code-sandbox.mdx‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
---
2+
title: "Code execution sandbox"
3+
sidebarTitle: "Code sandbox"
4+
description: "Warm an isolated sandbox on each chat turn, run an AI SDK executeCode tool, and tear down right before the run suspends — using chat.task hooks and chat.local."
5+
---
6+
7+
Use a **hosted code sandbox** (for example [E2B](https://e2b.dev)) when the model should run short scripts to analyze tool output (PostHog queries, CSV-like data, math) without executing arbitrary code on the Trigger worker host.
8+
9+
This page describes a **durable chat** pattern that fits `chat.task()`:
10+
11+
- **Warm** the sandbox at the start of each turn (**non-blocking**).
12+
- **Reuse** it for every `executeCode` tool call during that turn (and across turns in the same run if you keep the handle).
13+
- **Dispose** it **right before the run suspends** waiting for the next user message — using the **`onChatSuspend`** hook, not `onTurnComplete`.
14+
15+
<Info>
16+
The reference implementation lives in the monorepo at [`references/ai-chat`](https://github.com/triggerdotdev/trigger.dev/tree/main/references/ai-chat) (`code-sandbox.ts`, `chat-tools.ts`, `trigger/chat.ts`).
17+
</Info>
18+
19+
## Why not tear down in `onTurnComplete`?
20+
21+
After a turn finishes, the chat runtime still goes through an **idle** window and only then suspends. During that window the run is still executing — useful for `chat.defer()` work — and the run hasn't suspended yet.
22+
23+
The boundary you want for “turn done, about to sleep” is **`onChatSuspend`**, which fires right before the run transitions from idle to suspended. It provides the `phase` (`”preload”` or `”turn”`) and full chat context. See [onChatSuspend / onChatResume](/ai-chat/backend#onchatsuspend--onchatresume).
24+
25+
```mermaid
26+
sequenceDiagram
27+
participant TurnStart as onTurnStart
28+
participant Run as run / streamText
29+
participant TurnDone as onTurnComplete
30+
participant Idle as Idle window
31+
participant Suspend as onChatSuspend
32+
participant Sleep as suspended
33+
34+
TurnStart->>Run: warm sandbox (async)
35+
Run->>TurnDone: persist / inject / etc.
36+
TurnDone->>Idle: still running
37+
Idle->>Suspend: dispose sandbox
38+
Suspend->>Sleep: waiting for next message
39+
```
40+
41+
## Recommended provider: E2B
42+
43+
- **API key** auth — works from any Trigger.dev worker; no Vercel-only OIDC.
44+
- **Code Interpreter** SDK (`@e2b/code-interpreter`): long-lived sandbox, `runCode()`, `kill()`.
45+
46+
Alternatives (Modal, Daytona, raw Docker) are fine but more DIY. Vercel’s sandbox + AI SDK helpers are a better fit when execution stays **on Vercel**, not on the Trigger worker.
47+
48+
## Implementation sketch
49+
50+
### 1. Run-scoped sandbox map
51+
52+
Keep a `Map<runId, Promise<Sandbox>>` (or similar) in a **task-only module** so your Next.js app never imports it.
53+
54+
### 2. `onTurnStart` — warm without blocking
55+
56+
```ts
57+
onTurnStart: async ({ runId, ctx, ...rest }) => {
58+
warmCodeSandbox(runId); // fire-and-forget Sandbox.create()
59+
// ...persist messages, writer, etc.
60+
},
61+
```
62+
63+
### 3. `chat.local` — run id for tools
64+
65+
Tool `execute` functions do not receive hook payloads. Use [`chat.local()`](/ai-chat/features#per-run-data-with-chatlocal) to store the current run id for the sandbox key, **initialized from `onTurnStart`** (same `runId` as the map):
66+
67+
```ts
68+
// In the same task module as your tools
69+
import { chat } from "@trigger.dev/sdk/ai";
70+
71+
export const codeSandboxRun = chat.local<{ runId: string }>({ id: "codeSandboxRun" });
72+
73+
export function warmCodeSandbox(runId: string) {
74+
codeSandboxRun.init({ runId });
75+
// ...start Sandbox.create(), store promise in Map by runId
76+
}
77+
```
78+
79+
The **`executeCode`** tool reads `codeSandboxRun.runId` and awaits the sandbox promise before `runCode`.
80+
81+
### 4. `onChatSuspend` / `onComplete` — teardown
82+
83+
Use **`onChatSuspend`** to dispose the sandbox right before the run suspends, and **`onComplete`** as a safety net when the run ends entirely.
84+
85+
```ts
86+
export const aiChat = chat.task({
87+
id: "ai-chat",
88+
// ...
89+
onChatSuspend: async ({ phase, ctx }) => {
90+
await disposeCodeSandboxForRun(ctx.run.id);
91+
},
92+
onComplete: async ({ ctx }) => {
93+
await disposeCodeSandboxForRun(ctx.run.id);
94+
},
95+
});
96+
```
97+
98+
Unlike `onWait` (which fires for all wait types), `onChatSuspend` only fires at chat suspension points — no need to filter on `wait.type`. The `phase` discriminator tells you if this is a preload or post-turn suspension.
99+
100+
Optional **`onChatResume`**: log or reset flags; a fresh sandbox can be warmed again on the next **`onTurnStart`**.
101+
102+
### 5. AI SDK tool
103+
104+
Wrap the provider in a normal AI SDK `tool({ inputSchema, execute })` (same pattern as `webFetch`). Keep tool definitions in **task code**, not in the Next.js server bundle.
105+
106+
### 6. Environment
107+
108+
Set **`E2B_API_KEY`** (or your provider’s secret) on the **Trigger environment** for the worker — not in public client env.
109+
110+
## Typing `ctx`
111+
112+
Every `chat.task` lifecycle event and the `run` payload include **`ctx`**: the same **[`TaskRunContext`](/ai-chat/reference#task-context-ctx)** shape as `task({ run: (payload, { ctx }) => ... })`.
113+
114+
```ts
115+
import type { TaskRunContext } from "@trigger.dev/sdk";
116+
```
117+
118+
The alias **`Context`** is also exported from `@trigger.dev/sdk` and is the same type.
119+
120+
## See also
121+
122+
- [Database persistence for chat](/ai-chat/patterns/database-persistence) — conversation + session rows, hooks, token renewal
123+
- [Backend — Lifecycle hooks](/ai-chat/backend#lifecycle-hooks)
124+
- [API Reference — `ctx` on events](/ai-chat/reference#task-context-ctx)
125+
- [Per-run data with `chat.local`](/ai-chat/features#per-run-data-with-chatlocal)

‎docs/ai-chat/patterns/database-persistence.mdx‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
---
2+
title: "Database persistence for chat"
3+
sidebarTitle: "Database persistence"
4+
description: "Split conversation state and live session metadata across hooks — preload, turn start, turn complete — without tying the pattern to a specific ORM or schema."
5+
---
6+
7+
Durable chat runs can span **hours** and **many turns**. You usually want:
8+
9+
1. **Conversation state** — full **`UIMessage[]`** (or equivalent) keyed by **`chatId`**, so reloads and history views work.
10+
2. **Live session state** — the **current Trigger `runId`**, a **scoped access token** for realtime + input streams, and optionally **`lastEventId`** for stream resume.
11+
12+
This page describes a **hook mapping** that works with any database. The [ai-chat reference app](https://github.com/triggerdotdev/trigger.dev/tree/main/references/ai-chat) implements the same idea with a SQL database and an ORM; adapt table and column names to your stack.
13+
14+
## Conceptual data model
15+
16+
You can use one table or two; the important split is **semantic**:
17+
18+
| Concept | Purpose | Typical fields |
19+
| ------- | ------- | -------------- |
20+
| **Conversation** | Durable transcript + display metadata | Stable id (same as **`chatId`**), serialized **`uiMessages`**, title, model choice, owner/user id, timestamps |
21+
| **Active session** | Reconnect + resume the **same** run | Same **`chatId`** as key (or FK), **current `runId`**, **`publicAccessToken`** (or your stored PAT), optional **`lastEventId`** |
22+
23+
The **conversation** row is what your UI lists as “chats.” The **session** row is what the **transport** needs after a refresh or token expiry: *which run is live* and *how to authenticate* to it.
24+
25+
<Note>
26+
Store **`UIMessage[]`** in a JSON-compatible column, or normalize to a messages table — the pattern is *when* you read/write, not *how* you encode rows.
27+
</Note>
28+
29+
## Where each hook writes
30+
31+
### `onPreload` (optional)
32+
33+
When the user triggers [preload](/ai-chat/features#preload), the run starts **before** the first user message.
34+
35+
- Ensure the **conversation** row exists (create or no-op).
36+
- **Upsert session**: **`runId`**, **`chatAccessToken`** from the event (this is the turn-scoped token for that run).
37+
- Load any **user / tenant context** you need for prompts (`clientData`).
38+
39+
If you skip preload, do the equivalent in **`onChatStart`** when **`preloaded`** is false.
40+
41+
### `onChatStart` (turn 0, non-preloaded path)
42+
43+
- If **`preloaded`** is true, return early — **`onPreload`** already ran.
44+
- Otherwise mirror preload: user/context, conversation create, session upsert.
45+
- If **`continuation`** is true, the conversation row usually **already exists** (previous run ended or timed out); only update **session** fields so the **new** run id and token are stored.
46+
47+
### `onTurnStart`
48+
49+
- Persist **`uiMessages`** (full accumulated history including the new user turn) **before** streaming starts — so a mid-stream refresh still shows the user’s message.
50+
- Optionally use [`chat.defer()`](/ai-chat/features#chat-defer) so the write does not block the model if your driver is slow.
51+
52+
### `onTurnComplete`
53+
54+
- Persist **`uiMessages`** again with the **assistant** reply finalized.
55+
- **Upsert session** with **`runId`**, fresh **`chatAccessToken`**, and **`lastEventId`** from the event.
56+
57+
**`lastEventId`** lets the frontend [resume](/ai-chat/frontend) without replaying SSE events it already applied. Treat it as part of session state, not optional polish, if you care about duplicate chunks after refresh.
58+
59+
## Token renewal (app server)
60+
61+
Turn tokens expire (see **`chatAccessTokenTTL`** on **`chat.task`**). When the transport gets **401** on realtime or input streams, mint a **new** public access token with the **same** scopes the task uses — typically **read** for that **`runId`** and **write** for **input streams** on that run — then **persist** it on your **session** row.
62+
63+
Your **Next.js server action**, **Remix action**, or **API route** should:
64+
65+
1. Load **session** by **`chatId`** → **`runId`**.
66+
2. Call **`auth.createPublicToken`** (or your platform’s equivalent) with those scopes.
67+
3. Save the new token (and confirm **`runId`** is unchanged unless you started a new run).
68+
69+
No Trigger task code needs to run for renewal.
70+
71+
## Minimal pseudocode
72+
73+
```typescript
74+
// Pseudocode — replace saveConversation / saveSession with your DB layer.
75+
76+
chat.task({
77+
id: "my-chat",
78+
clientDataSchema: z.object({ userId: z.string() }),
79+
80+
onPreload: async ({ chatId, runId, chatAccessToken, clientData }) => {
81+
if (!clientData) return;
82+
await ensureUser(clientData.userId);
83+
await upsertConversation({ id: chatId, userId: clientData.userId /* ... */ });
84+
await upsertSession({ chatId, runId, publicAccessToken: chatAccessToken });
85+
},
86+
87+
onChatStart: async ({ chatId, runId, chatAccessToken, clientData, continuation, preloaded }) => {
88+
if (preloaded) return;
89+
await ensureUser(clientData.userId);
90+
if (!continuation) {
91+
await upsertConversation({ id: chatId, userId: clientData.userId /* ... */ });
92+
}
93+
await upsertSession({ chatId, runId, publicAccessToken: chatAccessToken });
94+
},
95+
96+
onTurnStart: async ({ chatId, uiMessages }) => {
97+
chat.defer(saveConversationMessages(chatId, uiMessages));
98+
},
99+
100+
onTurnComplete: async ({ chatId, uiMessages, runId, chatAccessToken, lastEventId }) => {
101+
await saveConversationMessages(chatId, uiMessages);
102+
await upsertSession({
103+
chatId,
104+
runId,
105+
publicAccessToken: chatAccessToken,
106+
lastEventId,
107+
});
108+
},
109+
110+
run: async ({ messages, signal }) => {
111+
/* streamText, etc. */
112+
},
113+
});
114+
```
115+
116+
## Design notes
117+
118+
- **`chatId`** is stable for the life of a thread; **`runId`** changes when the user starts a **new** run (timeout, cancel, explicit new chat). Session rows must always reflect the **current** run.
119+
- **`continuation: true`** means “same logical chat, new run” — update session, don’t assume an empty conversation.
120+
- Keep **task modules** that perform writes **out of** browser bundles; the pattern assumes persistence runs **in the worker** (or your BFF that the task calls).
121+
122+
## See also
123+
124+
- [Backend — Lifecycle hooks](/ai-chat/backend#lifecycle-hooks)
125+
- [Session management](/ai-chat/frontend#session-management) — `resume`, `lastEventId`, transport
126+
- [`chat.defer()`](/ai-chat/features#chat-defer) — non-blocking writes during a turn
127+
- [Code execution sandbox](/ai-chat/patterns/code-sandbox) — combines **`onWait`** / **`onComplete`** with this persistence model

0 commit comments

Comments
 (0)