Skip to content

Commit 585b781

Browse files
ericallamericallam
committed
Add background injection docs
1 parent 5bd9605 commit 585b781

File tree

3 files changed

+221
-0
lines changed

3 files changed

+221
-0
lines changed

‎docs/ai-chat/backend.mdx‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -584,6 +584,34 @@ On the frontend, the `usePendingMessages` hook handles sending, tracking, and re
584584
See [Pending Messages](/ai-chat/pending-messages) for the full guide — backend configuration, frontend hook, queuing vs steering, and how injection works with all three chat variants.
585585
</Tip>
586586

587+
### Background injection
588+
589+
Inject context from background work into the conversation using `chat.inject()`. Combine with `chat.defer()` to run analysis between turns and inject results before the next response — self-review, RAG augmentation, safety checks, etc.
590+
591+
```ts
592+
export const myChat = chat.task({
593+
id: "my-chat",
594+
onTurnComplete: async ({ messages }) => {
595+
chat.defer((async () => {
596+
const review = await generateObject({ /* ... */ });
597+
if (review.object.needsImprovement) {
598+
chat.inject([{
599+
role: "system",
600+
content: `[Self-review]\n${review.object.suggestions.join("\n")}`,
601+
}]);
602+
}
603+
})());
604+
},
605+
run: async ({ messages, signal }) => {
606+
return streamText({ ...chat.toStreamTextOptions({ registry }), messages, abortSignal: signal });
607+
},
608+
});
609+
```
610+
611+
<Tip>
612+
See [Background Injection](/ai-chat/background-injection) for the full guide — timing, self-review example, and how it differs from pending messages.
613+
</Tip>
614+
587615
### prepareMessages
588616

589617
Transform model messages before they're used anywhere — in `run()`, in compaction rebuilds, and in compaction results. Define once, applied everywhere.

‎docs/ai-chat/background-injection.mdx‎

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
---
2+
title: "Background injection"
3+
sidebarTitle: "Background injection"
4+
description: "Inject context from background work into the agent's conversation — self-review, RAG augmentation, or any async analysis."
5+
---
6+
7+
## Overview
8+
9+
`chat.inject()` queues model messages for injection into the conversation. Messages are picked up at the start of the next turn or at the next `prepareStep` boundary (between tool-call steps).
10+
11+
This is the backend counterpart to [pending messages](/ai-chat/pending-messages) — pending messages come from the user via the frontend, while `chat.inject()` comes from your task code.
12+
13+
## Basic usage
14+
15+
```ts
16+
import { chat } from "@trigger.dev/sdk/ai";
17+
18+
// Queue a system message for injection
19+
chat.inject([
20+
{
21+
role: "system",
22+
content: "The user's account was just upgraded to Pro.",
23+
},
24+
]);
25+
```
26+
27+
Messages are appended to the model messages before the next LLM inference call. The LLM sees them as part of the conversation context.
28+
29+
## Common pattern: defer + inject
30+
31+
The most powerful pattern combines `chat.defer()` (background work) with `chat.inject()` (inject results). Background work runs in parallel with the idle wait between turns, and results are injected before the next response.
32+
33+
```ts
34+
export const myChat = chat.task({
35+
id: "my-chat",
36+
onTurnComplete: async ({ messages }) => {
37+
// Kick off background analysis — doesn't block the turn
38+
chat.defer(
39+
(async () => {
40+
const analysis = await analyzeConversation(messages);
41+
chat.inject([
42+
{
43+
role: "system",
44+
content: `[Analysis of conversation so far]\n\n${analysis}`,
45+
},
46+
]);
47+
})()
48+
);
49+
},
50+
run: async ({ messages, signal }) => {
51+
return streamText({
52+
...chat.toStreamTextOptions({ registry }),
53+
messages,
54+
abortSignal: signal,
55+
});
56+
},
57+
});
58+
```
59+
60+
### Timing
61+
62+
1. Turn completes, `onTurnComplete` fires
63+
2. `chat.defer()` registers the background work
64+
3. The run immediately starts waiting for the next message (no blocking)
65+
4. Background work completes, `chat.inject()` queues the messages
66+
5. User sends next message, turn starts
67+
6. Injected messages are appended before `run()` executes
68+
7. The LLM sees the injected context alongside the new user message
69+
70+
If the background work finishes *during* a tool-call loop (not between turns), the messages are picked up at the next `prepareStep` boundary instead.
71+
72+
## Example: self-review
73+
74+
A cheap model reviews the agent's response after each turn and injects coaching for the next one. Uses [Prompts](/ai/prompts) for the review prompt and `generateObject` for structured output.
75+
76+
```ts
77+
import { chat } from "@trigger.dev/sdk/ai";
78+
import { prompts } from "@trigger.dev/sdk";
79+
import { streamText, generateObject, createProviderRegistry } from "ai";
80+
import { openai } from "@ai-sdk/openai";
81+
import { z } from "zod";
82+
83+
const registry = createProviderRegistry({ openai });
84+
85+
const selfReviewPrompt = prompts.define({
86+
id: "self-review",
87+
model: "openai:gpt-4o-mini",
88+
content: `You are a conversation quality reviewer. Analyze the assistant's most recent response.
89+
90+
Focus on:
91+
- Whether the response answered the user's question
92+
- Missed opportunities to use tools or provide more detail
93+
- Tone mismatches
94+
95+
Be concise. Only flag issues worth fixing.`,
96+
});
97+
98+
export const myChat = chat.task({
99+
id: "my-chat",
100+
onTurnComplete: async ({ messages }) => {
101+
chat.defer(
102+
(async () => {
103+
const resolved = await selfReviewPrompt.resolve({});
104+
105+
const review = await generateObject({
106+
model: registry.languageModel(resolved.model ?? "openai:gpt-4o-mini"),
107+
...resolved.toAISDKTelemetry(),
108+
system: resolved.text,
109+
prompt: messages
110+
.filter((m) => m.role === "user" || m.role === "assistant")
111+
.map((m) => {
112+
const text =
113+
typeof m.content === "string"
114+
? m.content
115+
: Array.isArray(m.content)
116+
? m.content
117+
.filter((p: any) => p.type === "text")
118+
.map((p: any) => p.text)
119+
.join("")
120+
: "";
121+
return `${m.role}: ${text}`;
122+
})
123+
.join("\n\n"),
124+
schema: z.object({
125+
needsImprovement: z.boolean(),
126+
suggestions: z.array(z.string()),
127+
}),
128+
});
129+
130+
if (review.object.needsImprovement) {
131+
chat.inject([
132+
{
133+
role: "system",
134+
content: `[Self-review]\n\n${review.object.suggestions.map((s) => `- ${s}`).join("\n")}\n\nApply these naturally.`,
135+
},
136+
]);
137+
}
138+
})()
139+
);
140+
},
141+
run: async ({ messages, signal }) => {
142+
return streamText({
143+
...chat.toStreamTextOptions({ registry }),
144+
messages,
145+
abortSignal: signal,
146+
});
147+
},
148+
});
149+
```
150+
151+
The self-review runs on `gpt-4o-mini` (fast, cheap) in the background. If the user sends another message before it completes, the coaching is still injected — `chat.inject()` persists across the idle wait.
152+
153+
## Other use cases
154+
155+
- **RAG augmentation**: After each turn, fetch relevant documents and inject them as context for the next response
156+
- **Safety checks**: Run a moderation model on the response, inject warnings if issues are detected
157+
- **Fact-checking**: Verify claims in the response using search tools, inject corrections
158+
- **Context enrichment**: Look up user/account data based on what was discussed, inject it as system context
159+
160+
## How it differs from pending messages
161+
162+
| | `chat.inject()` | [Pending messages](/ai-chat/pending-messages) |
163+
|---|---|---|
164+
| **Source** | Backend task code | Frontend user input |
165+
| **Triggered by** | Your code (e.g. `onTurnComplete` + `chat.defer()`) | User sending a message during streaming |
166+
| **Injection point** | Start of next turn, or next `prepareStep` boundary | Next `prepareStep` boundary only |
167+
| **Message role** | Any (`system`, `user`, `assistant`) | Typically `user` |
168+
| **Frontend visibility** | Not visible unless you write custom `data-*` chunks | Visible via `usePendingMessages` hook |
169+
170+
## API reference
171+
172+
### chat.inject()
173+
174+
```ts
175+
chat.inject(messages: ModelMessage[]): void
176+
```
177+
178+
Queue model messages for injection at the next opportunity. Messages persist across the idle wait between turns — they are not reset when a new turn starts.
179+
180+
**Parameters:**
181+
182+
| Parameter | Type | Description |
183+
|-----------|------|-------------|
184+
| `messages` | `ModelMessage[]` | Model messages to inject (from the `ai` package) |
185+
186+
Messages are drained (consumed) when:
187+
1. A new turn starts — before `run()` executes
188+
2. A `prepareStep` boundary is reached — between tool-call steps during streaming
189+
190+
<Note>
191+
`chat.inject()` writes to an in-memory queue in the current process. It works from any code running in the same task — lifecycle hooks, deferred work, tool execute functions, etc. It does not work from subtasks or other runs.
192+
</Note>

‎docs/docs.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@
9494
"ai-chat/features",
9595
"ai-chat/compaction",
9696
"ai-chat/pending-messages",
97+
"ai-chat/background-injection",
9798
"ai-chat/reference"
9899
]
99100
}

0 commit comments

Comments
 (0)