Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ Check if `temporal` CLI is installed. If not, follow the instructions at `refere
- Language-specific info at `references/{your_language}/versioning.md`
- **`references/core/standalone-activities.md`** - Standalone Activities: run an Activity directly from a Client without a Workflow (Public Preview)
- Language-specific info at `references/{your_language}/standalone-activities.md`
- **`references/core/serialization-context.md`** - Serialization context: External Storage drivers receive a context with namespace + Workflow/Activity identity, including a Standalone-Activity branch (Public Preview)
- Language-specific info at `references/{your_language}/serialization-context.md`
- **`references/core/troubleshooting.md`** - Decision trees, recovery procedures
- **`references/core/error-reference.md`** - Common error types, workflow status reference
- **`references/core/interactive-workflows.md`** - Testing signals, updates, queries
Expand Down
67 changes: 67 additions & 0 deletions references/core/serialization-context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
> [!NOTE]
> This feature is in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview.

# Serialization Context (Concepts)

For language-specific implementation, see `references/{your_language}/serialization-context.md`.

## What "serialization context" means in Temporal today

When the SDK serializes a Payload on the way out of your application, the **Data Conversion pipeline** runs Payload Converters first, then Payload Codecs, then — if configured — hands the encoded bytes to an **External Storage driver** that offloads large payloads to a backing store like Amazon S3.

The External Storage **storage driver** is the one stage of that pipeline that receives a context object alongside the payloads. This context carries identity information about the operation that triggered the serialization — namespace plus the owning Workflow or, for [Standalone Activities](references/core/standalone-activities.md), the owning Activity.

The current Public Preview surface only exposes this context to **storage drivers** (and to the driver-selection callback when multiple drivers are registered). Payload Converters and Payload Codecs do not currently receive a serialization-context argument in Temporal's documented SDK APIs.

## The context object and its `Target`

Two parallel context types exist, one per direction of the pipeline:

- A **store** context delivered to the driver's store/upload entry point.
- A **retrieve** context delivered to the driver's retrieve/download entry point.

Each context carries a `Target` (Go) / `target` (Python) field that is a **discriminated union** over two concrete types:

- A **WorkflowInfo** branch, present whenever the operation is associated with a Workflow Execution. Activities **started by a Workflow** are reported through this branch too.
- An **ActivityInfo** branch, present **only for Standalone Activities** (Activities started directly from a Temporal Client, not orchestrated by a Workflow).

This is the headline rule: if your driver branches on the target type, the `ActivityInfo` arm is the **Standalone-Activity-only** path. For Workflow-bound activities, you still receive `WorkflowInfo`.

## What's on each branch

The exact field names differ across SDKs — see the per-language reference for the verbatim API:

- **WorkflowInfo** carries the namespace and the Workflow ID.
- **ActivityInfo** carries the namespace and the Activity ID (no Workflow ID, because there is no Workflow).

Driver authors typically:

1. Switch / `isinstance` on the target type.
2. Build a storage key prefix from the namespace plus the workflow-or-activity ID.
3. Write payloads under that prefix so storage is naturally partitioned by execution.

## Where this context shows up

- The driver's **store** method (Go: `Store(ctx, payloads)`; Python: `async def store(self, context, payloads)`).
- The driver's **retrieve** method (Go: `Retrieve(ctx, claims)`; Python: `async def retrieve(self, context, claims)`).
- The **driver-selector** callback used when more than one storage driver is registered. The selector receives the store context plus the payload and returns the driver to use (or `nil` / `None` to keep the payload inline).

## SDK coverage

Documented SDK guides for External Storage and its context object exist for:

- **Go SDK** — see `references/go/serialization-context.md`.
- **Python SDK** — see `references/python/serialization-context.md`.

The encyclopedia External Storage page lists only those two SDKs. The Temporal **.NET SDK** documentation is currently silent on a serialization-context object; treat `references/dotnet/serialization-context.md` as a placeholder until .NET docs add coverage.

## Common mistakes

- **Assuming `ActivityInfo` covers Workflow-bound activities.** It does not. An Activity scheduled by a Workflow surfaces as `WorkflowInfo` (Workflow ID is the orchestrator's ID). Only [Standalone Activities](references/core/standalone-activities.md) produce `ActivityInfo`.
- **Reaching for workflow info from inside a custom Payload Converter or Payload Codec.** No documented SDK API exposes a context to those stages today; only the StorageDriver does. If you need the owning Workflow ID at serialization time, implement a custom storage driver (or driver selector) rather than smuggling state into a converter.
- **Treating `Target` as a struct with a `Kind` enum.** It is a discriminated union — a Go interface satisfied by either concrete type, or a Python target whose runtime type you check with `isinstance`. Use a type switch / `isinstance`.

## Resources

- External Storage encyclopedia page: <https://docs.temporal.io/external-storage>
- Standalone Activities concept page: see `references/core/standalone-activities.md`.
2 changes: 2 additions & 0 deletions references/dotnet/data-handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,3 +215,5 @@ public class MyWorkflow
3. Encrypt sensitive data with `IPayloadCodec`
4. Use `Workflow.NewGuid()` and `Workflow.Random` for deterministic values
5. Use camelCase converter if interoperating with other SDKs

For the serialization-context object that other SDKs pass to External Storage drivers — and the current .NET coverage gap — see `references/dotnet/serialization-context.md`.
1 change: 1 addition & 0 deletions references/dotnet/dotnet.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,3 +201,4 @@ See `references/dotnet/testing.md` for info on writing tests.
- **`references/dotnet/versioning.md`** — Patching API, workflow type versioning, Worker Versioning
- **`references/dotnet/standalone-activities.md`** — Standalone Activities: run an Activity directly from a Client without a Workflow (Public Preview). Concept overview at `references/core/standalone-activities.md`.
- **`references/dotnet/determinism-protection.md`** — Runtime task detection, .NET Task determinism rules
- **`references/dotnet/serialization-context.md`** — Serialization context for External Storage drivers. .NET docs are currently silent on this surface; the file is a placeholder with `` markers. Concept overview at `references/core/serialization-context.md`.
31 changes: 31 additions & 0 deletions references/dotnet/serialization-context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
> [!NOTE]
> External Storage and its serialization-context object are in Public Preview in the SDKs that ship it. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview.

## Overview

[Cross-SDK concept file](references/core/serialization-context.md).

The Temporal **.NET SDK documentation currently has no published surface for a serialization-context object** delivered to Data Converter components. The encyclopedia External Storage page lists SDK-specific guides for the Go SDK and Python SDK only; there is no .NET SDK guide.

What the .NET docs **do** describe is the existing Data Converter and Codec surface in `Temporalio.Converters`. `IPayloadCodec` exposes `EncodeAsync(IReadOnlyCollection<Payload>)` and `DecodeAsync(IReadOnlyCollection<Payload>)` — neither overload receives a context parameter.

Treat this page as a placeholder. If a user asks how to access Workflow/Activity identity inside a .NET custom converter or codec today, the supported answer is "the .NET SDK does not document one."

## Prerequisites

## Storage driver context

## Reading workflow vs activity metadata

## Driver selection

## Common mistakes

- **Assuming the .NET SDK has the same `SerializationContext` / `StorageDriver` API as Go or Python.** It does not, as of the current published documentation. Don't generate .NET code that imports `Temporalio.Converters.StorageDriverStoreContext` or similar based on the Go/Python names.
- **Reaching for a context parameter on `IPayloadCodec.EncodeAsync` / `DecodeAsync`.** Neither method receives a context argument in the documented .NET API.

## Resources

- .NET Data Converters and encryption: <https://docs.temporal.io/develop/dotnet/data-handling> (see "Converters and encryption").
- External Storage encyclopedia page (Go and Python only, today): <https://docs.temporal.io/external-storage>
- Cross-SDK concept file: `references/core/serialization-context.md`.
2 changes: 2 additions & 0 deletions references/go/data-handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,3 +262,5 @@ err := workflow.UpsertMemo(ctx, map[string]interface{}{
3. Keep payloads small -- see `references/core/gotchas.md` for limits
4. Use `PayloadCodec` for encryption; never store sensitive data unencrypted
5. Configure the same data converter on both client and worker

For External Storage drivers that need the owning Workflow or Standalone-Activity identity, see `references/go/serialization-context.md`.
1 change: 1 addition & 0 deletions references/go/go.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,3 +252,4 @@ See `references/go/testing.md` for info on writing tests.
- **`references/go/data-handling.md`** - Data converters, payload codecs, encryption
- **`references/go/versioning.md`** - Patching API (`workflow.GetVersion`), Worker Versioning
- **`references/go/determinism-protection.md`** - Information on **`workflowcheck`** tool to help statically check for determinism issues.
- **`references/go/serialization-context.md`** - Serialization context delivered to External Storage drivers (`converter.StorageDriverStoreContext` / `converter.StorageDriverRetrieveContext`), including the Standalone-Activity branch. Concept overview at `references/core/serialization-context.md`.
89 changes: 89 additions & 0 deletions references/go/serialization-context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
> [!NOTE]
> This feature is in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview.

## Overview

[Cross-SDK concept file](references/core/serialization-context.md).

The Go SDK delivers a **serialization context** — `converter.StorageDriverStoreContext` on the way in and `converter.StorageDriverRetrieveContext` on the way out — to every custom External Storage driver. The context's `Target` field is a discriminated union (`converter.StorageDriverWorkflowInfo` or `converter.StorageDriverActivityInfo`) that carries namespace and execution-identity metadata, including the Standalone-Activity case.

## Prerequisites

- A Temporal Go SDK version that supports External Storage and the `converter.StorageDriver` interface. External Storage is in Public Preview; consult the Go SDK release notes for the minimum supported version.
- An existing custom storage driver, or willingness to write one — see the `references/go/data-handling.md` file for the broader Data Converter setup that External Storage plugs into.

## Storage driver context

A `converter.StorageDriver` exposes `Store` and `Retrieve` methods. The first argument to each is the context object:

```go
func (d *MyDriver) Store(
ctx converter.StorageDriverStoreContext,
payloads []*commonpb.Payload,
) ([]converter.StorageDriverClaim, error) { ... }

func (d *MyDriver) Retrieve(
ctx converter.StorageDriverRetrieveContext,
claims []converter.StorageDriverClaim,
) ([]*commonpb.Payload, error) { ... }
```

Both context types carry a `Target` field that identifies the owning execution.

## Reading workflow vs activity metadata

`ctx.Target` is an interface satisfied by one of two concrete types. Use a Go type switch to extract identity fields:

```go
switch info := ctx.Target.(type) {
case converter.StorageDriverWorkflowInfo:
// Workflow-scoped operation, including Activities started by a Workflow.
// Fields: info.Namespace, info.WorkflowID
if info.WorkflowID != "" {
// build key from info.Namespace + info.WorkflowID
}
case converter.StorageDriverActivityInfo:
// StorageDriverActivityInfo is only used for standalone (non-workflow-bound)
// activities. Activities started by a workflow use StorageDriverWorkflowInfo.
// Fields: info.Namespace, info.ActivityID
if info.ActivityID != "" {
// build key from info.Namespace + info.ActivityID
}
}
```

Key facts:

- The `StorageDriverActivityInfo` arm fires **only for [Standalone Activities](references/core/standalone-activities.md)** — Activities started directly from a Client without a Workflow.
- An Activity scheduled by a Workflow is reported on the `StorageDriverWorkflowInfo` arm, with the orchestrating Workflow's ID.
- The docs describe the identity fields as "namespace, Workflow ID" depending on the operation; use the type switch to access the concrete values.

## Driver selection

When more than one driver is registered, a `StorageDriverSelector` decides which driver stores a given payload. Its `SelectDriver` method receives the same `StorageDriverStoreContext`, so the selection logic can branch on namespace, Workflow ID, or whether the operation is a Standalone Activity:

```go
func (s *PreferredSelector) SelectDriver(
ctx converter.StorageDriverStoreContext,
payload *commonpb.Payload,
) (converter.StorageDriver, error) {
// Inspect ctx.Target the same way Store does.
return s.preferred, nil
}
```

Return `nil` for the driver to keep a payload inline in Event History instead of offloading.

## Common mistakes

- **Switching on `Target` with a default-case fallthrough that assumes Workflow identity.** A driver that runs against Standalone Activities will land in `StorageDriverActivityInfo` and have no `WorkflowID`. Branch explicitly.
- **Reading `info.WorkflowID` on `StorageDriverActivityInfo`.** The activity branch carries `info.Namespace` and `info.ActivityID` only.
- **Expecting a context on `PayloadConverter` or `PayloadCodec`.** The Go SDK's serialization-context object is only passed to the `StorageDriver` interface (and the `StorageDriverSelector`). To key behavior off Workflow/Activity identity, do it in the storage driver, not in a custom Payload Converter.
- **Changing a driver's `Name()` after payloads have been written.** The SDK records the name on each claim; renaming breaks retrieval.

## Resources

- External Storage encyclopedia page: <https://docs.temporal.io/external-storage>
- Go SDK External Storage guide: <https://docs.temporal.io/develop/go/data-handling/external-storage>
- Cross-SDK concept file: `references/core/serialization-context.md`.
- Standalone Activities (Go): see the language-specific reference under `references/{your_language}/standalone-activities.md`.
2 changes: 1 addition & 1 deletion references/python/advanced-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ client = await Client.connect(

- The only field is `resolution_interval_millis: int = 30000` — how often to re-resolve DNS, in milliseconds.
- `DnsLoadBalancingConfig.default` is a pre-built instance with the default 30-second interval.
- `dns_load_balancing_config` defaults to 30 seconds if you don't pass anything explicitly.
- `dns_load_balancing_config` defaults to 30 seconds if you don't pass anything explicitly.
- Pass `dns_load_balancing_config=None` to disable DNS load balancing entirely.

### Mutual exclusion with HTTP CONNECT proxy
Expand Down
2 changes: 2 additions & 0 deletions references/python/data-handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,3 +230,5 @@ class MyWorkflow:
3. Encrypt sensitive data with PayloadCodec
4. Use dataclasses for simple data structures
5. Use `workflow.uuid4()` and `workflow.random()` for deterministic values

For External Storage drivers that need the owning Workflow or Standalone-Activity identity, see `references/python/serialization-context.md`.
1 change: 0 additions & 1 deletion references/python/integrations/google-adk.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ The integration is built on the Python SDK [Plugin system](https://docs.temporal
> [!NOTE]
> This feature is in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview.


For general Temporal AI/LLM patterns (retries, rate limits, multi-agent orchestration) see `references/core/ai-patterns.md` and `references/python/ai-patterns.md`.

## Prerequisites
Expand Down
2 changes: 1 addition & 1 deletion references/python/integrations/langgraph.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,4 +214,4 @@ For LangSmith tracing of LangGraph nodes and Temporal Activities together, use t

- `references/python/ai-patterns.md` — Python AI/LLM patterns (Pydantic data converter, LLM Activity design, retry/error classification).
- `references/core/ai-patterns.md` — language-agnostic AI/LLM patterns.
- `references/python/integrations/langsmith.md` - Companion LangSmith plugin.
- `references/python/integrations/langsmith.md` - Companion LangSmith plugin.
1 change: 0 additions & 1 deletion references/python/integrations/langsmith.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,6 @@ The plugin makes `@traceable` replay-safe in the Workflow sandbox. You do not ne
- The plugin injects metadata using `workflow.now()` for timestamps and `workflow.random()` for UUIDs instead of `datetime.now()` and `uuid4()`.
- LangSmith HTTP calls run on a background thread pool that does not interfere with deterministic Workflow execution.


## Context propagation

Trace context flows automatically across Client → Workflow → Activity → Child Workflow → Nexus via Temporal headers. Do not pass context manually.
Expand Down
1 change: 0 additions & 1 deletion references/python/integrations/openai-agents-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,6 @@ Note that the initial run context comes from the `context=...` argument you pass

In addition, since a `@function_tool` runs in the workflow, they can also call Temporal activities or other durable primitives themselves.


**Don't put I/O, system clock, or sources of randomness inside a `@function_tool` body.** Make it an `@activity.defn` and wrap with `activity_as_tool` instead.

### Picking between the two
Expand Down
1 change: 1 addition & 0 deletions references/python/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ See `references/python/testing.md` for info on writing tests.
- **`references/python/data-handling.md`** - Data converters, Pydantic, payload encryption
- **`references/python/versioning.md`** - Patching API, workflow type versioning, Worker Versioning
- **`references/python/standalone-activities.md`** - Standalone Activities: run an Activity directly from a Client without a Workflow (Public Preview). Concept overview at `references/core/standalone-activities.md`.
- **`references/python/serialization-context.md`** - Serialization context delivered to External Storage drivers (`StorageDriverStoreContext` / `StorageDriverRetrieveContext`), including the Standalone-Activity branch. Concept overview at `references/core/serialization-context.md`.
- **`references/python/determinism-protection.md`** - Python sandbox specifics, forbidden operations, pass-through imports
- **`references/python/ai-patterns.md`** - LLM integration, Pydantic data converter, AI workflow patterns
- **`references/python/workflow-streams.md`** - Public-Preview `temporalio.contrib.workflow_streams` library: durable, offset-addressed event channel for streaming progress to subscribers.
Expand Down
Loading