Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions references/go/go.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,5 +250,6 @@ See `references/go/testing.md` for info on writing tests.
- **`references/go/testing.md`** - TestWorkflowEnvironment, time-skipping, activity mocking
- **`references/go/advanced-features.md`** - Schedules, worker tuning, and more
- **`references/go/data-handling.md`** - Data converters, payload codecs, encryption
- **`references/go/payload-validation.md`** - Eager payload/memo size validation (Experimental, v1.43.0+): warning thresholds and the worker opt-out
- **`references/go/versioning.md`** - Patching API (`workflow.GetVersion`), Worker Versioning
- **`references/go/determinism-protection.md`** - Information on **`workflowcheck`** tool to help statically check for determinism issues.
69 changes: 69 additions & 0 deletions references/go/payload-validation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Go SDK Payload and Memo Size Validation

> [!NOTE]
> This feature is in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview.
>

The Temporal Service rejects payloads above its blob size limit (default 2 MB) and rejects gRPC messages above 4 MB.
Without validation, an oversized payload returned from an Activity or scheduled by a Workflow Task is uploaded to the server, gets rejected, and the workflow may end up stuck in a retry loop or terminated depending on where the payload originated.

The Go SDK validates payload and memo sizes against server-reported error limits **before** sending data to the server. When a size exceeds the limit, the worker fails the current Task locally — the Workflow Execution stays open so the operator can deploy a fix.

## Default behavior

`DisablePayloadErrorLimit` defaults to `false`, so validation is on by default.
The worker fetches the per-namespace error limits from the server. If the server doesn't report limits, the worker doesn't enforce them.

No code change is required to get the default protection — just upgrade the SDK to v1.43.0 or later.

## Warning thresholds — `client.PayloadLimitOptions`

Set warning thresholds on the Client to emit a log when individual payloads or aggregated memo size cross a soft limit, separately from the server-enforced error limit:

```go
import "go.temporal.io/sdk/client"

c, err := client.Dial(client.Options{
PayloadLimits: client.PayloadLimitOptions{
PayloadSizeWarning: 256 * 1024, // bytes
MemoSizeWarning: 4 * 1024, // bytes
},
})
```

Field reference:

- `PayloadSizeWarning int` — bytes; zero/unset defaults to 512 KiB.
- `MemoSizeWarning int` — bytes; zero/unset defaults to 512 KiB.

Warnings are logged via the configured logger; they do not fail tasks.

## Disabling enforcement — `worker.Options.DisablePayloadErrorLimit`

```go
import "go.temporal.io/sdk/worker"

w := worker.New(c, "task-queue", worker.Options{
DisablePayloadErrorLimit: true,
})
```

Set to `true` only when a gRPC proxy between the worker and the server modifies payload bytes after the worker has measured them — for example, a proxy that compresses or re-encodes payloads — so that the worker's pre-upload size measurement no longer reflects what the server will see.

## Effect on errors

When validation rejects a task:

- The worker fails the Workflow Task or Activity Task locally and never uploads the oversized data.
- The Workflow Execution remains open; the task is eligible for retry after the user reduces payload size, switches to External Storage, or otherwise resolves the cause.
- Without validation, oversized inputs cause `WORKFLOW_TASK_FAILED_CAUSE_PAYLOADS_TOO_LARGE`; oversized Workflow results historically caused a server-side retry loop.

## Reducing oversized data

Validation surfaces the problem earlier; it does not raise the limit. To handle data that legitimately exceeds 2 MB, offload to External Storage (claim-check pattern). See `references/go/data-handling.md` for the Payload Converter and Payload Codec, and `docs/develop/go/best-practices/data-handling/external-storage.mdx` for the built-in S3 driver and custom driver interface.

## Gotchas

- **Set `PayloadLimits` on the Client, not the Worker.** Warning thresholds are part of `client.Options`. The Worker option is the opt-out only.
- **Server must report limits for enforcement to engage.** A worker connected to a server that doesn't report payload/memo error limits behaves as if validation were disabled.
- **Don't conflate the 2 MB payload limit with the 4 MB gRPC message limit.** Eager validation covers the per-payload and aggregate memo size against server-reported limits; it does not protect against a Workflow Task whose combined commands exceed the 4 MB gRPC frame. Break large batches of commands into smaller batches.
1 change: 0 additions & 1 deletion references/java/integrations/spring-ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,6 @@ Media image = new Media(MimeTypeUtils.IMAGE_PNG, URI.create("https://cdn.example

For anything larger than a small thumbnail, route the bytes to a binary store from an Activity and pass only the URL across the conversation.


## Vector stores, embeddings, and MCP

When the corresponding Spring AI modules (`spring-ai-rag`, `spring-ai-mcp`) are on the classpath, the integration registers Activities for vector stores, embeddings, and MCP tool calls automatically. Inject the matching Spring AI types into your Activities or Workflows and use them as you would in any Spring AI application — each operation executes through a Temporal Activity.
Expand Down
94 changes: 94 additions & 0 deletions references/python/payload-validation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Python SDK Payload and Memo Size Validation

> [!NOTE]
> This feature is in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview.
>

The Temporal Service rejects payloads above its blob size limit (default 2 MB) and rejects gRPC messages above 4 MB.
Without validation, an oversized payload returned from an Activity or scheduled by a Workflow Task is uploaded to the server and rejected, which historically left workflows stuck or terminated depending on where the payload originated.

The Python SDK validates payload and memo sizes against server-reported error limits **before** sending data to the server. When a size exceeds the limit, the worker fails the current Workflow Task with cause `WORKFLOW_TASK_FAILED_CAUSE_PAYLOADS_TOO_LARGE`; the Workflow Execution stays open so the operator can deploy a fix.

## Default behavior

Validation is on by default starting in `temporalio` 1.23.0.
`Worker(..., disable_payload_error_limit=False, ...)` is the default — no code change is required to get the protection.

The worker fetches the per-namespace error limits from the server. If the server doesn't report limits, the worker doesn't enforce them.

## Warning thresholds — `PayloadLimitsConfig`

Configure warning thresholds on the `DataConverter` to log when payloads or memo size cross a soft limit, separately from the server-enforced error limit:

```python
import dataclasses
from temporalio.client import Client
from temporalio.converter import DataConverter, PayloadLimitsConfig

data_converter = dataclasses.replace(
DataConverter.default,
payload_limits=PayloadLimitsConfig(
payload_size_warning=256 * 1024, # bytes
memo_size_warning=4 * 1024, # bytes
),
)

client = await Client.connect(
"localhost:7233",
data_converter=data_converter,
)
```

Field reference:

- `payload_size_warning: int` — bytes; defaults to `512 * 1024`.
- `memo_size_warning: int` — bytes; defaults to `2 * 1024`.

`PayloadLimitsConfig` is a frozen dataclass.

## Warning class

When a payload or aggregate memo exceeds a warning threshold (but not the server error limit), the SDK issues a `PayloadSizeWarning`, which subclasses `RuntimeWarning`.

Filter or surface it with the standard `warnings` module:

```python
import warnings
from temporalio.converter import PayloadSizeWarning

warnings.simplefilter("always", PayloadSizeWarning)
```

## Disabling enforcement — `Worker(disable_payload_error_limit=...)`

```python
from temporalio.worker import Worker

worker = Worker(
client,
task_queue="my-task-queue",
workflows=[...],
activities=[...],
disable_payload_error_limit=True,
)
```

Set to `True` only when a gRPC proxy between the worker and the server modifies payload bytes after the worker has measured them — for example, a proxy that compresses or re-encodes payloads — so that the worker's pre-upload size measurement no longer reflects what the server will see.

## Effect on errors

When validation rejects a task:

- The worker fails the Workflow Task locally with cause `WORKFLOW_TASK_FAILED_CAUSE_PAYLOADS_TOO_LARGE` and never uploads the oversized data.
- The Workflow Execution remains open; once the cause is resolved (smaller payload, External Storage, etc.), the next Workflow Task succeeds.

## Reducing oversized data

Validation surfaces the problem earlier; it does not raise the limit. To handle data that legitimately exceeds 2 MB, offload to External Storage (claim-check pattern). See `references/python/data-handling.md` for the Payload Converter and Payload Codec, and `docs/develop/python/best-practices/data-handling/external-storage.mdx` for the built-in S3 driver and custom driver interface.

## Gotchas

- **`PayloadLimitsConfig` configures warnings only; error limits come from the server.** Setting `payload_size_warning` smaller than the server's error limit gives early visibility; it does not lower the enforcement threshold.
- **`_PayloadSizeError` is private.** The leading underscore is a stability signal — don't catch it by name. Workflow Task failure is observed through standard Temporal error events.
- **Server must report limits for enforcement to engage.** A worker connected to a server that doesn't report payload/memo error limits behaves as if validation were disabled.
- **Don't conflate the 2 MB payload limit with the 4 MB gRPC message limit.** Eager validation covers per-payload and aggregate memo size against server-reported limits; it does not protect against a Workflow Task whose combined commands exceed the 4 MB gRPC frame. Break large batches of commands into smaller batches.
1 change: 1 addition & 0 deletions references/python/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,7 @@ See `references/python/testing.md` for info on writing tests.
- **`references/python/sync-vs-async.md`** - Sync vs async activities, event loop blocking, executor configuration
- **`references/python/advanced-features.md`** - Schedules, worker tuning, and more
- **`references/python/data-handling.md`** - Data converters, Pydantic, payload encryption
- **`references/python/payload-validation.md`** - Eager payload/memo size validation (Experimental, 1.23.0+): warning thresholds and the worker opt-out
- **`references/python/versioning.md`** - Patching API, workflow type versioning, Worker Versioning
- **`references/python/determinism-protection.md`** - Python sandbox specifics, forbidden operations, pass-through imports
- **`references/python/ai-patterns.md`** - LLM integration, Pydantic data converter, AI workflow patterns
Expand Down