From 413895569cf363b29022ca4280cf853e966fc2f4 Mon Sep 17 00:00:00 2001 From: "skill-sync[bot]" Date: Thu, 14 May 2026 18:54:53 +0000 Subject: [PATCH] Finalize draft for 0013-payload-validation --- references/go/go.md | 1 + references/go/payload-validation.md | 69 +++++++++++++++++ references/java/integrations/spring-ai.md | 1 - references/python/payload-validation.md | 94 +++++++++++++++++++++++ references/python/python.md | 1 + 5 files changed, 165 insertions(+), 1 deletion(-) create mode 100644 references/go/payload-validation.md create mode 100644 references/python/payload-validation.md diff --git a/references/go/go.md b/references/go/go.md index 6c42bed..54a3b38 100644 --- a/references/go/go.md +++ b/references/go/go.md @@ -250,5 +250,6 @@ See `references/go/testing.md` for info on writing tests. - **`references/go/testing.md`** - TestWorkflowEnvironment, time-skipping, activity mocking - **`references/go/advanced-features.md`** - Schedules, worker tuning, and more - **`references/go/data-handling.md`** - Data converters, payload codecs, encryption +- **`references/go/payload-validation.md`** - Eager payload/memo size validation (Experimental, v1.43.0+): warning thresholds and the worker opt-out - **`references/go/versioning.md`** - Patching API (`workflow.GetVersion`), Worker Versioning - **`references/go/determinism-protection.md`** - Information on **`workflowcheck`** tool to help statically check for determinism issues. diff --git a/references/go/payload-validation.md b/references/go/payload-validation.md new file mode 100644 index 0000000..2c38f81 --- /dev/null +++ b/references/go/payload-validation.md @@ -0,0 +1,69 @@ +# Go SDK Payload and Memo Size Validation + +> [!NOTE] +> This feature is in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview. +> + +The Temporal Service rejects payloads above its blob size limit (default 2 MB) and rejects gRPC messages above 4 MB. +Without validation, an oversized payload returned from an Activity or scheduled by a Workflow Task is uploaded to the server, gets rejected, and the workflow may end up stuck in a retry loop or terminated depending on where the payload originated. + +The Go SDK validates payload and memo sizes against server-reported error limits **before** sending data to the server. When a size exceeds the limit, the worker fails the current Task locally — the Workflow Execution stays open so the operator can deploy a fix. + +## Default behavior + +`DisablePayloadErrorLimit` defaults to `false`, so validation is on by default. +The worker fetches the per-namespace error limits from the server. If the server doesn't report limits, the worker doesn't enforce them. + +No code change is required to get the default protection — just upgrade the SDK to v1.43.0 or later. + +## Warning thresholds — `client.PayloadLimitOptions` + +Set warning thresholds on the Client to emit a log when individual payloads or aggregated memo size cross a soft limit, separately from the server-enforced error limit: + +```go +import "go.temporal.io/sdk/client" + +c, err := client.Dial(client.Options{ + PayloadLimits: client.PayloadLimitOptions{ + PayloadSizeWarning: 256 * 1024, // bytes + MemoSizeWarning: 4 * 1024, // bytes + }, +}) +``` + +Field reference: + +- `PayloadSizeWarning int` — bytes; zero/unset defaults to 512 KiB. +- `MemoSizeWarning int` — bytes; zero/unset defaults to 512 KiB. + +Warnings are logged via the configured logger; they do not fail tasks. + +## Disabling enforcement — `worker.Options.DisablePayloadErrorLimit` + +```go +import "go.temporal.io/sdk/worker" + +w := worker.New(c, "task-queue", worker.Options{ + DisablePayloadErrorLimit: true, +}) +``` + +Set to `true` only when a gRPC proxy between the worker and the server modifies payload bytes after the worker has measured them — for example, a proxy that compresses or re-encodes payloads — so that the worker's pre-upload size measurement no longer reflects what the server will see. + +## Effect on errors + +When validation rejects a task: + +- The worker fails the Workflow Task or Activity Task locally and never uploads the oversized data. +- The Workflow Execution remains open; the task is eligible for retry after the user reduces payload size, switches to External Storage, or otherwise resolves the cause. +- Without validation, oversized inputs cause `WORKFLOW_TASK_FAILED_CAUSE_PAYLOADS_TOO_LARGE`; oversized Workflow results historically caused a server-side retry loop. + +## Reducing oversized data + +Validation surfaces the problem earlier; it does not raise the limit. To handle data that legitimately exceeds 2 MB, offload to External Storage (claim-check pattern). See `references/go/data-handling.md` for the Payload Converter and Payload Codec, and `docs/develop/go/best-practices/data-handling/external-storage.mdx` for the built-in S3 driver and custom driver interface. + +## Gotchas + +- **Set `PayloadLimits` on the Client, not the Worker.** Warning thresholds are part of `client.Options`. The Worker option is the opt-out only. +- **Server must report limits for enforcement to engage.** A worker connected to a server that doesn't report payload/memo error limits behaves as if validation were disabled. +- **Don't conflate the 2 MB payload limit with the 4 MB gRPC message limit.** Eager validation covers the per-payload and aggregate memo size against server-reported limits; it does not protect against a Workflow Task whose combined commands exceed the 4 MB gRPC frame. Break large batches of commands into smaller batches. diff --git a/references/java/integrations/spring-ai.md b/references/java/integrations/spring-ai.md index 5ee0704..ae5154f 100644 --- a/references/java/integrations/spring-ai.md +++ b/references/java/integrations/spring-ai.md @@ -217,7 +217,6 @@ Media image = new Media(MimeTypeUtils.IMAGE_PNG, URI.create("https://cdn.example For anything larger than a small thumbnail, route the bytes to a binary store from an Activity and pass only the URL across the conversation. - ## Vector stores, embeddings, and MCP When the corresponding Spring AI modules (`spring-ai-rag`, `spring-ai-mcp`) are on the classpath, the integration registers Activities for vector stores, embeddings, and MCP tool calls automatically. Inject the matching Spring AI types into your Activities or Workflows and use them as you would in any Spring AI application — each operation executes through a Temporal Activity. diff --git a/references/python/payload-validation.md b/references/python/payload-validation.md new file mode 100644 index 0000000..13e50fe --- /dev/null +++ b/references/python/payload-validation.md @@ -0,0 +1,94 @@ +# Python SDK Payload and Memo Size Validation + +> [!NOTE] +> This feature is in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview. +> + +The Temporal Service rejects payloads above its blob size limit (default 2 MB) and rejects gRPC messages above 4 MB. +Without validation, an oversized payload returned from an Activity or scheduled by a Workflow Task is uploaded to the server and rejected, which historically left workflows stuck or terminated depending on where the payload originated. + +The Python SDK validates payload and memo sizes against server-reported error limits **before** sending data to the server. When a size exceeds the limit, the worker fails the current Workflow Task with cause `WORKFLOW_TASK_FAILED_CAUSE_PAYLOADS_TOO_LARGE`; the Workflow Execution stays open so the operator can deploy a fix. + +## Default behavior + +Validation is on by default starting in `temporalio` 1.23.0. +`Worker(..., disable_payload_error_limit=False, ...)` is the default — no code change is required to get the protection. + +The worker fetches the per-namespace error limits from the server. If the server doesn't report limits, the worker doesn't enforce them. + +## Warning thresholds — `PayloadLimitsConfig` + +Configure warning thresholds on the `DataConverter` to log when payloads or memo size cross a soft limit, separately from the server-enforced error limit: + +```python +import dataclasses +from temporalio.client import Client +from temporalio.converter import DataConverter, PayloadLimitsConfig + +data_converter = dataclasses.replace( + DataConverter.default, + payload_limits=PayloadLimitsConfig( + payload_size_warning=256 * 1024, # bytes + memo_size_warning=4 * 1024, # bytes + ), +) + +client = await Client.connect( + "localhost:7233", + data_converter=data_converter, +) +``` + +Field reference: + +- `payload_size_warning: int` — bytes; defaults to `512 * 1024`. +- `memo_size_warning: int` — bytes; defaults to `2 * 1024`. + +`PayloadLimitsConfig` is a frozen dataclass. + +## Warning class + +When a payload or aggregate memo exceeds a warning threshold (but not the server error limit), the SDK issues a `PayloadSizeWarning`, which subclasses `RuntimeWarning`. + +Filter or surface it with the standard `warnings` module: + +```python +import warnings +from temporalio.converter import PayloadSizeWarning + +warnings.simplefilter("always", PayloadSizeWarning) +``` + +## Disabling enforcement — `Worker(disable_payload_error_limit=...)` + +```python +from temporalio.worker import Worker + +worker = Worker( + client, + task_queue="my-task-queue", + workflows=[...], + activities=[...], + disable_payload_error_limit=True, +) +``` + +Set to `True` only when a gRPC proxy between the worker and the server modifies payload bytes after the worker has measured them — for example, a proxy that compresses or re-encodes payloads — so that the worker's pre-upload size measurement no longer reflects what the server will see. + +## Effect on errors + +When validation rejects a task: + +- The worker fails the Workflow Task locally with cause `WORKFLOW_TASK_FAILED_CAUSE_PAYLOADS_TOO_LARGE` and never uploads the oversized data. +- The Workflow Execution remains open; once the cause is resolved (smaller payload, External Storage, etc.), the next Workflow Task succeeds. + +## Reducing oversized data + +Validation surfaces the problem earlier; it does not raise the limit. To handle data that legitimately exceeds 2 MB, offload to External Storage (claim-check pattern). See `references/python/data-handling.md` for the Payload Converter and Payload Codec, and `docs/develop/python/best-practices/data-handling/external-storage.mdx` for the built-in S3 driver and custom driver interface. + +## Gotchas + +- **`PayloadLimitsConfig` configures warnings only; error limits come from the server.** Setting `payload_size_warning` smaller than the server's error limit gives early visibility; it does not lower the enforcement threshold. +- **`_PayloadSizeError` is private.** The leading underscore is a stability signal — don't catch it by name. Workflow Task failure is observed through standard Temporal error events. +- **Server must report limits for enforcement to engage.** A worker connected to a server that doesn't report payload/memo error limits behaves as if validation were disabled. +- **Don't conflate the 2 MB payload limit with the 4 MB gRPC message limit.** Eager validation covers per-payload and aggregate memo size against server-reported limits; it does not protect against a Workflow Task whose combined commands exceed the 4 MB gRPC frame. Break large batches of commands into smaller batches. diff --git a/references/python/python.md b/references/python/python.md index 5493387..a125716 100644 --- a/references/python/python.md +++ b/references/python/python.md @@ -179,6 +179,7 @@ See `references/python/testing.md` for info on writing tests. - **`references/python/sync-vs-async.md`** - Sync vs async activities, event loop blocking, executor configuration - **`references/python/advanced-features.md`** - Schedules, worker tuning, and more - **`references/python/data-handling.md`** - Data converters, Pydantic, payload encryption +- **`references/python/payload-validation.md`** - Eager payload/memo size validation (Experimental, 1.23.0+): warning thresholds and the worker opt-out - **`references/python/versioning.md`** - Patching API, workflow type versioning, Worker Versioning - **`references/python/determinism-protection.md`** - Python sandbox specifics, forbidden operations, pass-through imports - **`references/python/ai-patterns.md`** - LLM integration, Pydantic data converter, AI workflow patterns