Skip to content

Implement planned topic: 0041-pydantic-ai#249

Open
skill-temporal-developer-updater[bot] wants to merge 1 commit into
mainfrom
draft/0041-pydantic-ai
Open

Implement planned topic: 0041-pydantic-ai#249
skill-temporal-developer-updater[bot] wants to merge 1 commit into
mainfrom
draft/0041-pydantic-ai

Conversation

@skill-temporal-developer-updater
Copy link
Copy Markdown
Contributor

Validation Report — Pydantic AI integration

Skill: temporal-developer
Topic: Pydantic AI Temporal integration (Python)
Files under validation:

  • references/python/integrations/pydantic-ai.md (new, 259 lines)
  • references/integrations.md (one new catalog row)

Source of truth used:

  • Upstream Pydantic AI guide — /tmp/upstream_temporal.md (mirrored from https://raw.githubusercontent.com/pydantic/pydantic-ai/main/docs/durable_execution/temporal.md)
  • Upstream Pydantic AI API reference — https://pydantic.dev/docs/ai/api/pydantic-ai/durable_exec/
  • Upstream pydantic_ai_slim/pyproject.toml
  • Upstream umbrella pyproject.toml
  • Upstream plugin source — pydantic_ai_slim/pydantic_ai/durable_exec/temporal/__init__.py
  • PyPI metadata for pydantic-ai (provides_extra, requires_dist)
  • Temporal Python SDK installed locally for signature inspection (temporalio.client.Client.connect)
  • Temporal docs (../documentation/docs/) for Temporal-side identifier resolution

Go / no-go

Check Result
Check 1 — citation audit FAIL (21/22 = 95.5%, below 98% threshold)
Check 2 — reverse-grep / fabrication PASS (all tokens resolve to upstream Pydantic AI or Temporal docs)
Check 3 — regression patterns PASS (zero hits)
Check 4 — independent re-verification FAIL (~14/16 = 87.5%, below 95% threshold; 2 substantively wrong claims)
Check 5 — integration layout PASS (all six sub-checks)
Check 6 — tone and scope PASS (no Pattern 1 workaround disclosure; one Pattern 9 + one minor Pattern 10)
Check 7 — cross-language structure SKIPPED (Python-only integration)

Overall verdict: RE-RUN AUTHORING

Triggered by Check 4 falling below 95%. Two of the failing claims are factually wrong in ways that an AI agent reader would propagate into user code: the umbrella install extra, and the "register plugin on both Client and Worker" instruction. These need to be re-grounded — they are not spot-fix-style typos. Check 6 cleanup (de-duplication of Hard Constraints vs Common Mistakes) can ride along but is independently a MINOR FIX.


Check 1 findings (citation audit)

22 inline citation tags inspected. 21 resolve cleanly. The single failure:

Finding 1.1 — pydantic-ai.md:21

  • Claim: The umbrella \pydantic-ai[temporal]` also works if you want the full Pydantic AI install.`
  • Cited source: <!-- pydantic-ai: docs/ai/overview/install/ -->
  • Problem: The cited install page does not document a [temporal] extra on the umbrella pydantic-ai distribution. PyPI metadata confirms provides_extra for pydantic-ai==1.106.0 does not include temporal (the list is a2a, dbos, examples, outlines-*, prefect, sentence-transformers, voyageai). A pip install "pydantic-ai[temporal]" dry-run produces:
    WARNING: pydantic-ai 1.106.0 does not provide the extra 'temporal'
    Temporal support reaches the user only because the umbrella's hardcoded base dependency line is pydantic-ai-slim[..., temporal, ...]==1.106.0 — the [temporal] suffix on the umbrella is silently ignored.
  • What the page actually supports: umbrella pip install pydantic-ai (no extra) works because the slim-with-temporal is pulled in transitively. The skill could correctly cite the install page for that claim, but not the pydantic-ai[temporal] form.

All hyperlinks are fully-qualified https:// URLs. The example-repo link https://github.com/pydantic/pydantic-ai-temporal-example returns HTTP 200.

Statistics: 22 citations, 21 supported, 1 unsupported → 95.5% (threshold 98%).


Check 2 findings (reverse-grep / fabrication)

Every factual token in the authored file resolves to either upstream Pydantic AI documentation/source or Temporal documentation. Spot examples:

  • Upstream Pydantic AI identifiers (TemporalAgent, PydanticAIPlugin, AgentPlugin, PydanticAIWorkflow, LogfirePlugin, TemporalRunContext, __pydantic_ai_agents__, provider_factory, event_stream_handler, model_activity_config, toolset_activity_config, tool_activity_config, run_context_type, pydantic_ai.durable_exec.temporal, pydantic_ai.exceptions.UserError) — all confirmed in upstream temporal.md, the API reference, or the plugin source.
  • Temporal-side identifiers (pydantic_data_converter, workflow.defn, workflow.run, workflow.unsafe.imports_passed_through, start_to_close_timeout) — all resolve in ../documentation/docs/.
  • The 13-field RunContext list (L144) matches upstream temporal.md:172 verbatim. (Upstream API reference lists three additional fields — loaded_capability_ids, discovered_tool_names, capability_loaded — but that is an upstream doc inconsistency, not a skill fabrication; the skill cites the guide, not the API page, for this list.)
  • temporalio>=1.24.0 version pin (L19) — confirmed verbatim in pydantic_ai_slim/pyproject.toml.

No fabrications found.


Check 3 findings (regression patterns)

Universal patterns (none present):

  • --profile — absent.
  • TEMPORAL_TLS_CLIENT_CERT_PATH / _KEY_PATH / _CA_CERT_PATH — absent.
  • tcld service-account — absent.
  • --output text / --output jsonl — absent.
  • saas-api.tmprl.cloud:7233 — absent.

Topic-specific:

  • temporalio.contrib.pydantic_ai — appears only at pydantic-ai.md:245 inside the Common Mistakes section, explicitly flagging it as the wrong path. Not a regression hit.
  • temporalio[pydantic-ai] extra — absent.

Zero regression-pattern hits.


Check 4 findings (independent re-verification)

Sample of 10 claims drawn evenly across the file, plus 6 spot checks. Two substantive mismatches.

Finding 4.1 — pydantic-ai.md:21 (also see Finding 1.1)

  • Authored: "The umbrella pydantic-ai[temporal] also works if you want the full Pydantic AI install."
  • Reality (PyPI + dry-run): pydantic-ai does not declare a temporal extra. pip install "pydantic-ai[temporal]" triggers WARNING: pydantic-ai 1.106.0 does not provide the extra 'temporal'; the install succeeds only because the umbrella's base dependency line already pulls in pydantic-ai-slim[..., temporal, ...].
  • Correct form: pip install pydantic-ai (no extra) installs temporalio, because the umbrella distribution already enumerates temporal inside its base pydantic-ai-slim[...] dependency. Alternatively, the slim form pip install "pydantic-ai-slim[temporal]" (which the file recommends earlier at L16) is the documented path.
  • Impact: An AI agent following this guidance writes pip install "pydantic-ai[temporal]" into a project's requirements.txt / install instructions and the user sees a pip warning. Mild reader confusion, easy fix.

Finding 4.2 — pydantic-ai.md:90, :237, and Common-Mistakes adjacency

  • Authored:
    • L90: "Pass PydanticAIPlugin() to both the Temporal Client and Worker"
    • L237 (hard constraint): "Register PydanticAIPlugin on both Client and Worker. The Client side configures pydantic_data_converter; the Worker side registers the dynamically created Activities."
  • Reality (Temporal Python SDK docstring on Client.connect, verified by inspect.getsource):

    "plugins: ... Any plugins that also implement temporalio.worker.Plugin will be used as worker plugins too so they should not be given when creating a worker."

  • Upstream Pydantic AI example confirms: the plugin is passed to Client.connect(plugins=[PydanticAIPlugin()]) only; Worker(client, task_queue=..., workflows=[...]) is constructed with no plugins= argument. The Worker inherits the plugin from the Client.
  • The file's own example code (L97-107) matches upstream — it only passes the plugin to Client.connect. So the prose at L90 and the hard constraint at L237 contradict both the upstream SDK guidance and the file's own example.
  • Impact: Following the prose, an agent would emit Worker(client, plugins=[PydanticAIPlugin()], ...). Per the Temporal SDK docstring, this is the wrong shape: plugins implementing both Client and Worker plugin protocols should be registered on the Client only. Worst-case this re-runs configuration hooks twice (duplicate activity registration, duplicate workflow_runner replacement) and produces sandbox / activity errors that are hard to diagnose. The file's example code does not exhibit the bug, but its constraint list and prose tell the reader to introduce it.

Spot checks that all matched: L5 wrapper-overview claim, L19 temporalio>=1.24.0, L42 / L44 name/id mandatory-and-stable, L86 60s default timeout, L112 PydanticAIPlugin's three responsibilities, L142 deps must be Pydantic-serializable, L144 RunContext field list (matches upstream guide verbatim), L150 streaming unsupported set, L114 don't double-wire data_converter, L86 per-tier overrides, L122 __pydantic_ai_agents__ pattern, L66 module-scope construction, L156-158 LogfirePlugin alongside on Client, L169 pandas import-race workaround.

Statistics: sample size 16 (10 primary + 6 spot checks). Matches: 14. Mismatches: 2. Match rate: 87.5% (threshold 95%).

Note on severity: both mismatches were caught only by going beyond temporal.md to the Temporal SDK's own docstring and PyPI metadata. The authored claims are internally plausible because Pydantic AI's PydanticAIPlugin source code is described as a "Temporal client and worker plugin" — but the Temporal SDK's plugin-propagation contract makes registering on both an anti-pattern. This is precisely the subtle-wrong shape Check 4 exists to catch.


Check 5 findings (integration layout)

All six sub-checks pass:

  1. File locationreferences/python/integrations/pydantic-ai.md matches the catalog convention. ✅
  2. Catalog row — exactly one row added to references/integrations.md for Pydantic AI, with name, language, description, reference path, and Related column. ✅
  3. Catalog summary terse — the description column is a single clause-chain consistent in length with existing peers (Google ADK, OpenAI Agents SDK). ✅
  4. No SKILL.md editgit diff HEAD -- SKILL.md is empty. ✅
  5. No per-integration line in references/python/python.mdgit diff HEAD -- references/python/python.md is empty. ✅
  6. Inline cross-link cap — no edits to ai-patterns.md (Python or core). The cross-references in pydantic-ai.md link outward to those files; that's expected for the new page. ✅

Check 6 findings (tone and scope)

No Pattern 1 (workaround-disclosure) findings. The streaming section uses the supported "Don't do X — use Y" form: "Agent.run_stream(), Agent.run_stream_events(), and Agent.iter() are not supported... The supported alternative is event_stream_handler..." This is good guidance, not a cookie-jar disclosure.

Finding 6.1 — Pattern 9 (constraints duplicated outside the canonical section)

  • The Hard Constraints section (L232–241) lists 7 items.
  • The Common Mistakes section (L243–250) re-states 5 of those 7 in different phrasing:
    • "Forgetting name= on the Agent" duplicates the L234 Agent.name constraint.
    • "Forgetting id= on @agent.toolset" duplicates the L235 toolset-id constraint.
    • "Importing from temporalio.contrib.pydantic_ai" duplicates the L236 ships-in-pydantic_ai.durable_exec.temporal constraint.
    • "Calling Agent.run_stream" duplicates the L239 streaming constraint.
    • "Passing a raw Model instance" duplicates the L238 Model-instances constraint.
  • The two genuinely-new common-mistake items are (a) "Setting data_converter=pydantic_data_converter in addition to PydanticAIPlugin()" and (b) the @agent.toolset framing (which adds the wrap-time-raise detail not in the hard constraint).
  • Recommendation: keep Hard Constraints as the authoritative list; in Common Mistakes, retain only items that add new actionable detail (the data-converter double-wiring, the wrap-time raise behavior). Drop the others.

Finding 6.2 — Pattern 10 (Resources pointing at a source tree, minor)

  • L259 links to the pydantic/pydantic-ai-temporal-example GitHub repo. Borderline acceptable since it is the upstream-blessed example, but per the pattern, prefer linking to a focused docs page when one exists. Low-priority.

Acceptable patterns observed (no findings):

  • L114 ("Do not also set data_converter=... — the plugin owns that wiring") — terse don't-do-X form.
  • The hard-constraint bullets at L234–241 are mostly single-sentence boundary statements; pattern 4 (multi-sentence bullet bloat) does not apply.
  • No "Public Preview" admonition is added, correctly — the upstream Pydantic AI integration carries no preview marker.
  • No DIY-rebuild bullets, no in-tree slang, no inline imports inside example function bodies (the with workflow.unsafe.imports_passed_through(): import pandas block is an upstream-prescribed pattern, not the imports-inside-function anti-pattern).

Check 7 findings (cross-language structure)

Skipped — Pydantic AI integration is Python-only; no references/core/<feature>.md page is expected.


Statistics

  • Inline citation tags inspected: 22 (21 supported → 95.5%)
  • Hyperlinks inspected: 4 (all fully-qualified https://, all reachable)
  • Token classes scanned for fabrication: Python identifiers (Pydantic AI + Temporal), constructor parameters, RunContext fields, version pins → zero unexplained misses
  • Regression patterns checked: 9 (7 universal + 2 topic-specific) → zero hits
  • Check 4 sample size: 16 (10 primary + 6 spot checks) → 14 match → 87.5%
  • Layout sub-checks (Check 5): 6 of 6 pass

Summary of required follow-up

  1. (blocking) Re-author the install advice (L21). Replace pydantic-ai[temporal] with either the no-extra umbrella form pip install pydantic-ai or simply drop the umbrella mention and rely on the slim form already given at L16.
  2. (blocking) Re-author the plugin-registration prose and hard constraint (L90, L237). The plugin is registered on Client.connect(plugins=[...]) only; the Worker inherits it via the Client, per the Temporal Python SDK Client.connect docstring. The file's own example code is already correct — only the prose / constraint list need to be brought in line. Also reconcile the "Common mistake" framing so it does not contradict this.
  3. (minor) Cleanup Hard Constraints vs Common Mistakes. Drop the duplicated bullets in Common Mistakes (Finding 6.1).
  4. (minor) Consider replacing the pydantic-ai-temporal-example repo link with a focused upstream doc page if one exists, otherwise leave as-is (Finding 6.2).

Items 1 and 2 jointly justify the RE-RUN AUTHORING verdict. Items 3 and 4 can be folded into a single cleanup commit alongside the re-authoring pass.

@skill-temporal-developer-updater skill-temporal-developer-updater Bot requested a review from a team as a code owner June 5, 2026 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants