Skip to content

add image search support in agents sdk#3474

Open
yuning-oai wants to merge 1 commit into
mainfrom
dev/yuning/image-search-web-search-tool
Open

add image search support in agents sdk#3474
yuning-oai wants to merge 1 commit into
mainfrom
dev/yuning/image-search-web-search-tool

Conversation

@yuning-oai
Copy link
Copy Markdown

@yuning-oai yuning-oai commented May 20, 2026

This pull request adds Agent SDK support for Responses API image search through the existing WebSearchTool.

Summary

  • Add optional search_content_types to WebSearchTool while preserving existing default payloads.
  • Forward search_content_types to Responses API tool payloads.
  • Automatically include web_search_call.results whenever image results are requested.
  • Add converter tests for omitted, mixed text/image, and image-only content types.

Test plan

  • Converter tests pass for the changed OpenAI Responses conversion path, including mixed text/image and image-only search content types.
  • Ruff lint and formatting checks pass for the touched files.
  • Pyright passes for the touched files.

Full repo verification was attempted with .agents/skills/code-change-verification/scripts/run.sh; it passed formatting and linting, then failed on missing optional extras/imports in the temporary clone environment (Runloop, voice/numpy, litellm, SQLAlchemy), not on this converter change.

@yuning-oai yuning-oai changed the title feat: add image search content types to WebSearchTool add image search support in agents dk May 20, 2026
@yuning-oai yuning-oai changed the title add image search support in agents dk add image search support in agents sdk May 20, 2026
@yuning-oai yuning-oai force-pushed the dev/yuning/image-search-web-search-tool branch from 3462d7b to 1bf3338 Compare May 20, 2026 17:41
@yuning-oai yuning-oai requested a review from qiyaoq-oai May 20, 2026 17:51
@yuning-oai yuning-oai marked this pull request as ready for review May 20, 2026 17:51
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1bf3338896

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +2024 to +2025
if tool.search_content_types is not None:
web_search_tool["search_content_types"] = list(tool.search_content_types)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the preview web-search tool for content types

When WebSearchTool(search_content_types=...) is used, this still serializes the tool as type: "web_search" and adds search_content_types to that GA payload. I checked the generated OpenAI Responses types: search_content_types is defined on WebSearchPreviewToolParam (web_search_preview...), not on the GA WebSearchToolParam, so callers exercising the new option send a tool shape the API rejects instead of enabling image search.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yuning-oai Can you resolve all the Codex review comments? This repo runs reviews per push.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. The backend accepts search_content_types on web_search, but the field is intentionally still hidden from public OpenAPI for now, so openai-python does not expose it on WebSearchToolParam yet.

I’m going to keep the Agents SDK payload as web_search rather than switch to preview; this is relying on the Responses API hidden field until we’re ready to expose it publicly.

Copy link
Copy Markdown
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once all Codex reviews are cleared, we can merge this

@seratch seratch added this to the 0.17.x milestone May 20, 2026
@yuning-oai
Copy link
Copy Markdown
Author

Once all Codex reviews are cleared, we can merge this

Thanks! shall we merge now or wait for the GA to happen first?

@seratch
Copy link
Copy Markdown
Member

seratch commented May 21, 2026

We can merge at the timing of either public beta or GA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants