Skip to content

feat: add Tavily search provider to core search utility#1056

Open
tavily-integrations wants to merge 1 commit intoScrapeGraphAI:mainfrom
Tavily-FDE:feat/tavily-migration/core-search-utility
Open

feat: add Tavily search provider to core search utility#1056
tavily-integrations wants to merge 1 commit intoScrapeGraphAI:mainfrom
Tavily-FDE:feat/tavily-migration/core-search-utility

Conversation

@tavily-integrations
Copy link
Copy Markdown

Summary

  • Added Tavily as a configurable search engine option in research_web.py, alongside existing providers (DuckDuckGo, Bing, SearXNG, Serper)
  • This is an additive change — all existing providers remain untouched
  • Tavily API key can be passed explicitly via tavily_api_key parameter or falls back to TAVILY_API_KEY environment variable

Changes

scrapegraphai/utils/research_web.py

  • Added os import for environment variable fallback
  • Added tavily_api_key field to SearchConfig Pydantic model
  • Added 'tavily' to the valid_engines set in validate_search_engine
  • Added tavily_api_key parameter to search_on_web() function signature and docstring
  • Added elif config.search_engine == "tavily" dispatch branch in search_on_web()
  • Implemented _search_tavily() helper function using TavilyClient from tavily-python

pyproject.toml

  • Added tavily-python>=0.3.0 to project dependencies

Dependency changes

  • Added tavily-python>=0.3.0 to pyproject.toml

Environment variable changes

  • Added TAVILY_API_KEY — used as fallback when tavily_api_key is not explicitly passed

Notes for reviewers

  • The tavily-python import is done lazily inside _search_tavily() to avoid import errors if the package isn't installed, with a clear error message guiding installation
  • Uses search_depth="basic" for cost efficiency (1 credit per call); can be changed to "advanced" for higher relevance

🤖 Generated with Claude Code

Automated Review

  • Passed after 1 attempt(s)
  • Final review: The Tavily migration for the core-search-utility unit is well-implemented and correct. The changes follow the existing patterns for other search providers, the SDK usage is valid, API key fallback via environment variable is properly handled, and all required files (research_web.py and pyproject.toml) are updated. Two minor issues were found but neither blocks approval.

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. dependencies Pull requests that update a dependency file enhancement New feature or request labels Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file enhancement New feature or request size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant