fix(search): restore SearchGraph by migrating DuckDuckGo backend to ddgs

[VinciGit00]

Summary

Fixes #1082 — SearchGraph (the "Search Scraper") stopped returning any results after a recent update, even for simple queries like NVIDIA news, while other modules kept working.

Root cause

The upstream duckduckgo-search package was renamed to ddgs. Recent langchain-community releases (we resolve 0.4.1) now do from ddgs import DDGS inside their DuckDuckGoSearchResults tool. Since the project still depended on the old duckduckgo-search package, instantiating that tool in research_web.search_on_web raised an ImportError, so the default DuckDuckGo search path returned nothing and SearchGraph produced no answer.

ImportError: Could not import ddgs python package. Please install it with `pip install -U ddgs`.

Changes

• scrapegraphai/utils/research_web.py — replace the langchain-community DuckDuckGoSearchResults wrapper with a direct ddgs.DDGS().text() call in a new _search_duckduckgo helper. URLs are now read from the structured href field instead of regex-parsing a formatted string, and proxy is passed through. A clear ImportError message is raised if ddgs is missing.
• pyproject.toml — swap duckduckgo-search>=8.1.1 for ddgs>=9.0.0.
• uv.lock — regenerated. Beyond the dependency swap, uv also pruned stale lock entries (Sphinx/docs deps that were already removed from pyproject.toml in an earlier commit and their transitives); uv lock --check confirms the lockfile is consistent with pyproject.toml.

Testing

• search_on_web(query="NVIDIA news", search_engine="duckduckgo") now returns URLs (previously ImportError/empty).
• Full end-to-end SearchGraph("NVIDIA news", ...) with an OpenAI model returns considered URLs and a structured answer.
• tests/test_search_graph.py passes. (The pre-existing test_omni_search_graph.py failures are unrelated — KeyError: 'model_provider' from the dummy-model config not using the required provider/model format.)

🤖 Generated with Claude Code

[github-actions]

Dependency Review

The following issues were found:

• ✅ 0 vulnerable package(s)
• ✅ 0 package(s) with incompatible licenses
• ✅ 0 package(s) with invalid SPDX license definitions
• ⚠️ 3 package(s) with unknown licenses.

See the Details below.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 8426cd1.

Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

License Issues

uv.lock

Package	Version	License	Issue Type
ddgs	9.14.4	Null	Unknown License
primp	1.3.1	Null	Unknown License
scrapegraph-py	2.1.0	Null	Unknown License

OpenSSF Scorecard

Scorecard details

Package	Version	Score	Details
pip/brotli	1.2.0	🟢 8	Details Check Score Reason Token-Permissions ⚠️ -1 No tokens found Packaging ⚠️ -1 packaging workflow not detected Dangerous-Workflow ⚠️ -1 no workflows found Code-Review 🟢 5 Found 7/12 approved changesets -- score normalized to 5 Security-Policy 🟢 10 security policy file detected Maintained 🟢 10 30 commit(s) and 9 issue activity found in the last 90 days -- score normalized to 10 Pinned-Dependencies ⚠️ -1 no dependencies found Binary-Artifacts 🟢 10 no binaries found in the repo CII-Best-Practices ⚠️ 2 badge detected: InProgress License 🟢 10 license file detected Fuzzing 🟢 10 project is fuzzed Vulnerabilities 🟢 10 0 existing vulnerabilities detected Signed-Releases ⚠️ 0 Project has not signed or included provenance with any releases. Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md SAST 🟢 8 SAST tool is not run on all commits -- score normalized to 8 Dependency-Update-Tool 🟢 10 update tool detected CI-Tests 🟢 10 14 out of 14 merged PRs checked by a CI test -- score normalized to 10 Contributors 🟢 10 project has 17 contributing companies or organizations
pip/brotlicffi	1.2.0.1	🟢 5.2	Details Check Score Reason Binary-Artifacts 🟢 10 no binaries found in the repo Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Maintained 🟢 3 4 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 3 Code-Review 🟢 6 Found 16/24 approved changesets -- score normalized to 6 Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Security-Policy 🟢 10 security policy file detected Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected Fuzzing ⚠️ 0 project is not fuzzed License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md Packaging 🟢 10 packaging workflow detected SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/cffi	2.0.0	Unknown	Unknown
pip/ddgs	9.14.4	Unknown	Unknown
pip/fake-useragent	2.2.0	Unknown	Unknown
pip/h2	4.3.0	Unknown	Unknown
pip/hpack	4.1.0	Unknown	Unknown
pip/hyperframe	6.1.0	Unknown	Unknown
pip/primp	1.3.1	Unknown	Unknown
pip/pycparser	3.0	Unknown	Unknown
pip/scrapegraph-py	2.1.0	Unknown	Unknown
pip/socksio	1.0.0	🟢 4	Details Check Score Reason Packaging ⚠️ -1 packaging workflow not detected Maintained ⚠️ 0 0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0 Code-Review 🟢 8 Found 25/30 approved changesets -- score normalized to 8 Binary-Artifacts 🟢 10 no binaries found in the repo Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected Security-Policy ⚠️ 0 security policy file not detected Fuzzing ⚠️ 0 project is not fuzzed License 🟢 10 license file detected Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Signed-Releases ⚠️ -1 no releases found Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0

Scanned Files

• uv.lock

[github-actions]

🎉 This PR is included in version 2.1.3 🎉

The release is available on:

• v2.1.3
• GitHub release

Your semantic-release bot 📦🚀