[codex] Add browser humanization controls#41
Conversation
📝 WalkthroughWalkthroughThis PR introduces browser window size humanization with user-facing CLI control. A new ChangesBrowser Humanization and Window Sizing
🎯 3 (Moderate) | ⏱️ ~25 minutes
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR introduces configurable “humanization” controls for browser-based scraping sessions, including weighted random desktop window sizing (with stable sizing for persistent profile directories) and an opt-in toggle for CloakBrowser’s humanized input behaviors. It threads these options through the CLI and parallel place-scraping workers, and updates documentation plus targeted tests to validate the new forwarding/launch behavior.
Changes:
- Add
BrowserSessionConfig.window_size(default"random") andBrowserSessionConfig.human_mouse(defaultFalse), and apply them when launching CloakBrowser contexts. - Add CLI flags
--human-mouseand--disable-random-window-size, forwarding them intoBrowserSessionConfig. - Propagate browser session options into parallel worker sessions and add focused unit coverage + docs updates.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
src/gmaps_scraper/browser_humanization.py |
New helper module to resolve weighted-random vs stable window sizes (and validate explicit sizes). |
src/gmaps_scraper/scraper.py |
Extends BrowserSessionConfig and applies resolved window sizing + humanization when launching CloakBrowser contexts. |
src/gmaps_scraper/cli.py |
Adds/forwards --human-mouse and --disable-random-window-size into BrowserSessionConfig. |
src/gmaps_scraper/place_scraper.py |
Ensures parallel worker sessions inherit window_size and human_mouse. |
tests/test_scraper.py |
Updates launch-context expectations and stabilizes window-size behavior via patching. |
tests/test_place_scraper.py |
Verifies worker session propagation of human_mouse and window_size. |
tests/test_cli.py |
Adds coverage ensuring CLI flags are forwarded into BrowserSessionConfig. |
README.md |
Updates example to demonstrate human_mouse=True. |
docs/USAGE.md |
Documents default random sizing behavior and the new CLI flags. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tests/test_cli.py (1)
213-247: 🏗️ Heavy liftStrengthen test by covering individual flag behavior and defaults.
The test verifies that both flags together produce the expected session configuration, but doesn't test individual flags or default behavior. This test would pass even if individual flags were broken, as long as the combination worked.
Consider adding test cases for:
--human-mousealone (should yieldhuman_mouse=True, window_size="random")--disable-random-window-sizealone (should yieldwindow_size=None, human_mouse=False)- Neither flag present (should yield default
window_size="random", human_mouse=False)As per coding guidelines, tests should encode intent and would be stronger if they caught single-flag regressions independently.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_cli.py` around lines 213 - 247, Add separate unit cases alongside test_forwards_browser_humanization_flags to verify each flag and the default: call main() with only "--human-mouse" and assert collect_saved_list_result was called with BrowserSessionConfig(window_size="random", human_mouse=True); call main() with only "--disable-random-window-size" and assert BrowserSessionConfig(window_size=None, human_mouse=False); and call main() with neither flag and assert BrowserSessionConfig(window_size="random", human_mouse=False). Use the same pattern of patching sys.argv, gmaps_scraper.cli.collect_saved_list_result, and redirect_stdout as in the existing test to assert exit_code, payload output, and the collect_saved_list_result call arguments.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tests/test_cli.py`:
- Around line 213-247: Add separate unit cases alongside
test_forwards_browser_humanization_flags to verify each flag and the default:
call main() with only "--human-mouse" and assert collect_saved_list_result was
called with BrowserSessionConfig(window_size="random", human_mouse=True); call
main() with only "--disable-random-window-size" and assert
BrowserSessionConfig(window_size=None, human_mouse=False); and call main() with
neither flag and assert BrowserSessionConfig(window_size="random",
human_mouse=False). Use the same pattern of patching sys.argv,
gmaps_scraper.cli.collect_saved_list_result, and redirect_stdout as in the
existing test to assert exit_code, payload output, and the
collect_saved_list_result call arguments.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 2b9c10e7-1714-47b4-abb7-61b6d2afa79b
📒 Files selected for processing (9)
README.mddocs/USAGE.mdsrc/gmaps_scraper/browser_humanization.pysrc/gmaps_scraper/cli.pysrc/gmaps_scraper/place_scraper.pysrc/gmaps_scraper/scraper.pytests/test_cli.pytests/test_place_scraper.pytests/test_scraper.py
Summary
BrowserSessionConfig(human_mouse=True)and the--human-mouseCLI flag.--disable-random-window-sizefor callers that need CloakBrowser's default viewport, and carry session options into parallel worker sessions.Validation
uv run python -m unittest tests.test_scraper tests.test_cli tests.test_place_scraperuv run python -m unittest discover -s tests./scripts/lint.sh./scripts/typecheck.shruff,mypygit diff --checkSummary by CodeRabbit
New Features
--disable-random-window-sizeand--human-mouseCLI flags to control browser behaviorDocumentation
Tests