Add API interception for hybrid scraping and update selectors
- Add new api_interceptor.py module for CDP network interception - Capture Google Maps internal API responses during scrolling - Parse protobuf-like JSON responses to extract review data - Merge API-captured reviews with DOM-scraped data - Update CSS selectors for January 2026 Google Maps structure - Add cookie consent dismissal for multiple languages - Add --api-intercept CLI flag and config option - Fix review card and pane selectors (.jftiEf, .XiKgde) - Improve review ID extraction from card elements Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
4
start.py
4
start.py
@@ -64,6 +64,10 @@ def main():
|
||||
# Update config with the provided custom parameters
|
||||
config["custom_params"].update(args.custom_params)
|
||||
|
||||
# Handle API interception option
|
||||
if args.enable_api_intercept:
|
||||
config["enable_api_intercept"] = True
|
||||
|
||||
# Initialize and run scraper
|
||||
scraper = GoogleReviewsScraper(config)
|
||||
scraper.scrape()
|
||||
|
||||
Reference in New Issue
Block a user