Commit Graph

11 Commits

Author SHA1 Message Date
Alejandro Gutiérrez
bdffb5eaac Add API interception for hybrid scraping and update selectors
- Add new api_interceptor.py module for CDP network interception
- Capture Google Maps internal API responses during scrolling
- Parse protobuf-like JSON responses to extract review data
- Merge API-captured reviews with DOM-scraped data
- Update CSS selectors for January 2026 Google Maps structure
- Add cookie consent dismissal for multiple languages
- Add --api-intercept CLI flag and config option
- Fix review card and pane selectors (.jftiEf, .XiKgde)
- Improve review ID extraction from card elements

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-17 21:51:10 +00:00
George Khananaev
262f0c0be7 migrate to SeleniumBase UC Mode for automatic version management
- Replace undetected-chromedriver with seleniumbase for better Chrome/ChromeDriver compatibility
- Automatic version matching eliminates manual cache clearing and version conflicts
- Enhanced anti-detection with UC Mode and CDP stealth settings
- Simplified requirements.txt (SeleniumBase manages common dependencies)
- Fix sort selection bug (was selecting wrong menu items)
- Improve scrolling patience (max_idle: 3→15, max_attempts: 10→50)
- Add scroll position tracking to detect when stuck
- Add fallback pane selectors for better reliability
- Update documentation (README, ARCHITECTURE, TROUBLESHOOTING)
- Add comprehensive test suite for SeleniumBase integration
- Version bump to 1.0.1

Developed by George Khananaev
2025-12-07 19:40:13 +07:00
George Khananaev
6b60b02eec Test 2025-08-20 02:46:01 +07:00
George Khananaev
dddf388422 Added api support, now the scrapper can be triggered from 3rd party services 2025-08-20 02:42:01 +07:00
RomanAbashin
72fcc6f162 Get original size images from Google 2025-08-09 10:55:51 +03:00
George Khananaev
50aaa9ce26 Added pytest + some tests.
Added AWS S3 Support (optional, for cloud image storage)
2025-06-03 00:12:11 +07:00
George Khananaev
84399dfbe8 Merge branch 'detached'
# Conflicts:
#	modules/scraper.py
2025-06-02 23:33:29 +07:00
George Khananaev
c4fa7ecd93 fixed the english scraper 2025-06-02 23:22:19 +07:00
George Khananaev
54f98ae921 fixed the issue with english localization 2025-06-02 13:22:50 +07:00
George Khananaev
cbc4bfe72d added config file. 2025-05-12 01:30:17 +07:00
George Khananaev
5bbaf455d8 Release Google Reviews Scraper Pro v1.0.0 (2025)
Initial release with multi-language support, MongoDB integration, image handling, URL replacement, and robust error handling. Includes detailed documentation, usage examples, and recommended usage guidelines. Built to effectively handle Google's 2025 interface changes.
2025-04-24 22:12:07 +07:00