Alejandro Gutiérrez d989178119 7x faster scraping with JS parsing + batch flushing ...

Performance improvements:
- JS-based DOM parsing (single browser call vs Selenium round-trips)
- Batch flushing to disk every 500 reviews to free memory
- Hide parsed elements (display:none) to reduce DOM overhead
- Cycle timing instrumentation for debugging slowdowns

Results: 2826 reviews in 6.7min (7.1/sec) vs 2190 in 37min (1.0/sec)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-22 10:01:22 +00:00

..

__init__.py

Release Google Reviews Scraper Pro v1.0.0 (2025)

2025-04-24 22:12:07 +07:00

api_interceptor.py

Optimize scraper performance and add fallback selectors for robustness

2026-01-18 19:49:24 +00:00

chrome_pool.py

Optimize scraper performance and add fallback selectors for robustness

2026-01-18 19:49:24 +00:00

cli.py

Add API interception for hybrid scraping and update selectors

2026-01-17 21:51:10 +00:00

config.py

fixed the issue with english localization

2025-06-02 13:22:50 +07:00

data_storage.py

fixed the issue with english localization

2025-06-02 13:22:50 +07:00

database.py

Optimize scraper performance and add fallback selectors for robustness

2026-01-18 19:49:24 +00:00

date_converter.py

Release Google Reviews Scraper Pro v1.0.0 (2025)

2025-04-24 22:12:07 +07:00

fast_scraper.py

Add fallback locale retry and pane-scoped selectors for robust review detection

2026-01-18 20:36:42 +00:00

health_checks.py

Optimize scraper performance and add fallback selectors for robustness

2026-01-18 19:49:24 +00:00

image_handler.py

Test

2025-08-20 02:46:01 +07:00

job_manager.py

Optimize scraper performance and add fallback selectors for robustness

2026-01-18 19:49:24 +00:00

models.py

Add API interception for hybrid scraping and update selectors

2026-01-17 21:51:10 +00:00

s3_handler.py

Added pytest + some tests.

2025-06-03 00:12:11 +07:00

scraper_clean.py

7x faster scraping with JS parsing + batch flushing

2026-01-22 10:01:22 +00:00

scraper.py

Implement continuous scrolling with smart gap-based timeout

2026-01-19 01:39:47 +00:00

utils.py

Release Google Reviews Scraper Pro v1.0.0 (2025)

2025-04-24 22:12:07 +07:00

webhooks.py

Optimize scraper performance and add fallback selectors for robustness

2026-01-18 19:49:24 +00:00