262f0c0be7
- Replace undetected-chromedriver with seleniumbase for better Chrome/ChromeDriver compatibility - Automatic version matching eliminates manual cache clearing and version conflicts - Enhanced anti-detection with UC Mode and CDP stealth settings - Simplified requirements.txt (SeleniumBase manages common dependencies) - Fix sort selection bug (was selecting wrong menu items) - Improve scrolling patience (max_idle: 3→15, max_attempts: 10→50) - Add scroll position tracking to detect when stuck - Add fallback pane selectors for better reliability - Update documentation (README, ARCHITECTURE, TROUBLESHOOTING) - Add comprehensive test suite for SeleniumBase integration - Version bump to 1.0.1 Developed by George Khananaev
Tests
This directory contains pytest tests for the Google Reviews Scraper.
Running Tests
- Install dependencies:
pip install -r requirements.txt
- Run all tests:
pytest tests/
- Run specific test files:
pytest tests/test_mongodb_connection.py
pytest tests/test_s3_connection.py
- Run with verbose output:
pytest tests/ -v
Test Coverage
MongoDB Connection Tests (test_mongodb_connection.py)
- Tests MongoDB connection when enabled in config
- Validates MongoDB configuration parameters
- Tests basic database operations (insert/find/delete)
- Skips tests when MongoDB is disabled
S3 Connection Tests (test_s3_connection.py)
- Tests S3 connection when enabled in config
- Validates S3 configuration parameters
- Tests file upload/download operations
- Tests S3Handler class initialization
- Skips tests when S3 is disabled
Configuration
Tests use the main config.yaml file in the project root. Make sure your configuration is properly set up:
- For MongoDB tests: Ensure
use_mongodb: trueand valid MongoDB credentials - For S3 tests: Ensure
use_s3: trueand valid AWS credentials
Test Results
- Tests will be skipped if the corresponding service (MongoDB/S3) is disabled in config
- Failed connection tests indicate configuration or service availability issues
- All tests should pass when services are properly configured and accessible