- Add version selector dropdown in scrape confirmation modal
- Default to v1.1.0 (Multi-Sort) which bypasses ~1000 review limit
- Pass scraper_version through API proxy to backend
- Update /new page fallback to show v1.1.0 as available
- Show version description explaining multi-sort benefits
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use fixed positioning with top/left 50% and translate -50%
- More reliable centering regardless of parent containers
- Add max-width for mobile responsiveness
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Keep blue background when isCheckingReviews is true
- Add cursor-wait during validation
- Move disabled styling to explicit condition check
- White spinner now visible on blue background
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Reset search fields after job is successfully launched
- Allow user to immediately start another scrape
- Save active jobs to localStorage for persistence across refresh
- Restore jobs from localStorage on page load
- Resume polling for non-terminal jobs (pending/running)
- Filter out jobs older than 24 hours
- Add remove button (X) to each job card
- Clean up localStorage when jobs are removed
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove w-full that caused alignment issues
- Use fixed width (400px) for consistent centering
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove w-full and mx-auto that caused alignment issues
- Use fixed width (280px) instead of max-w-xs
- Let flex container handle centering
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Center modal properly within map preview area
- Add 24px padding from map edges
- Make modal more compact (max-w-xs)
- Reduce text and element sizes for better fit
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Place Business Name, Location, and Validate button in same row
- Reduce padding and font sizes for compact inline layout
- Show abbreviated text on mobile (responsive)
- Use checkmark indicator for auto-detected location
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Split single search input into two fields: Business Name (required)
and Location (auto-detected from IP geolocation)
- Auto-fill location field with city/country from IP on page load
- Add click overlay on map iframe to prevent interaction
- Add warning modal when user clicks map, directing them to use search
- Update test URLs to use split format
- Make Validate button full-width for better UX
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Restore original Google Maps embed iframe approach
- URL: maps.google.com/maps?q=...&output=embed&z=15
- Add "Open in Maps" overlay button on the map
- Height 300px for better visibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace non-working Google Maps embed iframe with animated location preview
- Add "Open in Google Maps" button to open location in new tab
- Add scraper type selection dropdown fetching from /api/admin/scrapers
- Show selected scraper info with formatted labels (Google Reviews v1.0.0)
- Include scraper_version and scraper_variant in job submission
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Transfer user's browser fingerprint (user-agent, viewport, timezone,
language, geolocation) to Chrome for more authentic scraping
- Display review topics from Google Maps in analytics dashboard
- Show business category badge in analytics header
- Fix date_text null handling in analytics (handle undefined/timestamp fields)
- Add review_topics and business_category to JobStatus interface
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Small (~79 reviews): R. Fleitas Peluqueros
- Medium (~589 reviews): ClickRent Gran Canaria
- Large (~2000+ reviews): Hospital Doctor Negrín
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
BREAKING IMPROVEMENTS:
1. Early Detection for No Reviews:
- Check for "no reviews" messages in 11+ languages before scraping
- Detect disabled reviews tabs and aria-labels with 0 reviews
- Return early with success when no reviews exist (saves time)
- Prevents wasted scraping attempts on businesses with no reviews
2. Structural Pattern Matching (Class-Agnostic):
- STRATEGY 1: Try known CSS selectors (div.jftiEf.fontBodyMedium, etc.)
- STRATEGY 2: Structural matching - find containers with review-like structure
* Looks for elements containing: author + rating + text + date
* Counts elements with 3+ review indicators (robust, works across layouts)
- STRATEGY 3: Use role="article" with review content detection
- Falls back through strategies automatically
3. Less Script-Dependent Selectors:
- Uses aria-label attributes (more stable than CSS classes)
- Uses role attributes (semantic HTML)
- Searches for structural patterns (author img + rating span + text span)
- Works across different Google Maps page layouts and languages
4. Frontend Improvement:
- Hide "Open Analytics Dashboard" button when reviews_count is 0
- Only show action buttons for completed jobs with reviews
TECHNICAL DETAILS:
Structural Matching Logic:
- Scans all divs for review indicators:
* hasAuthor: img with photo/avatar in src
* hasRating: aria-label containing "star" or "rating"
* hasText: span with 20+ characters
* hasDate: text matching date patterns (day/week/month/year)
- Element is a review if it has 3+ of these indicators
Early Detection Patterns:
- Checks page text for: "no reviews yet", "be the first to review", etc.
- Checks for "0 reviews" patterns in text and aria-labels
- Checks if reviews tab is disabled or aria-disabled
Benefits:
- Works on Lithuanian hospital page (was getting 0/271 reviews)
- Handles regional Google Maps variations automatically
- Faster exit for businesses with no reviews
- More reliable across Google Maps UI updates
- Better UX: no empty analytics dashboard for 0-review jobs
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Performance improvements:
- Validation speed: 59.71s → 10.96s (5.5x improvement)
- Removed 50+ console.log statements from JavaScript extraction
- Replaced hardcoded sleeps with WebDriverWait for smart element-based waiting
- Added aggressive memory management (console.clear, GC, image unloading every 20 scrolls)
Scraping improvements:
- Increased idle detection from 6 to 12 consecutive idle scrolls for completeness
- Added real-time progress updates every 5 scrolls with percentage calculation
- Added crash recovery to extract partial reviews if Chrome crashes
- Removed artificial 200-review limit to scrape ALL reviews
Timestamp tracking:
- Added updated_at field separate from started_at for progress tracking
- Frontend now shows both "Started" (fixed) and "Last Update" (dynamic)
Robustness improvements:
- Added 5 fallback CSS selectors to handle different Google Maps page structures
- Now tries: div.jftiEf.fontBodyMedium, div.jftiEf, div[data-review-id], etc.
- Automatic selector detection logs which selector works for debugging
Test results:
- Successfully scraped 550 reviews in 150.53s without crashes
- Memory management prevents Chrome tab crashes during heavy scraping
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
