Documents:
- Data flow and architecture
- CLI options and usage
- Output schema with examples
- Scoring formulas
- Production guardrails
- Thresholds and domain mapping
- Testing instructions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Production fixes:
- Cross-business join safety: all queries join on (review_id, business_id)
- Timestamp normalization: iso_z() for all output timestamps
- Score formula alignment: matches PERIOD_SCORES_QUERY for consistency
- Invariant check: fails if scores.overall != comparisons.current
- primary_run_id: uses max(created_at) in time_window mode
- Language normalization: auto/auto-detect -> unknown
- Review language: majority voting over spans per review
Executive summary guardrails:
- Weakness priority: negative driver > qualifying dip > none
- Dip qualification: within 90 days AND review_count >= 3
- Most recent dip selection when multiple qualify
- No contradiction: "dip" cannot pair with "no major issues"
- Action grounding: must tie to cited weakness or top positive driver
CLI options:
- --no-summary: disable executive summary
- --require-summary: exit code 2 if LLM fails
- --summary-model: configurable model (default gpt-4o-mini)
Includes unit test suite (16 tests) for narrative guardrails.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously hardcoded "Last 12 months" which was misleading when data
spanned multiple years. Now calculates the actual period from the
earliest to latest review dates in the dataset.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Pydantic was coercing V2 StrengthToProtect dicts to the partial
ReportStrengthResponse type, dropping fields like `percentage` and
`top_quotes`. Changed to list[Any] to pass through raw data.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changed the business name query to ORDER BY COUNT(*) DESC instead of
arbitrary LIMIT 1, ensuring the correct business is identified even
when trace amounts of other business data leak into a job.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Extended SynthesisResponse model to support both legacy (v1) and
new 6-section (v2) report formats
- V2 format includes executive_summary, risk_scorecard, critical_issues,
action_matrix, and tracking_kpis sections
- Frontend type guards use report_version and executive_summary fields
to detect format and render appropriate components
- Backwards compatible: legacy v1 responses still work unchanged
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Change action_matrix reference from old 'actions' field.
Add issues_identified and health_score to stage result data.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create BusinessReport.tsx with 6 sections:
1. Executive Summary (health score, rating, momentum)
2. Risk Scorecard (indicators with colors/trends)
3. Critical Issues (evidence, solutions, timelines)
4. Strengths to Protect (quotes, leverage actions)
5. Action Matrix (effort/impact quadrants)
6. 90-Day Tracking (KPI targets table)
- Update types.ts with new interfaces:
- SynthesisV2 for new report format
- LegacySynthesis for backwards compatibility
- Type guard isSynthesisV2() for runtime detection
- Update ReportTab to auto-detect synthesis version
- Update AnalystReport, ReviewIQDashboard, StoryView
for backwards compatibility with union type
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Transform synthesis stage from consultant memo to productized report:
- New 6-section structure: Executive Summary, Risk Scorecard,
Critical Issues, Strengths, Action Matrix, 90-Day Tracking
- Add chart data aggregation (7 charts: gauge, pies, trends)
- Evidence-grounded LLM prompt requiring quote citations
- Hyper-specific solution generation with WHAT/WHO/WHEN/WHY
- Taxonomy-guided solutions contextualized to business type
- Staff name extraction from quotes for leverage actions
- Success metrics tied to actual complaint keywords
Report sections now include:
- Health score (1-100) with revenue-at-risk estimates
- Risk indicators with color-coded scores and trends
- Critical issues with evidence, root cause, and specific solutions
- Strengths with named staff and specific marketing actions
- Action matrix with effort/impact quadrants and deadlines
- 90-day KPIs with 30/60/90 day targets
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changed outer button to div with role="button" to avoid HTML validation
error of nested buttons (translate button inside complaint card).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Stage5Synthesizer class that generates AI narratives and action plans
- Add generate() method to LLMClient for synthesis generation
- Integrate Stage 5 into pipeline runner after route stage
- Add synthesis JSONB column to pipeline.executions table
- Update reviewiq_analytics API to return synthesis data
- Synthesis includes: executive narrative, sentiment/category/timeline insights,
action plan, marketing angles, and priority recommendations
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Frontend:
- Add Synthesis type with action plan, insights, annotations
- ExecutiveSummary: Accept synthesis prop for AI narrative
- SentimentPie: Accept insight prop for contextual explanation
- IntensityHeatmap: Accept insight + highlightDomain props
- TimelineChart: Accept insight + annotations props
- All components gracefully degrade when synthesis is null
Backend:
- Add Stage 4: Synthesize for generating AI narratives
- Gathers context from classified spans
- Generates executive narrative, section insights, action plan
- Produces timeline annotations and marketing angles
- Stores synthesis in pipeline.executions table
Components show AI insights with purple gradient styling when available,
fall back to existing behavior when synthesis is not yet generated.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add 2x2 matrix visualization (Quick Wins, Critical, Strategic, Nice to Have)
- Position items based on frequency/effort coordinates with dots and leader lines
- Add L-shaped axes with arrows showing Frequency (X) and Effort (Y) directions
- Include unified coordinate grid overlay across all quadrants
- Add clickable subcode labels with hover effects
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create ExecutionsView component with TanStack Table
- Add status filter buttons with count badges
- Add action buttons: Analytics, Metrics, Debug
- Add debug modal with AI copy-paste button for failed executions
- Generate detailed debug report with stage metrics and error context
- Update executions page to use new component
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add run pipeline page with job selection UI
- Add execution detail page with stage metrics visualization
- Add stage_metrics and total_duration_ms to pipeline.executions table
- Create Next.js API proxy routes for all pipeline endpoints
- Fix trailing slash issues in pipeline-api.ts URLs
- Add Docker volume mounts for pipeline packages
- Add REVIEWIQ_DATABASE_URL and LLM API keys to docker-compose
- Fix JSONB field parsing in execution detail endpoint
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add version selector dropdown in scrape confirmation modal
- Default to v1.1.0 (Multi-Sort) which bypasses ~1000 review limit
- Pass scraper_version through API proxy to backend
- Update /new page fallback to show v1.1.0 as available
- Show version description explaining multi-sort benefits
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit implements a plugin-like pipeline architecture with:
Pipeline Core Package (packages/pipeline-core/):
- BasePipeline abstract class all pipelines implement
- PipelineRegistry for database-backed discovery/management
- PipelineRunner for execution with status tracking
- DashboardConfig contracts for dynamic widget definitions
Database Migration (006_pipeline_registry.sql):
- pipeline.registry table for registered pipelines
- pipeline.executions table for execution history
- Views for execution stats and monitoring
ReviewIQ Pipeline Refactor:
- Implements BasePipeline interface
- Adds get_dashboard_config() with widget definitions
- Adds get_widget_data() methods for all dashboard widgets
- Maintains backward compatibility with Pipeline alias
Generic Pipeline API (api/routes/pipelines.py):
- GET /api/pipelines - List all registered pipelines
- GET /api/pipelines/{id} - Pipeline details
- POST /api/pipelines/{id}/execute - Execute pipeline
- GET /api/pipelines/{id}/dashboard - Dashboard config
- GET /api/pipelines/{id}/widgets/{w} - Widget data
- GET /api/pipelines/{id}/executions - Execution history
Frontend Dynamic Dashboard System:
- DynamicDashboard component renders from config
- WidgetRegistry maps types to components
- Widget components: StatCard, LineChart, BarChart,
PieChart, DataTable, Heatmap
- Pipeline API client library
Frontend Pipeline Pages:
- /pipelines - List all registered pipelines
- /pipelines/[id] - Dynamic dashboard for pipeline
- /pipelines/[id]/executions - Execution history
- Pipelines nav item in Sidebar
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Import both v1.0.0 and v1.1.0 scraper versions
- Add SCRAPER_VERSIONS registry mapping version strings to functions
- Add get_scraper_for_version() to route based on job metadata
- Default to v1.1.0 (multi-sort) for new jobs
- Frontend can select specific version via scraper_version parameter
- Validation endpoint continues using v1.0.0 for speed
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The multi-sort loop was calling get_dom_reviews() which doesn't exist.
API interception alone is sufficient for capturing reviews during
multi-sort passes, so we now use only api_reviews.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
v1.0.0 improvements:
- Add captcha detection (reCAPTCHA, unusual traffic, challenges)
- Block fonts, analytics, maps tiles for faster scrolling
- Add 95% close-enough threshold to skip unnecessary retries
- Stop immediately if captcha detected instead of retrying
v1.1.0 new features:
- Multi-sort strategy to bypass ~1000 review limit
- Cycles through newest/lowest/highest/relevant sorts
- Auto mode: enables multi-sort when total > 1000
- Diminishing returns detection (stops if <5% new per pass)
- Configurable sort order and thresholds
Also adds test_scraper_v110.py CLI tool for testing multi-sort.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add ScraperV1Adapter to transform scraped reviews into pipeline format
- Handles relative timestamps (centerDate)
- Generates deterministic IDs for DOM-sourced reviews
- Filters out empty (rating-only) reviews
- Add sample barbershop reviews (79 reviews, 46 with text)
- Real data from Las Palmas barbershop
- Multi-language: Spanish, English, German, Norwegian, Italian
- Add test_pipeline_real_data.py for E2E testing with real data
- Uses mock classifier based on keywords and rating
- Full pipeline flow: raw -> enriched -> spans -> issues -> facts
Test results with real data:
- 46 reviews processed
- 6 languages detected (es: 35, en: 7, de: 1, no: 1, it: 1, ca: 1)
- 3 issues identified from negative reviews
- 29 fact records aggregated across date range 2017-2025
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests the full pipeline flow:
- Stage 1: Insert raw reviews, normalize text
- Stage 2: Mock LLM classification, insert spans
- Stage 3: Route negative spans to issues
- Stage 4: Aggregate facts by URT code and date
Validates all pipeline.* tables are populated correctly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Import LogCapture from scraper module
- Remove unused StructuredLogger import
- Use correct log_capture parameter name
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Root cause: Cards were hidden but not removed from DOM, causing
memory buildup (400+ nodes) that crashed Chrome tabs.
Changes:
- Actually remove processed cards from DOM (not just hide them)
- Keep last 50 cards for scroll reference/continuity
- Remove adjacent separator elements along with cards
- Add logging when DOM cleanup removes cards
- Cards near scroll end stay visible for reference
This should prevent "tab crashed" errors during long scraping
sessions with 500+ reviews.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use updated_at (last successful data loop) instead of Date.now()
- Speed now reflects actual data retrieval rate, not declining over time
- Updated in table column, monitored job view, and stats row
- Fall back to Date.now() if updated_at is not available
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use fixed positioning with top/left 50% and translate -50%
- More reliable centering regardless of parent containers
- Add max-width for mobile responsiveness
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Keep blue background when isCheckingReviews is true
- Add cursor-wait during validation
- Move disabled styling to explicit condition check
- White spinner now visible on blue background
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Reset search fields after job is successfully launched
- Allow user to immediately start another scrape
- Save active jobs to localStorage for persistence across refresh
- Restore jobs from localStorage on page load
- Resume polling for non-terminal jobs (pending/running)
- Filter out jobs older than 24 hours
- Add remove button (X) to each job card
- Clean up localStorage when jobs are removed
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove w-full that caused alignment issues
- Use fixed width (400px) for consistent centering
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove w-full and mx-auto that caused alignment issues
- Use fixed width (280px) instead of max-w-xs
- Let flex container handle centering
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Center modal properly within map preview area
- Add 24px padding from map edges
- Make modal more compact (max-w-xs)
- Reduce text and element sizes for better fit
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Place Business Name, Location, and Validate button in same row
- Reduce padding and font sizes for compact inline layout
- Show abbreviated text on mobile (responsive)
- Use checkmark indicator for auto-detected location
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Split single search input into two fields: Business Name (required)
and Location (auto-detected from IP geolocation)
- Auto-fill location field with city/country from IP on page load
- Add click overlay on map iframe to prevent interaction
- Add warning modal when user clicks map, directing them to use search
- Update test URLs to use split format
- Make Validate button full-width for better UX
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Restore original Google Maps embed iframe approach
- URL: maps.google.com/maps?q=...&output=embed&z=15
- Add "Open in Maps" overlay button on the map
- Height 300px for better visibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace non-working Google Maps embed iframe with animated location preview
- Add "Open in Google Maps" button to open location in new tab
- Add scraper type selection dropdown fetching from /api/admin/scrapers
- Show selected scraper info with formatted labels (Google Reviews v1.0.0)
- Include scraper_version and scraper_variant in job submission
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
