Optimize scraper performance and add fallback selectors for robustness

Performance improvements:
- Validation speed: 59.71s → 10.96s (5.5x improvement)
- Removed 50+ console.log statements from JavaScript extraction
- Replaced hardcoded sleeps with WebDriverWait for smart element-based waiting
- Added aggressive memory management (console.clear, GC, image unloading every 20 scrolls)

Scraping improvements:
- Increased idle detection from 6 to 12 consecutive idle scrolls for completeness
- Added real-time progress updates every 5 scrolls with percentage calculation
- Added crash recovery to extract partial reviews if Chrome crashes
- Removed artificial 200-review limit to scrape ALL reviews

Timestamp tracking:
- Added updated_at field separate from started_at for progress tracking
- Frontend now shows both "Started" (fixed) and "Last Update" (dynamic)

Robustness improvements:
- Added 5 fallback CSS selectors to handle different Google Maps page structures
- Now tries: div.jftiEf.fontBodyMedium, div.jftiEf, div[data-review-id], etc.
- Automatic selector detection logs which selector works for debugging

Test results:
- Successfully scraped 550 reviews in 150.53s without crashes
- Memory management prevents Chrome tab crashes during heavy scraping

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Alejandro Gutiérrez
2026-01-18 19:49:24 +00:00
parent bdffb5eaac
commit faa0704737
108 changed files with 23632 additions and 54 deletions

41
web/.gitignore vendored Normal file
View File

@@ -0,0 +1,41 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
# dependencies
/node_modules
/.pnp
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/versions
# testing
/coverage
# next.js
/.next/
/out/
# production
/build
# misc
.DS_Store
*.pem
# debug
npm-debug.log*
yarn-debug.log*
yarn-error.log*
.pnpm-debug.log*
# env files (can opt-in for committing if needed)
.env*
# vercel
.vercel
# typescript
*.tsbuildinfo
next-env.d.ts

90
web/README.md Normal file
View File

@@ -0,0 +1,90 @@
# Google Reviews Scraper - Testing Interface
A Next.js web interface for testing the containerized Google Reviews Scraper API.
## Features
- 🎯 **URL Input** - Paste any Google Maps business URL
- 📊 **Real-time Status** - Live job tracking with polling
-**Performance Metrics** - Reviews count, time, speed
- 📱 **Review Display** - Beautiful UI for scraped reviews
- 💾 **Export JSON** - Download reviews as JSON
## Quick Start
### 1. Start the Scraper API
First, make sure the containerized scraper is running:
```bash
cd ..
docker-compose -f docker-compose.production.yml up -d
```
The API should be running at `http://localhost:8000`
### 2. Start the Web Interface
```bash
npm install
npm run dev
```
Open [http://localhost:3000](http://localhost:3000)
## Usage
1. **Paste a Google Maps URL**
```
https://www.google.com/maps/place/Business+Name/...
```
2. **Click "Scrape"**
- Job is submitted to the API
- Status updates in real-time
- Reviews appear when complete
3. **View Results**
- See all scraped reviews
- Export as JSON
- View performance metrics
## Environment Variables
Create `.env.local` if you need to customize:
```bash
# API URL (default: http://localhost:8000)
NEXT_PUBLIC_API_URL=http://localhost:8000
```
## API Endpoints Used
This interface connects to:
- `POST /scrape` - Submit scraping job
- `GET /jobs/{job_id}` - Get job status
- `GET /jobs/{job_id}/reviews` - Get reviews
## Tech Stack
- **Next.js 15** - React framework
- **TypeScript** - Type safety
- **Tailwind CSS** - Styling
- **API Proxy** - Next.js API routes proxy to scraper API
## Development
```bash
npm run dev # Start dev server
npm run build # Build for production
npm run start # Start production server
npm run lint # Run ESLint
```
## Notes
- The interface polls job status every 2 seconds
- Polling stops when job completes or fails
- Reviews are fetched with a limit of 1000 by default
- Export button downloads reviews as formatted JSON

View File

@@ -0,0 +1,37 @@
import { NextRequest, NextResponse } from 'next/server';
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000';
export async function POST(request: NextRequest) {
try {
const { url } = await request.json();
if (!url) {
return NextResponse.json({ error: 'URL is required' }, { status: 400 });
}
// Call the containerized scraper API to check if reviews exist
const response = await fetch(`${API_BASE_URL}/check-reviews`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ url }),
});
const data = await response.json();
if (!response.ok) {
return NextResponse.json(
{ error: data.detail || 'Failed to check reviews' },
{ status: response.status }
);
}
return NextResponse.json(data);
} catch (error) {
console.error('Check reviews API error:', error);
return NextResponse.json(
{ error: 'Failed to connect to scraper API' },
{ status: 500 }
);
}
}

View File

@@ -0,0 +1,33 @@
import { NextRequest, NextResponse } from 'next/server';
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000';
export async function GET(
request: NextRequest,
{ params }: { params: Promise<{ jobId: string }> }
) {
try {
const { jobId } = await params;
const { searchParams } = new URL(request.url);
const limit = searchParams.get('limit') || '1000';
const response = await fetch(`${API_BASE_URL}/jobs/${jobId}/reviews?limit=${limit}`);
if (!response.ok) {
return NextResponse.json(
{ error: 'Failed to get reviews' },
{ status: response.status }
);
}
const data = await response.json();
// API returns { job_id, reviews: [...], count }, we just need the reviews array
return NextResponse.json({ reviews: data.reviews || [] });
} catch (error) {
console.error('Reviews API error:', error);
return NextResponse.json(
{ error: 'Failed to get reviews' },
{ status: 500 }
);
}
}

View File

@@ -0,0 +1,30 @@
import { NextRequest, NextResponse } from 'next/server';
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000';
export async function GET(
request: NextRequest,
{ params }: { params: Promise<{ jobId: string }> }
) {
try {
const { jobId } = await params;
const response = await fetch(`${API_BASE_URL}/jobs/${jobId}`);
const data = await response.json();
if (!response.ok) {
return NextResponse.json(
{ error: data.detail || 'Job not found' },
{ status: response.status }
);
}
return NextResponse.json(data);
} catch (error) {
console.error('Job status API error:', error);
return NextResponse.json(
{ error: 'Failed to get job status' },
{ status: 500 }
);
}
}

View File

@@ -0,0 +1,37 @@
import { NextRequest, NextResponse } from 'next/server';
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000';
export async function POST(request: NextRequest) {
try {
const { url } = await request.json();
if (!url) {
return NextResponse.json({ error: 'URL is required' }, { status: 400 });
}
// Call the containerized scraper API
const response = await fetch(`${API_BASE_URL}/scrape`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ url }),
});
const data = await response.json();
if (!response.ok) {
return NextResponse.json(
{ error: data.detail || 'Failed to start scraping' },
{ status: response.status }
);
}
return NextResponse.json(data);
} catch (error) {
console.error('Scrape API error:', error);
return NextResponse.json(
{ error: 'Failed to connect to scraper API' },
{ status: 500 }
);
}
}

BIN
web/app/favicon.ico Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

26
web/app/globals.css Normal file
View File

@@ -0,0 +1,26 @@
@import "tailwindcss";
:root {
--background: #ffffff;
--foreground: #171717;
}
@theme inline {
--color-background: var(--background);
--color-foreground: var(--foreground);
--font-sans: var(--font-geist-sans);
--font-mono: var(--font-geist-mono);
}
@media (prefers-color-scheme: dark) {
:root {
--background: #0a0a0a;
--foreground: #ededed;
}
}
body {
background: var(--background);
color: var(--foreground);
font-family: Arial, Helvetica, sans-serif;
}

34
web/app/layout.tsx Normal file
View File

@@ -0,0 +1,34 @@
import type { Metadata } from "next";
import { Geist, Geist_Mono } from "next/font/google";
import "./globals.css";
const geistSans = Geist({
variable: "--font-geist-sans",
subsets: ["latin"],
});
const geistMono = Geist_Mono({
variable: "--font-geist-mono",
subsets: ["latin"],
});
export const metadata: Metadata = {
title: "Create Next App",
description: "Generated by create next app",
};
export default function RootLayout({
children,
}: Readonly<{
children: React.ReactNode;
}>) {
return (
<html lang="en">
<body
className={`${geistSans.variable} ${geistMono.variable} antialiased`}
>
{children}
</body>
</html>
);
}

38
web/app/page.tsx Normal file
View File

@@ -0,0 +1,38 @@
import ScraperTest from '@/components/ScraperTest';
export default function Home() {
return (
<div className="min-h-screen bg-gradient-to-br from-blue-600 to-indigo-700 py-12 px-4">
<main className="max-w-5xl mx-auto">
<div className="text-center mb-10">
<h1 className="text-4xl md:text-5xl font-bold text-white mb-3">
Google Reviews Scraper
</h1>
<p className="text-blue-100 text-lg">
Test the containerized scraper API
</p>
<div className="mt-4 inline-flex items-center gap-2 px-4 py-2 bg-blue-500/30 rounded-lg text-blue-100 text-sm">
<div className="w-2 h-2 bg-green-400 rounded-full animate-pulse"></div>
Powered by SeleniumBase UC Mode
</div>
</div>
<div className="bg-white rounded-2xl shadow-2xl p-6 md:p-8">
<ScraperTest />
</div>
<div className="mt-8 text-center text-blue-100 text-sm space-y-2">
<p className="font-medium">💡 Example URLs to test:</p>
<div className="space-y-1 text-xs">
<p className="font-mono bg-blue-500/20 rounded px-3 py-1 inline-block">
https://www.google.com/maps/place/Soho+Club/...
</p>
</div>
<p className="mt-4 text-blue-200">
API running at: <span className="font-mono">localhost:8000</span>
</p>
</div>
</main>
</div>
);
}

View File

@@ -0,0 +1,703 @@
'use client';
import { useState, useMemo } from 'react';
import {
useReactTable,
getCoreRowModel,
getFilteredRowModel,
getSortedRowModel,
getPaginationRowModel,
ColumnDef,
flexRender,
SortingState,
ColumnFiltersState,
} from '@tanstack/react-table';
import { BarChart, Bar, XAxis, YAxis, CartesianGrid, Tooltip, ResponsiveContainer, PieChart, Pie, Cell, LineChart, Line } from 'recharts';
import { Star, TrendingUp, Image, FileText, MessageSquare, Calendar, ArrowUpDown, ArrowUp, ArrowDown, Search, Download, Filter, AlertTriangle, ThumbsUp, ThumbsDown } from 'lucide-react';
import { Review, calculateReviewStats, getSentimentLabel, getSentimentColor, DateRange, filterReviewsByDateRange, calculateTimelineData } from '@/lib/analytics';
interface ReviewAnalyticsProps {
reviews: Review[];
businessName?: string;
}
export default function ReviewAnalytics({ reviews, businessName }: ReviewAnalyticsProps) {
const [sorting, setSorting] = useState<SortingState>([{ id: 'date', desc: true }]); // Default: newest first
const [columnFilters, setColumnFiltersState] = useState<ColumnFiltersState>([]);
const [globalFilter, setGlobalFilter] = useState('');
const [selectedRatings, setSelectedRatings] = useState<number[]>([1, 2, 3, 4, 5]);
const [selectedSentiments, setSelectedSentiments] = useState<('positive' | 'neutral' | 'negative')[]>(['positive', 'neutral', 'negative']);
const [dateRange, setDateRange] = useState<DateRange>('all');
// Filter reviews by date range
const dateFilteredReviews = useMemo(() => {
return filterReviewsByDateRange(reviews, dateRange);
}, [reviews, dateRange]);
// Calculate statistics on date-filtered reviews
const stats = useMemo(() => calculateReviewStats(dateFilteredReviews), [dateFilteredReviews]);
// Calculate timeline data for chart
const timelineData = useMemo(() => calculateTimelineData(dateFilteredReviews), [dateFilteredReviews]);
// Filter reviews by selected ratings and sentiments (for table)
const filteredReviews = useMemo(() => {
return dateFilteredReviews.filter(r => {
const matchesRating = selectedRatings.includes(r.rating);
const sentiment = getSentimentLabel(r.rating);
const matchesSentiment = selectedSentiments.includes(sentiment);
const matchesSearch = !globalFilter ||
r.author.toLowerCase().includes(globalFilter.toLowerCase()) ||
r.text?.toLowerCase().includes(globalFilter.toLowerCase()) ||
r.date_text.toLowerCase().includes(globalFilter.toLowerCase());
return matchesRating && matchesSentiment && matchesSearch;
});
}, [dateFilteredReviews, selectedRatings, selectedSentiments, globalFilter]);
const toggleRating = (rating: number) => {
setSelectedRatings(prev =>
prev.includes(rating) ? prev.filter(r => r !== rating) : [...prev, rating]
);
};
const toggleSentiment = (sentiment: 'positive' | 'neutral' | 'negative') => {
setSelectedSentiments(prev =>
prev.includes(sentiment) ? prev.filter(s => s !== sentiment) : [...prev, sentiment]
);
};
const clearAllFilters = () => {
setDateRange('all');
setSelectedRatings([1, 2, 3, 4, 5]);
setSelectedSentiments(['positive', 'neutral', 'negative']);
setGlobalFilter('');
};
const hasActiveFilters = dateRange !== 'all' ||
selectedRatings.length < 5 ||
selectedSentiments.length < 3 ||
globalFilter !== '';
const exportFilteredData = () => {
const dataStr = JSON.stringify(filteredReviews, null, 2);
const dataBlob = new Blob([dataStr], { type: 'application/json' });
const url = URL.createObjectURL(dataBlob);
const link = document.createElement('a');
link.href = url;
link.download = `reviews-filtered-${dateRange}-${new Date().toISOString().split('T')[0]}.json`;
link.click();
};
// Chart colors
const COLORS = {
positive: '#16a34a',
neutral: '#ca8a04',
negative: '#dc2626',
};
// Table columns
const columns = useMemo<ColumnDef<Review>[]>(
() => [
{
accessorKey: 'author',
header: ({ column }) => {
return (
<button
onClick={() => column.toggleSorting(column.getIsSorted() === 'asc')}
className="flex items-center gap-2 hover:text-blue-700 font-semibold"
>
Author
{column.getIsSorted() === 'asc' ? <ArrowUp className="w-4 h-4" /> : column.getIsSorted() === 'desc' ? <ArrowDown className="w-4 h-4" /> : <ArrowUpDown className="w-4 h-4 opacity-50" />}
</button>
);
},
cell: ({ row }) => (
<div className="flex items-center gap-2">
{row.original.avatar_url && (
<img src={row.original.avatar_url} alt={row.original.author} className="w-8 h-8 rounded-full" />
)}
<span className="font-medium text-gray-900">{row.original.author}</span>
</div>
),
},
{
accessorKey: 'rating',
header: ({ column }) => {
return (
<button
onClick={() => column.toggleSorting(column.getIsSorted() === 'asc')}
className="flex items-center gap-2 hover:text-blue-700 font-semibold"
>
Rating
{column.getIsSorted() === 'asc' ? <ArrowUp className="w-4 h-4" /> : column.getIsSorted() === 'desc' ? <ArrowDown className="w-4 h-4" /> : <ArrowUpDown className="w-4 h-4 opacity-50" />}
</button>
);
},
cell: ({ row }) => (
<div className="flex items-center gap-1">
{[...Array(5)].map((_, i) => (
<Star
key={i}
className={`w-4 h-4 ${i < row.original.rating ? 'text-yellow-500 fill-yellow-500' : 'text-gray-300'}`}
/>
))}
<span className="ml-2 font-bold text-gray-900">{row.original.rating}</span>
</div>
),
filterFn: (row, id, value) => {
return value.includes(row.getValue(id));
},
},
{
accessorKey: 'centerDate',
id: 'date',
header: ({ column }) => {
return (
<button
onClick={() => column.toggleSorting(column.getIsSorted() === 'asc')}
className="flex items-center gap-2 hover:text-blue-700 font-semibold"
>
Date
{column.getIsSorted() === 'asc' ? <ArrowUp className="w-4 h-4" /> : column.getIsSorted() === 'desc' ? <ArrowDown className="w-4 h-4" /> : <ArrowUpDown className="w-4 h-4 opacity-50" />}
</button>
);
},
sortingFn: (rowA, rowB) => {
const dateA = rowA.original.centerDate?.getTime() || 0;
const dateB = rowB.original.centerDate?.getTime() || 0;
return dateA - dateB;
},
cell: ({ row }) => {
const formatDate = (date: Date) => {
return date.toLocaleDateString('en-US', { year: 'numeric', month: 'short', day: 'numeric' });
};
const getUncertaintyDays = (minDate: Date, maxDate: Date) => {
const diffMs = Math.abs(maxDate.getTime() - minDate.getTime());
return Math.round(diffMs / (1000 * 60 * 60 * 24));
};
return (
<div className="space-y-1">
<div className="text-gray-900 font-medium">{row.original.date_text}</div>
{row.original.minDate && row.original.maxDate && row.original.centerDate && (
<div className="text-xs text-gray-500 space-y-0.5">
<div>Range: {formatDate(row.original.maxDate)} - {formatDate(row.original.minDate)}</div>
<div className="text-purple-700 font-semibold">
Center: {formatDate(row.original.centerDate)}
</div>
<div className="text-blue-600">
±{getUncertaintyDays(row.original.minDate, row.original.maxDate)} days uncertainty
</div>
</div>
)}
</div>
);
},
},
{
accessorKey: 'text',
header: 'Review',
cell: ({ row }) => {
const [expanded, setExpanded] = useState(false);
const text = row.original.text || 'No review text';
const sentiment = getSentimentLabel(row.original.rating);
return (
<div className="max-w-2xl">
<div className={`inline-block px-2 py-1 rounded-md text-xs font-semibold mb-2 border ${getSentimentColor(sentiment)}`}>
{sentiment.toUpperCase()}
</div>
<p className={`text-gray-800 ${!expanded && 'line-clamp-2'}`}>
{text}
</p>
{text.length > 100 && (
<button
onClick={() => setExpanded(!expanded)}
className="text-blue-700 hover:text-blue-800 text-sm font-semibold mt-1"
>
{expanded ? 'Show less' : 'Show more'}
</button>
)}
</div>
);
},
},
],
[]
);
const table = useReactTable({
data: filteredReviews,
columns,
state: {
sorting,
},
onSortingChange: setSorting,
getCoreRowModel: getCoreRowModel(),
getSortedRowModel: getSortedRowModel(),
getPaginationRowModel: getPaginationRowModel(),
initialState: {
pagination: {
pageSize: 10,
},
},
});
return (
<div className="space-y-6">
{/* Header */}
<div className="flex items-center justify-between">
<div>
<h2 className="text-3xl font-bold text-gray-900">
{businessName ? `${businessName} - Analytics` : 'Review Analytics'}
</h2>
<p className="text-gray-600 mt-1">Comprehensive insights from {reviews.length} total reviews</p>
</div>
</div>
{/* Enhanced Filters */}
<div className="bg-white border-2 border-gray-300 rounded-xl p-5 shadow-sm space-y-4">
{/* Time Period Filter */}
<div className="flex items-center gap-3 flex-wrap">
<Filter className="w-5 h-5 text-gray-700" />
<span className="font-semibold text-gray-900">Time Period:</span>
{(['week', 'month', 'year', 'all'] as DateRange[]).map((range) => (
<button
key={range}
onClick={() => setDateRange(range)}
className={`px-4 py-2 rounded-lg font-semibold transition-all border-2 ${
dateRange === range
? 'bg-blue-600 text-white border-blue-700 shadow-md'
: 'bg-white text-gray-700 border-gray-300 hover:border-blue-400 hover:bg-blue-50'
}`}
>
{range === 'week' ? 'Last Week' : range === 'month' ? 'Last Month' : range === 'year' ? 'Last Year' : 'All Time'}
</button>
))}
</div>
{/* Sentiment Filter */}
<div className="flex items-center gap-3 flex-wrap">
<TrendingUp className="w-5 h-5 text-gray-700" />
<span className="font-semibold text-gray-900">Sentiment:</span>
{(['positive', 'neutral', 'negative'] as const).map((sentiment) => (
<button
key={sentiment}
onClick={() => toggleSentiment(sentiment)}
className={`px-4 py-2 rounded-lg font-semibold transition-all border-2 ${
selectedSentiments.includes(sentiment)
? sentiment === 'positive' ? 'bg-green-600 text-white border-green-700 shadow-md'
: sentiment === 'neutral' ? 'bg-yellow-600 text-white border-yellow-700 shadow-md'
: 'bg-red-600 text-white border-red-700 shadow-md'
: 'bg-white text-gray-700 border-gray-300 hover:border-blue-400 hover:bg-blue-50'
}`}
>
{sentiment === 'positive' ? '😊 Positive (4-5★)' : sentiment === 'neutral' ? '😐 Neutral (3★)' : '😞 Negative (1-2★)'}
</button>
))}
</div>
{/* Filter Summary */}
<div className="flex items-center justify-between pt-2 border-t-2 border-gray-200">
<span className="text-sm font-medium text-gray-600">
Showing {filteredReviews.length} of {reviews.length} reviews
{hasActiveFilters && <span className="text-blue-700 ml-1">(filtered)</span>}
</span>
{hasActiveFilters && (
<button
onClick={clearAllFilters}
className="px-3 py-1.5 bg-gray-100 text-gray-700 rounded-lg hover:bg-gray-200 font-semibold border-2 border-gray-300 text-sm"
>
Clear All Filters
</button>
)}
</div>
</div>
{/* KPI Cards */}
<div className="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-4 gap-4">
{/* Average Rating */}
<div className="bg-gradient-to-br from-yellow-100 to-yellow-200 border-2 border-yellow-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow">
<div className="flex items-center justify-between mb-2">
<div className="flex items-center gap-2">
<Star className="w-5 h-5 text-yellow-700" />
<span className="text-sm font-bold text-yellow-900">Avg Rating</span>
</div>
</div>
<div className="text-3xl font-bold text-yellow-900">{stats.averageRating.toFixed(1)}</div>
<div className="text-xs text-yellow-800 mt-1 font-medium">
{stats.totalReviews} total reviews
</div>
</div>
{/* Positive Reviews */}
<div className="bg-gradient-to-br from-green-100 to-green-200 border-2 border-green-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow cursor-pointer" onClick={() => { setSelectedSentiments(['positive']); setDateRange('all'); }}>
<div className="flex items-center justify-between mb-2">
<div className="flex items-center gap-2">
<ThumbsUp className="w-5 h-5 text-green-700" />
<span className="text-sm font-bold text-green-900">Positive</span>
</div>
</div>
<div className="text-3xl font-bold text-green-900">{stats.sentimentBreakdown.positive}</div>
<div className="text-xs text-green-800 mt-1 font-medium">
{stats.sentimentScore.toFixed(0)}% positive (4-5)
</div>
</div>
{/* Neutral Reviews */}
<div className="bg-gradient-to-br from-yellow-50 to-yellow-100 border-2 border-yellow-300 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow cursor-pointer" onClick={() => { setSelectedSentiments(['neutral']); setDateRange('all'); }}>
<div className="flex items-center justify-between mb-2">
<div className="flex items-center gap-2">
<MessageSquare className="w-5 h-5 text-yellow-700" />
<span className="text-sm font-bold text-yellow-800">Neutral</span>
</div>
</div>
<div className="text-3xl font-bold text-yellow-800">{stats.sentimentBreakdown.neutral}</div>
<div className="text-xs text-yellow-700 mt-1 font-medium">
{((stats.sentimentBreakdown.neutral / stats.totalReviews) * 100).toFixed(0)}% neutral (3)
</div>
</div>
{/* Negative Reviews - Alert */}
<div className="bg-gradient-to-br from-red-100 to-red-200 border-2 border-red-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow cursor-pointer" onClick={() => { setSelectedSentiments(['negative']); setDateRange('all'); }}>
<div className="flex items-center justify-between mb-2">
<div className="flex items-center gap-2">
<AlertTriangle className="w-5 h-5 text-red-700" />
<span className="text-sm font-bold text-red-900">Negative</span>
</div>
</div>
<div className="text-3xl font-bold text-red-900">{stats.negativeReviews}</div>
<div className="text-xs text-red-800 mt-1 font-medium">
{((stats.negativeReviews / stats.totalReviews) * 100).toFixed(0)}% negative (1-2)
</div>
</div>
{/* Recent Activity */}
<div className="bg-gradient-to-br from-blue-100 to-blue-200 border-2 border-blue-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow cursor-pointer" onClick={() => setDateRange('month')}>
<div className="flex items-center justify-between mb-2">
<div className="flex items-center gap-2">
<Calendar className="w-5 h-5 text-blue-700" />
<span className="text-sm font-bold text-blue-900">Recent</span>
</div>
</div>
<div className="text-3xl font-bold text-blue-900">{stats.recentReviews}</div>
<div className="text-xs text-blue-800 mt-1 font-medium">last 30 days</div>
</div>
{/* Review Length */}
<div className="bg-gradient-to-br from-purple-100 to-purple-200 border-2 border-purple-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow">
<div className="flex items-center justify-between mb-2">
<div className="flex items-center gap-2">
<FileText className="w-5 h-5 text-purple-700" />
<span className="text-sm font-bold text-purple-900">Avg Length</span>
</div>
</div>
<div className="text-3xl font-bold text-purple-900">{stats.avgReviewLength}</div>
<div className="text-xs text-purple-800 mt-1 font-medium">words per review</div>
</div>
{/* Photos */}
<div className="bg-gradient-to-br from-pink-100 to-pink-200 border-2 border-pink-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow">
<div className="flex items-center justify-between mb-2">
<div className="flex items-center gap-2">
<Image className="w-5 h-5 text-pink-700" />
<span className="text-sm font-bold text-pink-900">With Photos</span>
</div>
</div>
<div className="text-3xl font-bold text-pink-900">{stats.photoCount}</div>
<div className="text-xs text-pink-800 mt-1 font-medium">
{((stats.photoCount / stats.totalReviews) * 100).toFixed(0)}% have avatars
</div>
</div>
{/* Total Reviews */}
<div className="bg-gradient-to-br from-indigo-100 to-indigo-200 border-2 border-indigo-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow">
<div className="flex items-center justify-between mb-2">
<div className="flex items-center gap-2">
<MessageSquare className="w-5 h-5 text-indigo-700" />
<span className="text-sm font-bold text-indigo-900">Total</span>
</div>
</div>
<div className="text-3xl font-bold text-indigo-900">{stats.totalReviews}</div>
<div className="text-xs text-indigo-800 mt-1 font-medium">all time</div>
</div>
</div>
{/* Rating Timeline with Rolling Average */}
{timelineData.length > 0 && (
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
<h3 className="text-xl font-bold mb-4 text-gray-900">Rating Trend Over Time</h3>
<ResponsiveContainer width="100%" height={300}>
<LineChart data={timelineData}>
<CartesianGrid strokeDasharray="3 3" stroke="#e5e7eb" />
<XAxis
dataKey="date"
tick={{ fill: '#374151', fontWeight: 600 }}
tickLine={{ stroke: '#9ca3af' }}
/>
<YAxis
domain={[0, 5]}
ticks={[0, 1, 2, 3, 4, 5]}
tick={{ fill: '#374151', fontWeight: 600 }}
tickLine={{ stroke: '#9ca3af' }}
/>
<Tooltip
contentStyle={{
backgroundColor: '#ffffff',
border: '2px solid #3b82f6',
borderRadius: '8px',
fontWeight: 600
}}
/>
<Line
type="monotone"
dataKey="rating"
stroke="#94a3b8"
strokeWidth={2}
name="Monthly Avg"
dot={{ fill: '#64748b', r: 4 }}
/>
<Line
type="monotone"
dataKey="rollingAvg"
stroke="#3b82f6"
strokeWidth={3}
name="3-Month Rolling Avg"
dot={{ fill: '#2563eb', r: 5 }}
/>
</LineChart>
</ResponsiveContainer>
</div>
)}
{/* Charts Grid */}
<div className="grid md:grid-cols-3 gap-6">
{/* Rating Distribution - Interactive */}
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
<h3 className="text-lg font-bold mb-4 text-gray-900">
Rating Distribution
<span className="text-xs font-normal text-gray-500 ml-2">(click to filter)</span>
</h3>
<ResponsiveContainer width="100%" height={250}>
<BarChart
data={stats.ratingDistribution}
onClick={(data) => {
if (data && data.activePayload && data.activePayload[0]) {
const rating = data.activePayload[0].payload.rating;
setSelectedRatings([rating]);
setSelectedSentiments(['positive', 'neutral', 'negative']);
}
}}
style={{ cursor: 'pointer' }}
>
<CartesianGrid strokeDasharray="3 3" stroke="#e5e7eb" />
<XAxis
dataKey="rating"
tick={{ fill: '#374151', fontWeight: 600 }}
tickLine={{ stroke: '#9ca3af' }}
/>
<YAxis
tick={{ fill: '#374151', fontWeight: 600 }}
tickLine={{ stroke: '#9ca3af' }}
/>
<Tooltip
contentStyle={{
backgroundColor: '#ffffff',
border: '2px solid #3b82f6',
borderRadius: '8px',
fontWeight: 600
}}
content={({ active, payload }) => {
if (active && payload && payload.length) {
return (
<div className="bg-white border-2 border-blue-600 rounded-lg p-2 shadow-lg">
<p className="font-bold text-gray-900">{payload[0].payload.rating}</p>
<p className="text-sm text-gray-600">{payload[0].value} reviews ({payload[0].payload.percentage.toFixed(1)}%)</p>
<p className="text-xs text-blue-600 mt-1">Click to filter</p>
</div>
);
}
return null;
}}
/>
<Bar dataKey="count" fill="#3b82f6" radius={[8, 8, 0, 0]} />
</BarChart>
</ResponsiveContainer>
</div>
{/* Sentiment Breakdown - Interactive */}
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
<h3 className="text-lg font-bold mb-4 text-gray-900">
Sentiment Breakdown
<span className="text-xs font-normal text-gray-500 ml-2">(click to filter)</span>
</h3>
<ResponsiveContainer width="100%" height={250}>
<PieChart>
<Pie
data={[
{ name: 'Positive', value: stats.sentimentBreakdown.positive, sentiment: 'positive' },
{ name: 'Neutral', value: stats.sentimentBreakdown.neutral, sentiment: 'neutral' },
{ name: 'Negative', value: stats.sentimentBreakdown.negative, sentiment: 'negative' },
]}
cx="50%"
cy="50%"
labelLine={false}
label={({ name, percent }) => `${name} ${(percent * 100).toFixed(0)}%`}
outerRadius={80}
fill="#8884d8"
dataKey="value"
style={{ fontWeight: 700, fontSize: '13px', cursor: 'pointer' }}
onClick={(data) => {
if (data && data.sentiment) {
setSelectedSentiments([data.sentiment as 'positive' | 'neutral' | 'negative']);
setSelectedRatings([1, 2, 3, 4, 5]);
}
}}
>
<Cell fill={COLORS.positive} />
<Cell fill={COLORS.neutral} />
<Cell fill={COLORS.negative} />
</Pie>
<Tooltip
contentStyle={{
backgroundColor: '#ffffff',
border: '2px solid #3b82f6',
borderRadius: '8px',
fontWeight: 600
}}
content={({ active, payload }) => {
if (active && payload && payload.length) {
return (
<div className="bg-white border-2 border-blue-600 rounded-lg p-2 shadow-lg">
<p className="font-bold text-gray-900">{payload[0].name}</p>
<p className="text-sm text-gray-600">{payload[0].value} reviews</p>
<p className="text-xs text-blue-600 mt-1">Click to filter</p>
</div>
);
}
return null;
}}
/>
</PieChart>
</ResponsiveContainer>
</div>
{/* Top Keywords */}
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
<h3 className="text-lg font-bold mb-4 text-gray-900">Top Keywords</h3>
<ResponsiveContainer width="100%" height={250}>
<BarChart data={stats.topKeywords} layout="vertical">
<CartesianGrid strokeDasharray="3 3" stroke="#e5e7eb" />
<XAxis
type="number"
tick={{ fill: '#374151', fontWeight: 600 }}
tickLine={{ stroke: '#9ca3af' }}
/>
<YAxis
type="category"
dataKey="word"
width={80}
tick={{ fill: '#374151', fontWeight: 600 }}
tickLine={{ stroke: '#9ca3af' }}
/>
<Tooltip
contentStyle={{
backgroundColor: '#ffffff',
border: '2px solid #3b82f6',
borderRadius: '8px',
fontWeight: 600
}}
/>
<Bar dataKey="count" fill="#8b5cf6" radius={[0, 8, 8, 0]} />
</BarChart>
</ResponsiveContainer>
</div>
</div>
{/* Reviews Table */}
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
<div className="flex items-center justify-between mb-4">
<h3 className="text-xl font-bold text-gray-900">Review Details</h3>
<button
onClick={exportFilteredData}
className="flex items-center gap-2 px-4 py-2 bg-green-600 text-white rounded-lg hover:bg-green-700 font-semibold shadow-md border-2 border-green-700"
>
<Download className="w-4 h-4" />
Export Filtered Data
</button>
</div>
{/* Search */}
<div className="mb-6">
<div className="relative">
<Search className="absolute left-3 top-1/2 transform -translate-y-1/2 w-5 h-5 text-gray-500" />
<input
type="text"
value={globalFilter}
onChange={e => setGlobalFilter(e.target.value)}
placeholder="Search by author, review text, or date..."
className="w-full pl-10 pr-4 py-3 border-2 border-gray-300 rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500 font-medium"
/>
</div>
</div>
{/* Table */}
<div className="overflow-x-auto border-2 border-gray-300 rounded-lg">
<table className="w-full">
<thead className="bg-gray-100 border-b-2 border-gray-300">
{table.getHeaderGroups().map(headerGroup => (
<tr key={headerGroup.id}>
{headerGroup.headers.map(header => (
<th key={header.id} className="px-6 py-4 text-left text-gray-900">
{header.isPlaceholder
? null
: flexRender(header.column.columnDef.header, header.getContext())}
</th>
))}
</tr>
))}
</thead>
<tbody className="divide-y-2 divide-gray-200">
{table.getRowModel().rows.map(row => (
<tr key={row.id} className="hover:bg-gray-50">
{row.getVisibleCells().map(cell => (
<td key={cell.id} className="px-6 py-4">
{flexRender(cell.column.columnDef.cell, cell.getContext())}
</td>
))}
</tr>
))}
</tbody>
</table>
</div>
{/* Pagination */}
<div className="flex items-center justify-between mt-6">
<div className="text-sm text-gray-700 font-medium">
Showing {table.getState().pagination.pageIndex * table.getState().pagination.pageSize + 1} to{' '}
{Math.min((table.getState().pagination.pageIndex + 1) * table.getState().pagination.pageSize, filteredReviews.length)} of{' '}
{filteredReviews.length} reviews
</div>
<div className="flex gap-2">
<button
onClick={() => table.previousPage()}
disabled={!table.getCanPreviousPage()}
className="px-4 py-2 border-2 border-gray-300 rounded-lg disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50 font-semibold text-gray-900"
>
Previous
</button>
<button
onClick={() => table.nextPage()}
disabled={!table.getCanNextPage()}
className="px-4 py-2 border-2 border-gray-300 rounded-lg disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50 font-semibold text-gray-900"
>
Next
</button>
</div>
</div>
</div>
</div>
);
}

View File

@@ -0,0 +1,909 @@
'use client';
import { useState, useEffect, useRef } from 'react';
import ReviewAnalytics from './ReviewAnalytics';
interface Review {
author: string;
rating: number;
text: string | null;
date_text: string;
avatar_url: string | null;
profile_url: string | null;
review_id: string;
}
interface JobStatus {
job_id: string;
status: 'pending' | 'running' | 'completed' | 'failed';
url: string;
created_at: string;
started_at: string | null;
completed_at: string | null;
updated_at: string | null; // Last update time for progress tracking
reviews_count: number | null;
total_reviews: number | null;
scrape_time: number | null;
error_message: string | null;
}
export default function ScraperTest() {
const [searchQuery, setSearchQuery] = useState('');
const [searchedQuery, setSearchedQuery] = useState('');
const [jobs, setJobs] = useState<Map<string, JobStatus>>(new Map());
const [activeJobId, setActiveJobId] = useState<string | null>(null);
const [reviews, setReviews] = useState<Review[]>([]);
const [error, setError] = useState('');
const [isSubmitting, setIsSubmitting] = useState(false);
const [showAnalytics, setShowAnalytics] = useState(false);
const [isLoadingReviews, setIsLoadingReviews] = useState(false);
const [showConfirmModal, setShowConfirmModal] = useState(false);
const [isCheckingReviews, setIsCheckingReviews] = useState(false);
const [hasReviews, setHasReviews] = useState<boolean | null>(null);
const [availableReviewCount, setAvailableReviewCount] = useState<number | null>(null);
const [businessName, setBusinessName] = useState<string | null>(null);
const [businessAddress, setBusinessAddress] = useState<string | null>(null);
const [businessRating, setBusinessRating] = useState<number | null>(null);
const debounceRef = useRef<NodeJS.Timeout | null>(null);
const pollingIntervals = useRef<Map<string, NodeJS.Timeout>>(new Map());
const abortControllerRef = useRef<AbortController | null>(null);
// Debounce: update map preview as user types (500ms after stopping)
useEffect(() => {
if (searchQuery.trim().length >= 2) {
if (debounceRef.current) {
clearTimeout(debounceRef.current);
}
debounceRef.current = setTimeout(() => {
setSearchedQuery(searchQuery.trim());
}, 500);
return () => {
if (debounceRef.current) {
clearTimeout(debounceRef.current);
}
};
}
}, [searchQuery]);
// Clear validation results when user starts typing a new search
useEffect(() => {
// If searchQuery is different from searchedQuery, clear results
if (searchQuery.trim() !== searchedQuery && searchedQuery) {
// Abort any pending validation request
if (abortControllerRef.current) {
abortControllerRef.current.abort();
}
setHasReviews(null);
setAvailableReviewCount(null);
setBusinessName(null);
setBusinessAddress(null);
setBusinessRating(null);
}
}, [searchQuery, searchedQuery]);
// Check for reviews function (called manually when user clicks Validate)
const checkReviews = async (query: string) => {
// Abort any previous validation request
if (abortControllerRef.current) {
abortControllerRef.current.abort();
}
setIsCheckingReviews(true);
setHasReviews(null);
setAvailableReviewCount(null);
setBusinessName(null);
setBusinessAddress(null);
setBusinessRating(null);
setError('');
// Create new abort controller with 30 second timeout
const controller = new AbortController();
abortControllerRef.current = controller;
const timeoutId = setTimeout(() => controller.abort(), 30000);
try {
const url = `https://www.google.com/maps/search/?api=1&query=${encodeURIComponent(query)}`;
const response = await fetch('/api/check-reviews', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ url }),
signal: controller.signal,
});
clearTimeout(timeoutId);
const data = await response.json();
if (response.ok && data.success) {
setHasReviews(data.has_reviews);
setAvailableReviewCount(data.total_reviews || 0);
setBusinessName(data.name);
setBusinessAddress(data.address);
setBusinessRating(data.rating);
} else {
console.error('Failed to get business info:', data.error);
// Business not found
setHasReviews(false);
setAvailableReviewCount(0);
}
} catch (err) {
clearTimeout(timeoutId);
// Ignore AbortError (happens when user starts a new validation)
if (err instanceof Error && err.name === 'AbortError') {
console.log('Validation cancelled (new validation started)');
return;
}
console.error('Error getting business info:', err);
// Error occurred
setHasReviews(false);
setAvailableReviewCount(0);
} finally {
// Only clear loading state if this controller wasn't aborted
if (!controller.signal.aborted) {
setIsCheckingReviews(false);
}
}
};
// Poll job status for all active jobs
const startPolling = (jobId: string) => {
// Don't start if already polling this job
if (pollingIntervals.current.has(jobId)) return;
const pollInterval = setInterval(async () => {
try {
const response = await fetch(`/api/jobs/${jobId}`);
const data = await response.json();
// Update job in map
setJobs(prev => {
const newMap = new Map(prev);
newMap.set(jobId, data);
return newMap;
});
// Stop polling if job is done
if (data.status === 'completed' || data.status === 'failed') {
const interval = pollingIntervals.current.get(jobId);
if (interval) {
clearInterval(interval);
pollingIntervals.current.delete(jobId);
}
}
} catch (err) {
console.error('Poll error for job', jobId, err);
}
}, 2000); // Poll every 2 seconds
pollingIntervals.current.set(jobId, pollInterval);
};
// Cleanup polling intervals and abort controllers on unmount
useEffect(() => {
return () => {
pollingIntervals.current.forEach(interval => clearInterval(interval));
pollingIntervals.current.clear();
if (abortControllerRef.current) {
abortControllerRef.current.abort();
}
};
}, []);
const handleSearch = () => {
if (searchQuery.trim().length < 2) return;
const query = searchQuery.trim();
// Clear any pending debounce
if (debounceRef.current) {
clearTimeout(debounceRef.current);
}
// Immediately update map preview and trigger validation
setSearchedQuery(query);
checkReviews(query);
};
const handlePreviewBusiness = (e: React.FormEvent) => {
e.preventDefault();
setShowConfirmModal(true);
};
const handleConfirmScrape = async () => {
setError('');
setIsSubmitting(true);
setShowConfirmModal(false);
// Use the search query to create a Google Maps search URL
const url = `https://www.google.com/maps/search/?api=1&query=${encodeURIComponent(searchedQuery)}`;
try {
const response = await fetch('/api/scrape', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ url }),
});
const data = await response.json();
if (!response.ok) {
throw new Error(data.error || 'Failed to start scraping');
}
// Add job to Map with initial status
setJobs(prev => {
const newMap = new Map(prev);
newMap.set(data.job_id, {
job_id: data.job_id,
status: 'pending',
url: url,
created_at: new Date().toISOString(),
started_at: null,
completed_at: null,
reviews_count: null,
total_reviews: null,
scrape_time: null,
error_message: null,
});
return newMap;
});
// Set as active job and start polling
setActiveJobId(data.job_id);
startPolling(data.job_id);
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to submit job');
} finally {
setIsSubmitting(false);
}
};
const getStatusColor = (status: string) => {
switch (status) {
case 'completed': return 'text-green-700';
case 'running': return 'text-blue-700';
case 'failed': return 'text-red-700';
default: return 'text-gray-800';
}
};
const getStatusIcon = (status: string) => {
switch (status) {
case 'completed':
return (
<svg className="w-5 h-5 text-green-500" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
</svg>
);
case 'running':
return <div className="w-5 h-5 border-2 border-blue-500 border-t-transparent rounded-full animate-spin" />;
case 'failed':
return (
<svg className="w-5 h-5 text-red-500" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zM8.707 7.293a1 1 0 00-1.414 1.414L8.586 10l-1.293 1.293a1 1 0 101.414 1.414L10 11.414l1.293 1.293a1 1 0 001.414-1.414L11.414 10l1.293-1.293a1 1 0 00-1.414-1.414L10 8.586 8.707 7.293z" clipRule="evenodd" />
</svg>
);
default:
return (
<svg className="w-5 h-5 text-gray-400" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm1-12a1 1 0 10-2 0v4a1 1 0 00.293.707l2.828 2.829a1 1 0 101.415-1.415L11 9.586V6z" clipRule="evenodd" />
</svg>
);
}
};
const embedUrl = searchedQuery
? `https://maps.google.com/maps?q=${encodeURIComponent(searchedQuery)}&output=embed&z=15`
: '';
const [mapClicked, setMapClicked] = useState(false);
const searchInputRef = useRef<HTMLInputElement>(null);
const handleMapClick = () => {
setMapClicked(true);
};
const closeModal = () => {
setMapClicked(false);
};
const focusSearchBar = () => {
setMapClicked(false);
searchInputRef.current?.focus();
};
return (
<div className="w-full max-w-4xl mx-auto">
{/* Search Interface */}
<>
<div className="mb-4 flex gap-2">
<div className="relative flex-1">
<div className="absolute left-4 top-1/2 -translate-y-1/2 text-gray-400">
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z" />
</svg>
</div>
<input
ref={searchInputRef}
type="text"
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter' && searchQuery.trim().length >= 2 && !isCheckingReviews) {
e.preventDefault();
handleSearch();
}
}}
placeholder="Business name and location (e.g., Soho Club Vilnius)..."
className="w-full pl-12 pr-4 py-3 text-gray-900 bg-white border-2 border-gray-200 rounded-xl focus:border-blue-500 focus:ring-4 focus:ring-blue-100 outline-none transition-all"
/>
</div>
<button
onClick={handleSearch}
disabled={searchQuery.trim().length < 2 || isCheckingReviews}
className="px-6 py-3 bg-blue-600 text-white font-semibold rounded-xl hover:bg-blue-700 disabled:bg-gray-300 disabled:cursor-not-allowed transition-colors flex items-center gap-2"
>
{isCheckingReviews ? (
<>
<div className="w-4 h-4 border-2 border-white border-t-transparent rounded-full animate-spin" />
Validating...
</>
) : (
<>
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M9 12l2 2 4-4m6 2a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
Validate
</>
)}
</button>
</div>
{/* Map Preview with Click Overlay */}
<div className="mb-4 rounded-xl overflow-hidden border-2 border-gray-200 bg-gray-100 relative">
{searchedQuery ? (
<>
<iframe
src={embedUrl}
width="100%"
height="350"
style={{ border: 0, pointerEvents: 'none' }}
allowFullScreen
loading="lazy"
referrerPolicy="no-referrer-when-downgrade"
title="Google Maps"
/>
{/* Click detection overlay - always present to capture clicks */}
<div
className="absolute inset-0 cursor-pointer"
onClick={handleMapClick}
/>
{/* Modal centered on map card */}
{mapClicked && (
<div
className="absolute inset-0 flex items-center justify-center backdrop-blur-md bg-gray-900/30 p-4"
onClick={closeModal}
>
<div
className="bg-white rounded-2xl p-6 sm:p-8 shadow-2xl w-full max-w-md border-2 border-blue-500 animate-fade-in"
onClick={(e) => e.stopPropagation()}
>
<div className="text-center mb-4 sm:mb-6">
<div className="text-4xl sm:text-5xl mb-2 sm:mb-3">🎯</div>
<p className="text-xl sm:text-2xl font-bold text-gray-900 mb-2">Want a specific business?</p>
<p className="text-xs sm:text-sm text-gray-600">
Search for the <strong>exact business name</strong> to scrape its reviews
</p>
</div>
<div className="bg-blue-50 border-2 border-blue-200 rounded-lg p-3 mb-4">
<p className="text-xs text-blue-900 font-medium mb-1">💡 Example:</p>
<p className="text-sm font-semibold text-blue-800">"Starbucks Downtown Seattle"</p>
<p className="text-xs text-gray-500 mt-1">instead of just "coffee"</p>
</div>
<div className="flex gap-2">
<button
onClick={focusSearchBar}
className="flex-1 py-3 bg-blue-600 hover:bg-blue-700 text-white rounded-lg font-bold transition-all flex items-center justify-center gap-2 shadow-md"
>
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z" />
</svg>
Search
</button>
<button
onClick={closeModal}
className="px-4 py-3 bg-gray-200 hover:bg-gray-300 text-gray-700 rounded-lg font-bold transition-all"
>
</button>
</div>
</div>
</div>
)}
</>
) : (
<div className="h-[350px] flex items-center justify-center text-gray-400">
<div className="text-center">
<svg className="w-12 h-12 mx-auto mb-3 opacity-50" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M17.657 16.657L13.414 20.9a1.998 1.998 0 01-2.827 0l-4.244-4.243a8 8 0 1111.314 0z" />
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M15 11a3 3 0 11-6 0 3 3 0 016 0z" />
</svg>
<p>Search for a business to see the map</p>
</div>
</div>
)}
</div>
{/* Business Card - Validation Results */}
{searchedQuery && hasReviews !== null && (
<div className="mb-6">
{hasReviews ? (
// Success - Show Business Card
<div className="bg-white border-2 border-green-500 rounded-2xl shadow-lg overflow-hidden mb-4">
{/* Header */}
<div className="bg-gradient-to-r from-green-500 to-emerald-500 px-6 py-4">
<div className="flex items-center gap-2 text-white">
<svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M5 13l4 4L19 7" />
</svg>
<span className="font-bold text-lg">Business Found</span>
</div>
</div>
{/* Business Info */}
<div className="p-6">
{/* Business Name */}
<h3 className="text-2xl font-bold text-gray-900 mb-3">{businessName}</h3>
{/* Rating */}
{businessRating && (
<div className="flex items-center gap-1 mb-3">
<span className="text-2xl font-bold text-gray-900">{businessRating.toFixed(1)}</span>
<div className="flex items-center ml-1">
{[...Array(5)].map((_, i) => (
<svg
key={i}
className={`w-5 h-5 ${i < Math.floor(businessRating) ? 'text-yellow-400' : 'text-gray-300'}`}
fill="currentColor"
viewBox="0 0 20 20"
>
<path d="M9.049 2.927c.3-.921 1.603-.921 1.902 0l1.07 3.292a1 1 0 00.95.69h3.462c.969 0 1.371 1.24.588 1.81l-2.8 2.034a1 1 0 00-.364 1.118l1.07 3.292c.3.921-.755 1.688-1.54 1.118l-2.8-2.034a1 1 0 00-1.175 0l-2.8 2.034c-.784.57-1.838-.197-1.539-1.118l1.07-3.292a1 1 0 00-.364-1.118L2.98 8.72c-.783-.57-.38-1.81.588-1.81h3.461a1 1 0 00.951-.69l1.07-3.292z" />
</svg>
))}
</div>
</div>
)}
{/* Address */}
{businessAddress && (
<div className="flex items-start gap-2 text-gray-600 mb-4">
<span className="text-lg">📍</span>
<span className="text-sm">{businessAddress}</span>
</div>
)}
{/* Start Scraping Button */}
<form onSubmit={handlePreviewBusiness}>
<button
type="submit"
disabled={isSubmitting}
className="w-full py-4 bg-gradient-to-r from-green-600 to-emerald-600 hover:from-green-700 hover:to-emerald-700 text-white rounded-xl font-bold transition-all flex items-center justify-center gap-2 shadow-lg text-lg disabled:opacity-50 disabled:cursor-not-allowed"
>
{isSubmitting ? (
<>
<div className="w-5 h-5 border-2 border-white border-t-transparent rounded-full animate-spin" />
Starting scrape...
</>
) : (
<>
<svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M13 10V3L4 14h7v7l9-11h-7z" />
</svg>
Start Scraping Reviews
</>
)}
</button>
</form>
</div>
</div>
) : (
// No Reviews - Show Warning
<div className="p-4 bg-yellow-50 border-2 border-yellow-300 rounded-xl">
<div className="flex items-start gap-3">
<div className="w-10 h-10 bg-yellow-500 rounded-lg flex items-center justify-center flex-shrink-0">
<svg className="w-6 h-6 text-white" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-3L13.732 4c-.77-1.333-2.694-1.333-3.464 0L3.34 16c-.77 1.333.192 3 1.732 3z" />
</svg>
</div>
<div className="flex-1">
<p className="font-bold text-yellow-900 text-lg">No reviews available</p>
{businessName && (
<p className="text-sm text-yellow-800 mt-1">
Business: <strong>{businessName}</strong>
</p>
)}
<p className="text-xs text-yellow-700 mt-1">
This business has no reviews to scrape. Try a different search.
</p>
</div>
</div>
</div>
)}
</div>
)}
</>
{/* Error */}
{error && (
<div className="mb-6 p-4 bg-red-100 border-2 border-red-300 rounded-xl">
<div className="flex items-start gap-3">
<svg className="w-6 h-6 text-red-700 flex-shrink-0 mt-0.5" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zM8.707 7.293a1 1 0 00-1.414 1.414L8.586 10l-1.293 1.293a1 1 0 101.414 1.414L10 11.414l1.293 1.293a1 1 0 001.414-1.414L11.414 10l1.293-1.293a1 1 0 00-1.414-1.414L10 8.586 8.707 7.293z" clipRule="evenodd" />
</svg>
<div>
<p className="font-bold text-red-900 text-lg">Error</p>
<p className="text-red-800 mt-1">{error}</p>
</div>
</div>
</div>
)}
{/* Jobs List */}
{jobs.size > 0 && (
<div className="mb-6 space-y-4">
<div className="flex items-center justify-between mb-2">
<h2 className="text-2xl font-bold text-gray-900">
Scraping Jobs
</h2>
<span className="px-3 py-1 bg-blue-100 text-blue-800 font-semibold rounded-full text-sm">
{jobs.size} {jobs.size === 1 ? 'Job' : 'Jobs'}
</span>
</div>
{Array.from(jobs.values())
.sort((a, b) => new Date(b.created_at).getTime() - new Date(a.created_at).getTime())
.map(job => (
<div
key={job.job_id}
className={`p-6 rounded-xl transition-all shadow-md ${
job.job_id === activeJobId
? 'bg-blue-50 border-2 border-blue-500 shadow-lg'
: 'bg-white border-2 border-gray-300'
}`}
>
<div className="flex items-start justify-between mb-4">
<div className="flex-1">
<div className="flex items-center gap-2 mb-3">
{getStatusIcon(job.status)}
<h3 className="text-lg font-bold text-gray-900">
Status: <span className={`${getStatusColor(job.status)} font-extrabold`}>{job.status.toUpperCase()}</span>
</h3>
</div>
<p className="text-xs font-mono text-gray-600 mb-2 bg-gray-100 px-2 py-1 rounded inline-block">{job.job_id}</p>
<p className="text-sm text-gray-700 truncate max-w-2xl font-medium">{job.url}</p>
</div>
</div>
{/* Progress Bar for Running Jobs */}
{job.status === 'running' && job.total_reviews !== null && job.reviews_count !== null && (
<div className="mb-4 p-4 bg-blue-50 border-2 border-blue-200 rounded-lg">
<div className="flex items-center justify-between mb-2">
<span className="text-sm font-bold text-blue-900">Extracting Reviews</span>
<span className="text-sm font-bold text-blue-700">
{job.reviews_count} / {job.total_reviews}
</span>
</div>
<div className="w-full bg-blue-200 rounded-full h-3 overflow-hidden">
<div
className="bg-gradient-to-r from-blue-500 to-indigo-600 h-3 rounded-full transition-all duration-500 ease-out flex items-center justify-end pr-1"
style={{ width: `${Math.min((job.reviews_count / job.total_reviews) * 100, 100)}%` }}
>
{job.reviews_count > 0 && (
<span className="text-xs font-bold text-white drop-shadow">
{Math.round((job.reviews_count / job.total_reviews) * 100)}%
</span>
)}
</div>
</div>
</div>
)}
<div className="grid grid-cols-2 md:grid-cols-4 gap-4 mb-4">
{job.reviews_count !== null && (
<div className="p-4 bg-blue-100 border-2 border-blue-200 rounded-lg">
<div className="text-3xl font-bold text-blue-800">{job.reviews_count}</div>
<div className="text-xs font-semibold text-blue-700 mt-1">Reviews</div>
</div>
)}
{job.scrape_time !== null && (
<div className="p-4 bg-green-100 border-2 border-green-200 rounded-lg">
<div className="text-3xl font-bold text-green-800">{job.scrape_time.toFixed(1)}s</div>
<div className="text-xs font-semibold text-green-700 mt-1">Time</div>
</div>
)}
{job.scrape_time && job.reviews_count && (
<div className="p-4 bg-purple-100 border-2 border-purple-200 rounded-lg">
<div className="text-3xl font-bold text-purple-800">
{(job.reviews_count / job.scrape_time).toFixed(1)}
</div>
<div className="text-xs font-semibold text-purple-700 mt-1">Reviews/sec</div>
</div>
)}
{job.started_at && (
<div className="p-4 bg-gray-100 border-2 border-gray-300 rounded-lg">
<div className="text-lg font-bold text-gray-800">
{new Date(job.started_at).toLocaleTimeString()}
</div>
<div className="text-xs font-semibold text-gray-700 mt-1">Started</div>
</div>
)}
{job.status === 'running' && job.updated_at && (
<div className="p-4 bg-blue-100 border-2 border-blue-200 rounded-lg">
<div className="text-lg font-bold text-blue-800">
{new Date(job.updated_at).toLocaleTimeString()}
</div>
<div className="text-xs font-semibold text-blue-700 mt-1">Last Update</div>
</div>
)}
</div>
{/* Action Buttons - Show when completed */}
{job.status === 'completed' && (
<div className="flex gap-3">
<button
onClick={async () => {
setError('');
setIsLoadingReviews(true);
try {
console.log('Fetching reviews for job:', job.job_id);
const reviewsResponse = await fetch(`/api/jobs/${job.job_id}/reviews?limit=10000`);
if (!reviewsResponse.ok) {
throw new Error(`Failed to fetch reviews: ${reviewsResponse.status}`);
}
const reviewsData = await reviewsResponse.json();
console.log('Reviews fetched:', reviewsData);
if (!reviewsData.reviews || reviewsData.reviews.length === 0) {
setError('No reviews found for this job');
setIsLoadingReviews(false);
return;
}
setReviews(reviewsData.reviews);
setActiveJobId(job.job_id);
setShowAnalytics(true);
} catch (err) {
console.error('Failed to fetch reviews:', err);
setError(err instanceof Error ? err.message : 'Failed to load reviews for analysis');
} finally {
setIsLoadingReviews(false);
}
}}
disabled={isLoadingReviews}
className="flex-1 py-4 bg-gradient-to-r from-blue-600 to-indigo-700 text-white rounded-xl font-bold hover:from-blue-700 hover:to-indigo-800 transition-all flex items-center justify-center gap-2 shadow-lg disabled:opacity-50 disabled:cursor-not-allowed text-lg border-2 border-blue-500"
>
{isLoadingReviews ? (
<>
<div className="w-5 h-5 border-2 border-white border-t-transparent rounded-full animate-spin" />
Loading Reviews...
</>
) : (
<>
<svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M9 19v-6a2 2 0 00-2-2H5a2 2 0 00-2 2v6a2 2 0 002 2h2a2 2 0 002-2zm0 0V9a2 2 0 012-2h2a2 2 0 012 2v10m-6 0a2 2 0 002 2h2a2 2 0 002-2m0 0V5a2 2 0 012-2h2a2 2 0 012 2v14a2 2 0 01-2 2h-2a2 2 0 01-2-2z" />
</svg>
📊 Open Analytics Dashboard
</>
)}
</button>
<button
onClick={async () => {
try {
const reviewsResponse = await fetch(`/api/jobs/${job.job_id}/reviews?limit=10000`);
if (reviewsResponse.ok) {
const reviewsData = await reviewsResponse.json();
const data = JSON.stringify(reviewsData.reviews, null, 2);
const blob = new Blob([data], { type: 'application/json' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = `reviews-${job.job_id}.json`;
a.click();
}
} catch (err) {
console.error('Failed to export reviews:', err);
}
}}
className="px-6 py-4 bg-gray-700 hover:bg-gray-800 text-white border-2 border-gray-600 rounded-xl font-bold transition-colors flex items-center justify-center gap-2 shadow-md"
>
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 16v1a3 3 0 003 3h10a3 3 0 003-3v-1m-4-4l-4 4m0 0l-4-4m4 4V4" />
</svg>
Export JSON
</button>
</div>
)}
{/* Error Message */}
{job.status === 'failed' && job.error_message && (
<div className="mt-4 p-4 bg-red-100 border-2 border-red-300 rounded-lg">
<div className="flex items-start gap-2">
<svg className="w-5 h-5 text-red-700 flex-shrink-0 mt-0.5" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zM8.707 7.293a1 1 0 00-1.414 1.414L8.586 10l-1.293 1.293a1 1 0 101.414 1.414L10 11.414l1.293 1.293a1 1 0 001.414-1.414L11.414 10l1.293-1.293a1 1 0 00-1.414-1.414L10 8.586 8.707 7.293z" clipRule="evenodd" />
</svg>
<div>
<p className="font-bold text-red-900">Error</p>
<p className="text-sm text-red-800 mt-1">{job.error_message}</p>
</div>
</div>
</div>
)}
</div>
))}
</div>
)}
{/* Analytics Dashboard or Simple Review List */}
{reviews.length > 0 && (
<>
{showAnalytics ? (
<div>
<div className="mb-4">
<button
onClick={() => setShowAnalytics(false)}
className="flex items-center gap-2 text-blue-600 hover:text-blue-700 font-medium"
>
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M10 19l-7-7m0 0l7-7m-7 7h18" />
</svg>
Back to Simple View
</button>
</div>
<ReviewAnalytics reviews={reviews} businessName={searchedQuery || 'Business'} />
</div>
) : (
<div>
<div className="flex items-center justify-between mb-4">
<h3 className="text-xl font-bold text-gray-900">
Reviews ({reviews.length})
</h3>
<button
onClick={() => {
const data = JSON.stringify(reviews, null, 2);
const blob = new Blob([data], { type: 'application/json' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = `reviews-${activeJobId || 'export'}.json`;
a.click();
}}
className="px-4 py-2 text-sm bg-gray-700 hover:bg-gray-800 text-white border-2 border-gray-600 rounded-lg font-bold transition-colors shadow-md"
>
Export JSON
</button>
</div>
<div className="space-y-3 max-h-[600px] overflow-y-auto pr-2">
{reviews.map((review, index) => (
<div key={`${index}-${review.review_id}`} className="p-4 bg-white border border-gray-200 rounded-xl hover:border-gray-300 transition-colors">
<div className="flex items-start gap-3">
{review.avatar_url && (
<img
src={review.avatar_url}
alt={review.author}
className="w-10 h-10 rounded-full"
/>
)}
<div className="flex-1 min-w-0">
<div className="flex items-center justify-between mb-1">
<span className="font-medium text-gray-900">{review.author}</span>
<div className="flex items-center gap-1">
{[...Array(5)].map((_, i) => (
<svg
key={i}
className={`w-4 h-4 ${i < review.rating ? 'text-yellow-400' : 'text-gray-300'}`}
fill="currentColor"
viewBox="0 0 20 20"
>
<path d="M9.049 2.927c.3-.921 1.603-.921 1.902 0l1.07 3.292a1 1 0 00.95.69h3.462c.969 0 1.371 1.24.588 1.81l-2.8 2.034a1 1 0 00-.364 1.118l1.07 3.292c.3.921-.755 1.688-1.54 1.118l-2.8-2.034a1 1 0 00-1.175 0l-2.8 2.034c-.784.57-1.838-.197-1.539-1.118l1.07-3.292a1 1 0 00-.364-1.118L2.98 8.72c-.783-.57-.38-1.81.588-1.81h3.461a1 1 0 00.951-.69l1.07-3.292z" />
</svg>
))}
</div>
</div>
<p className="text-xs text-gray-500 mb-2">{review.date_text}</p>
{review.text && (
<p className="text-sm text-gray-700 leading-relaxed">{review.text}</p>
)}
</div>
</div>
</div>
))}
</div>
</div>
)}
</>
)}
{/* Confirmation Modal */}
{showConfirmModal && (
<div
className="fixed inset-0 z-50 flex items-center justify-center bg-black/50 backdrop-blur-sm p-4"
onClick={() => setShowConfirmModal(false)}
>
<div
className="bg-white rounded-2xl shadow-2xl w-full max-w-md border-2 border-green-500 animate-fade-in"
onClick={(e) => e.stopPropagation()}
>
{/* Header */}
<div className="bg-gradient-to-r from-green-600 to-emerald-600 text-white px-6 py-5 rounded-t-xl">
<div className="flex items-center gap-3">
<div className="w-10 h-10 bg-white/20 rounded-lg flex items-center justify-center">
<svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M13 10V3L4 14h7v7l9-11h-7z" />
</svg>
</div>
<h2 className="text-xl font-bold">Start Scraping?</h2>
</div>
</div>
{/* Content */}
<div className="px-6 py-5">
<p className="text-gray-700 mb-4">
This will start scraping reviews for:
</p>
<div className="bg-green-50 border-2 border-green-200 rounded-lg p-4 mb-4">
<p className="font-bold text-green-900 text-lg">{businessName}</p>
{businessAddress && (
<p className="text-sm text-green-700 mt-1">{businessAddress}</p>
)}
</div>
<p className="text-sm text-gray-600">
The scraping job will run in the background. You can monitor progress below.
</p>
</div>
{/* Actions */}
<div className="px-6 py-4 bg-gray-50 rounded-b-xl border-t-2 border-gray-200 flex gap-3">
<button
onClick={() => setShowConfirmModal(false)}
className="flex-1 py-3 px-4 bg-gray-200 hover:bg-gray-300 text-gray-800 rounded-lg font-semibold transition-all"
>
Cancel
</button>
<button
onClick={handleConfirmScrape}
disabled={isSubmitting}
className="flex-1 py-3 px-4 bg-gradient-to-r from-green-600 to-emerald-600 hover:from-green-700 hover:to-emerald-700 text-white rounded-lg font-semibold transition-all flex items-center justify-center gap-2 disabled:opacity-50"
>
{isSubmitting ? (
<>
<div className="w-4 h-4 border-2 border-white border-t-transparent rounded-full animate-spin" />
Starting...
</>
) : (
<>
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M5 13l4 4L19 7" />
</svg>
Confirm
</>
)}
</button>
</div>
</div>
</div>
)}
</div>
);
}

18
web/eslint.config.mjs Normal file
View File

@@ -0,0 +1,18 @@
import { defineConfig, globalIgnores } from "eslint/config";
import nextVitals from "eslint-config-next/core-web-vitals";
import nextTs from "eslint-config-next/typescript";
const eslintConfig = defineConfig([
...nextVitals,
...nextTs,
// Override default ignores of eslint-config-next.
globalIgnores([
// Default ignores of eslint-config-next:
".next/**",
"out/**",
"build/**",
"next-env.d.ts",
]),
]);
export default eslintConfig;

398
web/lib/analytics.ts Normal file
View File

@@ -0,0 +1,398 @@
// Analytics utility functions
export interface Review {
author: string;
rating: number;
text: string | null;
date_text: string;
avatar_url: string | null;
profile_url: string | null;
review_id: string;
// Derived fields (computed on load)
parsedDate?: Date;
dateCategory?: 'recent' | 'month' | 'year' | 'older'; // Time range category
minDate?: Date; // Earliest possible date (accounting for Google's uncertainty)
maxDate?: Date; // Latest possible date (accounting for Google's uncertainty)
centerDate?: Date; // Midpoint of the range (used for calculations and sorting)
}
export type DateRange = 'week' | 'month' | 'year' | 'all';
export interface TimelineDataPoint {
date: string;
rating: number;
rollingAvg: number;
}
export interface ReviewStats {
totalReviews: number;
averageRating: number;
sentimentScore: number;
photoCount: number;
avgReviewLength: number;
recentReviews: number;
ratingDistribution: { rating: number; count: number; percentage: number }[];
topKeywords: { word: string; count: number }[];
sentimentBreakdown: { positive: number; neutral: number; negative: number };
negativeReviews: number;
responseRate: number;
averageResponseTime: string;
}
export function calculateReviewStats(reviews: Review[]): ReviewStats {
// Populate minDate/maxDate/centerDate on reviews for display
reviews.forEach(r => {
if (!r.minDate || !r.maxDate || !r.centerDate) {
const range = parseDateTextToRange(r.date_text);
r.minDate = range.minDate;
r.maxDate = range.maxDate;
// Calculate centerDate as midpoint
const midpointTime = (range.minDate.getTime() + range.maxDate.getTime()) / 2;
r.centerDate = new Date(midpointTime);
}
});
const totalReviews = reviews.length;
// Average rating
const averageRating = reviews.reduce((sum, r) => sum + r.rating, 0) / totalReviews;
// Sentiment score (% of 4-5 star reviews)
const positiveReviews = reviews.filter(r => r.rating >= 4).length;
const sentimentScore = (positiveReviews / totalReviews) * 100;
// Photo count (reviews with avatars as proxy)
const photoCount = reviews.filter(r => r.avatar_url).length;
// Average review length
const avgReviewLength = Math.round(
reviews.reduce((sum, r) => sum + (r.text?.split(' ').length || 0), 0) / totalReviews
);
// Recent reviews (last 30 days - simplified check)
const recentReviews = reviews.filter(r => {
const text = r.date_text.toLowerCase();
return text.includes('day') || text.includes('week') || text.includes('hour');
}).length;
// Rating distribution
const ratingCounts: Record<number, number> = { 1: 0, 2: 0, 3: 0, 4: 0, 5: 0 };
reviews.forEach(r => {
ratingCounts[r.rating] = (ratingCounts[r.rating] || 0) + 1;
});
const ratingDistribution = [5, 4, 3, 2, 1].map(rating => ({
rating,
count: ratingCounts[rating] || 0,
percentage: ((ratingCounts[rating] || 0) / totalReviews) * 100,
}));
// Extract keywords from review text
const allWords = reviews
.filter(r => r.text)
.flatMap(r =>
r.text!
.toLowerCase()
.replace(/[^\w\s]/g, '')
.split(/\s+/)
.filter(w => w.length > 3)
);
const stopWords = new Set(['this', 'that', 'with', 'from', 'have', 'been', 'were', 'very', 'great', 'good', 'best', 'nice', 'here', 'there', 'they', 'their', 'about', 'would', 'could', 'should', 'place', 'really']);
const wordCounts: Record<string, number> = {};
allWords.forEach(word => {
if (!stopWords.has(word)) {
wordCounts[word] = (wordCounts[word] || 0) + 1;
}
});
const topKeywords = Object.entries(wordCounts)
.sort(([, a], [, b]) => b - a)
.slice(0, 10)
.map(([word, count]) => ({ word, count }));
// Sentiment breakdown
const sentimentBreakdown = {
positive: reviews.filter(r => r.rating >= 4).length,
neutral: reviews.filter(r => r.rating === 3).length,
negative: reviews.filter(r => r.rating <= 2).length,
};
// Negative reviews count
const negativeReviews = reviews.filter(r => r.rating <= 2).length;
// Response rate (placeholder - would need owner_response field)
const responseRate = 0; // TODO: Calculate when owner responses are available
// Average response time (placeholder)
const averageResponseTime = 'N/A'; // TODO: Calculate when response data is available
return {
totalReviews,
averageRating,
sentimentScore,
photoCount,
avgReviewLength,
recentReviews,
ratingDistribution,
topKeywords,
sentimentBreakdown,
negativeReviews,
responseRate,
averageResponseTime,
};
}
export function getSentimentLabel(rating: number): 'positive' | 'neutral' | 'negative' {
if (rating >= 4) return 'positive';
if (rating === 3) return 'neutral';
return 'negative';
}
// Helper function to get date range boundaries for preset buttons
export function getDateRangeBoundaries(range: DateRange): { from: Date | null; to: Date | null } {
if (range === 'all') return { from: null, to: null };
const now = new Date();
const to = new Date(now); // Today as end date
const from = new Date();
switch (range) {
case 'week':
from.setDate(now.getDate() - 7);
break;
case 'month':
from.setMonth(now.getMonth() - 1);
break;
case 'year':
from.setFullYear(now.getFullYear() - 1);
break;
}
// Set to start of day for from, end of day for to
from.setHours(0, 0, 0, 0);
to.setHours(23, 59, 59, 999);
return { from, to };
}
export function getSentimentColor(sentiment: 'positive' | 'neutral' | 'negative'): string {
switch (sentiment) {
case 'positive': return 'text-green-700 bg-green-50 border-green-300';
case 'neutral': return 'text-yellow-700 bg-yellow-50 border-yellow-300';
case 'negative': return 'text-red-700 bg-red-50 border-red-300';
}
}
function extractNumber(text: string): number {
// Extract first number from text (e.g., "2 weeks ago" -> 2, "Hace 2 semanas" -> 2)
const match = text.match(/\d+/);
if (match) {
return parseInt(match[0]);
}
// Handle singular: "a month ago", "un mes", "una semana"
if (text.match(/^a\s+\w+\s+ago/) || text.includes('un ') || text.includes('una ')) {
return 1;
}
return 1; // Default to 1 if no number found
}
/**
* Parse date_text into time range boundaries (min/max dates)
*
* This accounts for Google's inherent uncertainty in relative dates.
* Based on reverse-engineered patterns from 244 reviews.
*
* Examples:
* - "a month ago" → { min: 30 days ago, max: 59 days ago }
* - "2 months ago" → { min: 60 days ago, max: 89 days ago }
* - "a year ago" → { min: 365 days ago, max: 729 days ago }
*/
export function parseDateTextToRange(dateText: string): { minDate: Date; maxDate: Date } {
const now = new Date();
const text = dateText.toLowerCase();
// Remove "Edited " prefix if present
const cleaned = text.replace(/^edited\s+/i, '');
// Helper to create date from days ago
const daysAgo = (days: number) => new Date(now.getTime() - days * 24 * 60 * 60 * 1000);
// Seconds: 1-59 seconds
if (cleaned.includes('second')) {
const seconds = extractNumber(cleaned);
const minDate = new Date(now.getTime() - seconds * 1000);
const maxDate = new Date(now.getTime() - seconds * 1000);
return { minDate, maxDate };
}
// Minutes: 1-59 minutes
if (cleaned.includes('minute')) {
const minutes = extractNumber(cleaned);
const minDate = new Date(now.getTime() - minutes * 60 * 1000);
const maxDate = new Date(now.getTime() - minutes * 60 * 1000);
return { minDate, maxDate };
}
// Hours: 1-23 hours
if (cleaned.includes('hora') || cleaned.includes('hour')) {
const hours = extractNumber(cleaned);
const minDate = new Date(now.getTime() - hours * 60 * 60 * 1000);
const maxDate = new Date(now.getTime() - hours * 60 * 60 * 1000);
return { minDate, maxDate };
}
// Days: 1-6 days
if (cleaned.includes('día') || cleaned.includes('day')) {
const days = extractNumber(cleaned);
const minDate = daysAgo(days);
const maxDate = daysAgo(days);
return { minDate, maxDate };
}
// Weeks: 2-3 weeks (Google never shows "1 week ago" or "4 weeks ago")
if (cleaned.includes('semana') || cleaned.includes('week')) {
const weeks = extractNumber(cleaned);
// Each week pattern represents a 7-day range
const minDays = weeks * 7;
const maxDays = weeks * 7 + 6; // Up to 6 extra days before switching to next week
return { minDate: daysAgo(maxDays), maxDate: daysAgo(minDays) };
}
// Months: Singular "a month ago" or plural "2-11 months ago"
if (cleaned.includes('mes') || cleaned.includes('month')) {
const months = extractNumber(cleaned);
// "a month ago" = 30-59 days (before switching to "2 months ago")
if (months === 1) {
return { minDate: daysAgo(59), maxDate: daysAgo(30) };
}
// "2 months ago" = 60-89 days
// "3 months ago" = 90-119 days
// Pattern: N months = (N*30) to ((N+1)*30 - 1) days
const minDays = months * 30;
const maxDays = (months + 1) * 30 - 1;
return { minDate: daysAgo(maxDays), maxDate: daysAgo(minDays) };
}
// Years: Singular "a year ago" or plural "2-11 years ago"
if (cleaned.includes('año') || cleaned.includes('year')) {
const years = extractNumber(cleaned);
// "a year ago" = 365-729 days (12-24 months before switching to "2 years ago")
if (years === 1) {
return { minDate: daysAgo(729), maxDate: daysAgo(365) };
}
// "2 years ago" = 730-1094 days (24-36 months)
// Pattern: N years = (N*365) to ((N+1)*365 - 1) days
const minDays = years * 365;
const maxDays = (years + 1) * 365 - 1;
return { minDate: daysAgo(maxDays), maxDate: daysAgo(minDays) };
}
// Default: very old (10+ years)
return { minDate: daysAgo(3650 + 365), maxDate: daysAgo(3650) };
}
export function parseDateText(dateText: string): Date {
// Get the time range and return the midpoint
const { minDate, maxDate } = parseDateTextToRange(dateText);
const midpointTime = (minDate.getTime() + maxDate.getTime()) / 2;
return new Date(midpointTime);
}
export function filterReviewsByDateRange(reviews: Review[], range: DateRange): Review[] {
if (range === 'all') return reviews;
const now = new Date();
const filterStart = new Date();
switch (range) {
case 'week':
filterStart.setDate(now.getDate() - 7);
break;
case 'month':
filterStart.setMonth(now.getMonth() - 1);
break;
case 'year':
filterStart.setFullYear(now.getFullYear() - 1);
break;
}
const filterEnd = now;
// Use range overlap logic: Include review if its time range overlaps with filter range
// Review range: [minDate, maxDate]
// Filter range: [filterStart, filterEnd]
// Overlap occurs when: minDate <= filterEnd AND maxDate >= filterStart
return reviews.filter(r => {
const { minDate, maxDate } = parseDateTextToRange(r.date_text);
return minDate <= filterEnd && maxDate >= filterStart;
});
}
export function filterReviewsByCustomDateRange(reviews: Review[], fromDate: Date | null, toDate: Date | null): Review[] {
if (!fromDate && !toDate) return reviews;
return reviews.filter(r => {
const reviewDate = parseDateText(r.date_text);
// If only fromDate is set, filter reviews >= fromDate
if (fromDate && !toDate) {
return reviewDate >= fromDate;
}
// If only toDate is set, filter reviews <= toDate (end of day)
if (!fromDate && toDate) {
const endOfDay = new Date(toDate);
endOfDay.setHours(23, 59, 59, 999);
return reviewDate <= endOfDay;
}
// Both dates set - filter reviews within range
const endOfDay = new Date(toDate!);
endOfDay.setHours(23, 59, 59, 999);
return reviewDate >= fromDate! && reviewDate <= endOfDay;
});
}
export function calculateTimelineData(reviews: Review[]): TimelineDataPoint[] {
// Sort reviews by date (newest first)
const sortedReviews = [...reviews]
.map(r => ({ ...r, parsedDate: parseDateText(r.date_text) }))
.sort((a, b) => b.parsedDate.getTime() - a.parsedDate.getTime());
// Group by month
const monthlyData: Record<string, { ratings: number[]; date: Date }> = {};
sortedReviews.forEach(review => {
const monthKey = `${review.parsedDate.getFullYear()}-${String(review.parsedDate.getMonth() + 1).padStart(2, '0')}`;
if (!monthlyData[monthKey]) {
monthlyData[monthKey] = { ratings: [], date: review.parsedDate };
}
monthlyData[monthKey].ratings.push(review.rating);
});
// Calculate averages and rolling average
const dataPoints: TimelineDataPoint[] = Object.entries(monthlyData)
.map(([monthKey, data]) => ({
date: monthKey,
rating: data.ratings.reduce((a, b) => a + b, 0) / data.ratings.length,
rollingAvg: 0, // Will calculate below
}))
.sort((a, b) => a.date.localeCompare(b.date));
// Calculate 3-month rolling average
dataPoints.forEach((point, idx) => {
const start = Math.max(0, idx - 2);
const end = idx + 1;
const window = dataPoints.slice(start, end);
point.rollingAvg = window.reduce((sum, p) => sum + p.rating, 0) / window.length;
});
return dataPoints;
}

7
web/next.config.ts Normal file
View File

@@ -0,0 +1,7 @@
import type { NextConfig } from "next";
const nextConfig: NextConfig = {
/* config options here */
};
export default nextConfig;

7
web/postcss.config.mjs Normal file
View File

@@ -0,0 +1,7 @@
const config = {
plugins: {
"@tailwindcss/postcss": {},
},
};
export default config;

1
web/public/file.svg Normal file
View File

@@ -0,0 +1 @@
<svg fill="none" viewBox="0 0 16 16" xmlns="http://www.w3.org/2000/svg"><path d="M14.5 13.5V5.41a1 1 0 0 0-.3-.7L9.8.29A1 1 0 0 0 9.08 0H1.5v13.5A2.5 2.5 0 0 0 4 16h8a2.5 2.5 0 0 0 2.5-2.5m-1.5 0v-7H8v-5H3v12a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1M9.5 5V2.12L12.38 5zM5.13 5h-.62v1.25h2.12V5zm-.62 3h7.12v1.25H4.5zm.62 3h-.62v1.25h7.12V11z" clip-rule="evenodd" fill="#666" fill-rule="evenodd"/></svg>

After

Width:  |  Height:  |  Size: 391 B

1
web/public/globe.svg Normal file
View File

@@ -0,0 +1 @@
<svg fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16"><g clip-path="url(#a)"><path fill-rule="evenodd" clip-rule="evenodd" d="M10.27 14.1a6.5 6.5 0 0 0 3.67-3.45q-1.24.21-2.7.34-.31 1.83-.97 3.1M8 16A8 8 0 1 0 8 0a8 8 0 0 0 0 16m.48-1.52a7 7 0 0 1-.96 0H7.5a4 4 0 0 1-.84-1.32q-.38-.89-.63-2.08a40 40 0 0 0 3.92 0q-.25 1.2-.63 2.08a4 4 0 0 1-.84 1.31zm2.94-4.76q1.66-.15 2.95-.43a7 7 0 0 0 0-2.58q-1.3-.27-2.95-.43a18 18 0 0 1 0 3.44m-1.27-3.54a17 17 0 0 1 0 3.64 39 39 0 0 1-4.3 0 17 17 0 0 1 0-3.64 39 39 0 0 1 4.3 0m1.1-1.17q1.45.13 2.69.34a6.5 6.5 0 0 0-3.67-3.44q.65 1.26.98 3.1M8.48 1.5l.01.02q.41.37.84 1.31.38.89.63 2.08a40 40 0 0 0-3.92 0q.25-1.2.63-2.08a4 4 0 0 1 .85-1.32 7 7 0 0 1 .96 0m-2.75.4a6.5 6.5 0 0 0-3.67 3.44 29 29 0 0 1 2.7-.34q.31-1.83.97-3.1M4.58 6.28q-1.66.16-2.95.43a7 7 0 0 0 0 2.58q1.3.27 2.95.43a18 18 0 0 1 0-3.44m.17 4.71q-1.45-.12-2.69-.34a6.5 6.5 0 0 0 3.67 3.44q-.65-1.27-.98-3.1" fill="#666"/></g><defs><clipPath id="a"><path fill="#fff" d="M0 0h16v16H0z"/></clipPath></defs></svg>

After

Width:  |  Height:  |  Size: 1.0 KiB

1
web/public/next.svg Normal file
View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 394 80"><path fill="#000" d="M262 0h68.5v12.7h-27.2v66.6h-13.6V12.7H262V0ZM149 0v12.7H94v20.4h44.3v12.6H94v21h55v12.6H80.5V0h68.7zm34.3 0h-17.8l63.8 79.4h17.9l-32-39.7 32-39.6h-17.9l-23 28.6-23-28.6zm18.3 56.7-9-11-27.1 33.7h17.8l18.3-22.7z"/><path fill="#000" d="M81 79.3 17 0H0v79.3h13.6V17l50.2 62.3H81Zm252.6-.4c-1 0-1.8-.4-2.5-1s-1.1-1.6-1.1-2.6.3-1.8 1-2.5 1.6-1 2.6-1 1.8.3 2.5 1a3.4 3.4 0 0 1 .6 4.3 3.7 3.7 0 0 1-3 1.8zm23.2-33.5h6v23.3c0 2.1-.4 4-1.3 5.5a9.1 9.1 0 0 1-3.8 3.5c-1.6.8-3.5 1.3-5.7 1.3-2 0-3.7-.4-5.3-1s-2.8-1.8-3.7-3.2c-.9-1.3-1.4-3-1.4-5h6c.1.8.3 1.6.7 2.2s1 1.2 1.6 1.5c.7.4 1.5.5 2.4.5 1 0 1.8-.2 2.4-.6a4 4 0 0 0 1.6-1.8c.3-.8.5-1.8.5-3V45.5zm30.9 9.1a4.4 4.4 0 0 0-2-3.3 7.5 7.5 0 0 0-4.3-1.1c-1.3 0-2.4.2-3.3.5-.9.4-1.6 1-2 1.6a3.5 3.5 0 0 0-.3 4c.3.5.7.9 1.3 1.2l1.8 1 2 .5 3.2.8c1.3.3 2.5.7 3.7 1.2a13 13 0 0 1 3.2 1.8 8.1 8.1 0 0 1 3 6.5c0 2-.5 3.7-1.5 5.1a10 10 0 0 1-4.4 3.5c-1.8.8-4.1 1.2-6.8 1.2-2.6 0-4.9-.4-6.8-1.2-2-.8-3.4-2-4.5-3.5a10 10 0 0 1-1.7-5.6h6a5 5 0 0 0 3.5 4.6c1 .4 2.2.6 3.4.6 1.3 0 2.5-.2 3.5-.6 1-.4 1.8-1 2.4-1.7a4 4 0 0 0 .8-2.4c0-.9-.2-1.6-.7-2.2a11 11 0 0 0-2.1-1.4l-3.2-1-3.8-1c-2.8-.7-5-1.7-6.6-3.2a7.2 7.2 0 0 1-2.4-5.7 8 8 0 0 1 1.7-5 10 10 0 0 1 4.3-3.5c2-.8 4-1.2 6.4-1.2 2.3 0 4.4.4 6.2 1.2 1.8.8 3.2 2 4.3 3.4 1 1.4 1.5 3 1.5 5h-5.8z"/></svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

1
web/public/vercel.svg Normal file
View File

@@ -0,0 +1 @@
<svg fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1155 1000"><path d="m577.3 0 577.4 1000H0z" fill="#fff"/></svg>

After

Width:  |  Height:  |  Size: 128 B

1
web/public/window.svg Normal file
View File

@@ -0,0 +1 @@
<svg fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16"><path fill-rule="evenodd" clip-rule="evenodd" d="M1.5 2.5h13v10a1 1 0 0 1-1 1h-11a1 1 0 0 1-1-1zM0 1h16v11.5a2.5 2.5 0 0 1-2.5 2.5h-11A2.5 2.5 0 0 1 0 12.5zm3.75 4.5a.75.75 0 1 0 0-1.5.75.75 0 0 0 0 1.5M7 4.75a.75.75 0 1 1-1.5 0 .75.75 0 0 1 1.5 0m1.75.75a.75.75 0 1 0 0-1.5.75.75 0 0 0 0 1.5" fill="#666"/></svg>

After

Width:  |  Height:  |  Size: 385 B