Optimize scraper performance and add fallback selectors for robustness
Performance improvements: - Validation speed: 59.71s → 10.96s (5.5x improvement) - Removed 50+ console.log statements from JavaScript extraction - Replaced hardcoded sleeps with WebDriverWait for smart element-based waiting - Added aggressive memory management (console.clear, GC, image unloading every 20 scrolls) Scraping improvements: - Increased idle detection from 6 to 12 consecutive idle scrolls for completeness - Added real-time progress updates every 5 scrolls with percentage calculation - Added crash recovery to extract partial reviews if Chrome crashes - Removed artificial 200-review limit to scrape ALL reviews Timestamp tracking: - Added updated_at field separate from started_at for progress tracking - Frontend now shows both "Started" (fixed) and "Last Update" (dynamic) Robustness improvements: - Added 5 fallback CSS selectors to handle different Google Maps page structures - Now tries: div.jftiEf.fontBodyMedium, div.jftiEf, div[data-review-id], etc. - Automatic selector detection logs which selector works for debugging Test results: - Successfully scraped 550 reviews in 150.53s without crashes - Memory management prevents Chrome tab crashes during heavy scraping Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
41
web/.gitignore
vendored
Normal file
41
web/.gitignore
vendored
Normal file
@@ -0,0 +1,41 @@
|
||||
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
|
||||
|
||||
# dependencies
|
||||
/node_modules
|
||||
/.pnp
|
||||
.pnp.*
|
||||
.yarn/*
|
||||
!.yarn/patches
|
||||
!.yarn/plugins
|
||||
!.yarn/releases
|
||||
!.yarn/versions
|
||||
|
||||
# testing
|
||||
/coverage
|
||||
|
||||
# next.js
|
||||
/.next/
|
||||
/out/
|
||||
|
||||
# production
|
||||
/build
|
||||
|
||||
# misc
|
||||
.DS_Store
|
||||
*.pem
|
||||
|
||||
# debug
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
.pnpm-debug.log*
|
||||
|
||||
# env files (can opt-in for committing if needed)
|
||||
.env*
|
||||
|
||||
# vercel
|
||||
.vercel
|
||||
|
||||
# typescript
|
||||
*.tsbuildinfo
|
||||
next-env.d.ts
|
||||
90
web/README.md
Normal file
90
web/README.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# Google Reviews Scraper - Testing Interface
|
||||
|
||||
A Next.js web interface for testing the containerized Google Reviews Scraper API.
|
||||
|
||||
## Features
|
||||
|
||||
- 🎯 **URL Input** - Paste any Google Maps business URL
|
||||
- 📊 **Real-time Status** - Live job tracking with polling
|
||||
- ⚡ **Performance Metrics** - Reviews count, time, speed
|
||||
- 📱 **Review Display** - Beautiful UI for scraped reviews
|
||||
- 💾 **Export JSON** - Download reviews as JSON
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Start the Scraper API
|
||||
|
||||
First, make sure the containerized scraper is running:
|
||||
|
||||
```bash
|
||||
cd ..
|
||||
docker-compose -f docker-compose.production.yml up -d
|
||||
```
|
||||
|
||||
The API should be running at `http://localhost:8000`
|
||||
|
||||
### 2. Start the Web Interface
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
Open [http://localhost:3000](http://localhost:3000)
|
||||
|
||||
## Usage
|
||||
|
||||
1. **Paste a Google Maps URL**
|
||||
```
|
||||
https://www.google.com/maps/place/Business+Name/...
|
||||
```
|
||||
|
||||
2. **Click "Scrape"**
|
||||
- Job is submitted to the API
|
||||
- Status updates in real-time
|
||||
- Reviews appear when complete
|
||||
|
||||
3. **View Results**
|
||||
- See all scraped reviews
|
||||
- Export as JSON
|
||||
- View performance metrics
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Create `.env.local` if you need to customize:
|
||||
|
||||
```bash
|
||||
# API URL (default: http://localhost:8000)
|
||||
NEXT_PUBLIC_API_URL=http://localhost:8000
|
||||
```
|
||||
|
||||
## API Endpoints Used
|
||||
|
||||
This interface connects to:
|
||||
|
||||
- `POST /scrape` - Submit scraping job
|
||||
- `GET /jobs/{job_id}` - Get job status
|
||||
- `GET /jobs/{job_id}/reviews` - Get reviews
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Next.js 15** - React framework
|
||||
- **TypeScript** - Type safety
|
||||
- **Tailwind CSS** - Styling
|
||||
- **API Proxy** - Next.js API routes proxy to scraper API
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
npm run dev # Start dev server
|
||||
npm run build # Build for production
|
||||
npm run start # Start production server
|
||||
npm run lint # Run ESLint
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- The interface polls job status every 2 seconds
|
||||
- Polling stops when job completes or fails
|
||||
- Reviews are fetched with a limit of 1000 by default
|
||||
- Export button downloads reviews as formatted JSON
|
||||
37
web/app/api/check-reviews/route.ts
Normal file
37
web/app/api/check-reviews/route.ts
Normal file
@@ -0,0 +1,37 @@
|
||||
import { NextRequest, NextResponse } from 'next/server';
|
||||
|
||||
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000';
|
||||
|
||||
export async function POST(request: NextRequest) {
|
||||
try {
|
||||
const { url } = await request.json();
|
||||
|
||||
if (!url) {
|
||||
return NextResponse.json({ error: 'URL is required' }, { status: 400 });
|
||||
}
|
||||
|
||||
// Call the containerized scraper API to check if reviews exist
|
||||
const response = await fetch(`${API_BASE_URL}/check-reviews`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ url }),
|
||||
});
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json(
|
||||
{ error: data.detail || 'Failed to check reviews' },
|
||||
{ status: response.status }
|
||||
);
|
||||
}
|
||||
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
console.error('Check reviews API error:', error);
|
||||
return NextResponse.json(
|
||||
{ error: 'Failed to connect to scraper API' },
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
}
|
||||
33
web/app/api/jobs/[jobId]/reviews/route.ts
Normal file
33
web/app/api/jobs/[jobId]/reviews/route.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { NextRequest, NextResponse } from 'next/server';
|
||||
|
||||
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000';
|
||||
|
||||
export async function GET(
|
||||
request: NextRequest,
|
||||
{ params }: { params: Promise<{ jobId: string }> }
|
||||
) {
|
||||
try {
|
||||
const { jobId } = await params;
|
||||
const { searchParams } = new URL(request.url);
|
||||
const limit = searchParams.get('limit') || '1000';
|
||||
|
||||
const response = await fetch(`${API_BASE_URL}/jobs/${jobId}/reviews?limit=${limit}`);
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json(
|
||||
{ error: 'Failed to get reviews' },
|
||||
{ status: response.status }
|
||||
);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
// API returns { job_id, reviews: [...], count }, we just need the reviews array
|
||||
return NextResponse.json({ reviews: data.reviews || [] });
|
||||
} catch (error) {
|
||||
console.error('Reviews API error:', error);
|
||||
return NextResponse.json(
|
||||
{ error: 'Failed to get reviews' },
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
}
|
||||
30
web/app/api/jobs/[jobId]/route.ts
Normal file
30
web/app/api/jobs/[jobId]/route.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
import { NextRequest, NextResponse } from 'next/server';
|
||||
|
||||
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000';
|
||||
|
||||
export async function GET(
|
||||
request: NextRequest,
|
||||
{ params }: { params: Promise<{ jobId: string }> }
|
||||
) {
|
||||
try {
|
||||
const { jobId } = await params;
|
||||
|
||||
const response = await fetch(`${API_BASE_URL}/jobs/${jobId}`);
|
||||
const data = await response.json();
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json(
|
||||
{ error: data.detail || 'Job not found' },
|
||||
{ status: response.status }
|
||||
);
|
||||
}
|
||||
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
console.error('Job status API error:', error);
|
||||
return NextResponse.json(
|
||||
{ error: 'Failed to get job status' },
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
}
|
||||
37
web/app/api/scrape/route.ts
Normal file
37
web/app/api/scrape/route.ts
Normal file
@@ -0,0 +1,37 @@
|
||||
import { NextRequest, NextResponse } from 'next/server';
|
||||
|
||||
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000';
|
||||
|
||||
export async function POST(request: NextRequest) {
|
||||
try {
|
||||
const { url } = await request.json();
|
||||
|
||||
if (!url) {
|
||||
return NextResponse.json({ error: 'URL is required' }, { status: 400 });
|
||||
}
|
||||
|
||||
// Call the containerized scraper API
|
||||
const response = await fetch(`${API_BASE_URL}/scrape`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ url }),
|
||||
});
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json(
|
||||
{ error: data.detail || 'Failed to start scraping' },
|
||||
{ status: response.status }
|
||||
);
|
||||
}
|
||||
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
console.error('Scrape API error:', error);
|
||||
return NextResponse.json(
|
||||
{ error: 'Failed to connect to scraper API' },
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
}
|
||||
BIN
web/app/favicon.ico
Normal file
BIN
web/app/favicon.ico
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
26
web/app/globals.css
Normal file
26
web/app/globals.css
Normal file
@@ -0,0 +1,26 @@
|
||||
@import "tailwindcss";
|
||||
|
||||
:root {
|
||||
--background: #ffffff;
|
||||
--foreground: #171717;
|
||||
}
|
||||
|
||||
@theme inline {
|
||||
--color-background: var(--background);
|
||||
--color-foreground: var(--foreground);
|
||||
--font-sans: var(--font-geist-sans);
|
||||
--font-mono: var(--font-geist-mono);
|
||||
}
|
||||
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--background: #0a0a0a;
|
||||
--foreground: #ededed;
|
||||
}
|
||||
}
|
||||
|
||||
body {
|
||||
background: var(--background);
|
||||
color: var(--foreground);
|
||||
font-family: Arial, Helvetica, sans-serif;
|
||||
}
|
||||
34
web/app/layout.tsx
Normal file
34
web/app/layout.tsx
Normal file
@@ -0,0 +1,34 @@
|
||||
import type { Metadata } from "next";
|
||||
import { Geist, Geist_Mono } from "next/font/google";
|
||||
import "./globals.css";
|
||||
|
||||
const geistSans = Geist({
|
||||
variable: "--font-geist-sans",
|
||||
subsets: ["latin"],
|
||||
});
|
||||
|
||||
const geistMono = Geist_Mono({
|
||||
variable: "--font-geist-mono",
|
||||
subsets: ["latin"],
|
||||
});
|
||||
|
||||
export const metadata: Metadata = {
|
||||
title: "Create Next App",
|
||||
description: "Generated by create next app",
|
||||
};
|
||||
|
||||
export default function RootLayout({
|
||||
children,
|
||||
}: Readonly<{
|
||||
children: React.ReactNode;
|
||||
}>) {
|
||||
return (
|
||||
<html lang="en">
|
||||
<body
|
||||
className={`${geistSans.variable} ${geistMono.variable} antialiased`}
|
||||
>
|
||||
{children}
|
||||
</body>
|
||||
</html>
|
||||
);
|
||||
}
|
||||
38
web/app/page.tsx
Normal file
38
web/app/page.tsx
Normal file
@@ -0,0 +1,38 @@
|
||||
import ScraperTest from '@/components/ScraperTest';
|
||||
|
||||
export default function Home() {
|
||||
return (
|
||||
<div className="min-h-screen bg-gradient-to-br from-blue-600 to-indigo-700 py-12 px-4">
|
||||
<main className="max-w-5xl mx-auto">
|
||||
<div className="text-center mb-10">
|
||||
<h1 className="text-4xl md:text-5xl font-bold text-white mb-3">
|
||||
Google Reviews Scraper
|
||||
</h1>
|
||||
<p className="text-blue-100 text-lg">
|
||||
Test the containerized scraper API
|
||||
</p>
|
||||
<div className="mt-4 inline-flex items-center gap-2 px-4 py-2 bg-blue-500/30 rounded-lg text-blue-100 text-sm">
|
||||
<div className="w-2 h-2 bg-green-400 rounded-full animate-pulse"></div>
|
||||
Powered by SeleniumBase UC Mode
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="bg-white rounded-2xl shadow-2xl p-6 md:p-8">
|
||||
<ScraperTest />
|
||||
</div>
|
||||
|
||||
<div className="mt-8 text-center text-blue-100 text-sm space-y-2">
|
||||
<p className="font-medium">💡 Example URLs to test:</p>
|
||||
<div className="space-y-1 text-xs">
|
||||
<p className="font-mono bg-blue-500/20 rounded px-3 py-1 inline-block">
|
||||
https://www.google.com/maps/place/Soho+Club/...
|
||||
</p>
|
||||
</div>
|
||||
<p className="mt-4 text-blue-200">
|
||||
API running at: <span className="font-mono">localhost:8000</span>
|
||||
</p>
|
||||
</div>
|
||||
</main>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
703
web/components/ReviewAnalytics.tsx
Normal file
703
web/components/ReviewAnalytics.tsx
Normal file
@@ -0,0 +1,703 @@
|
||||
'use client';
|
||||
|
||||
import { useState, useMemo } from 'react';
|
||||
import {
|
||||
useReactTable,
|
||||
getCoreRowModel,
|
||||
getFilteredRowModel,
|
||||
getSortedRowModel,
|
||||
getPaginationRowModel,
|
||||
ColumnDef,
|
||||
flexRender,
|
||||
SortingState,
|
||||
ColumnFiltersState,
|
||||
} from '@tanstack/react-table';
|
||||
import { BarChart, Bar, XAxis, YAxis, CartesianGrid, Tooltip, ResponsiveContainer, PieChart, Pie, Cell, LineChart, Line } from 'recharts';
|
||||
import { Star, TrendingUp, Image, FileText, MessageSquare, Calendar, ArrowUpDown, ArrowUp, ArrowDown, Search, Download, Filter, AlertTriangle, ThumbsUp, ThumbsDown } from 'lucide-react';
|
||||
import { Review, calculateReviewStats, getSentimentLabel, getSentimentColor, DateRange, filterReviewsByDateRange, calculateTimelineData } from '@/lib/analytics';
|
||||
|
||||
interface ReviewAnalyticsProps {
|
||||
reviews: Review[];
|
||||
businessName?: string;
|
||||
}
|
||||
|
||||
export default function ReviewAnalytics({ reviews, businessName }: ReviewAnalyticsProps) {
|
||||
const [sorting, setSorting] = useState<SortingState>([{ id: 'date', desc: true }]); // Default: newest first
|
||||
const [columnFilters, setColumnFiltersState] = useState<ColumnFiltersState>([]);
|
||||
const [globalFilter, setGlobalFilter] = useState('');
|
||||
const [selectedRatings, setSelectedRatings] = useState<number[]>([1, 2, 3, 4, 5]);
|
||||
const [selectedSentiments, setSelectedSentiments] = useState<('positive' | 'neutral' | 'negative')[]>(['positive', 'neutral', 'negative']);
|
||||
const [dateRange, setDateRange] = useState<DateRange>('all');
|
||||
|
||||
// Filter reviews by date range
|
||||
const dateFilteredReviews = useMemo(() => {
|
||||
return filterReviewsByDateRange(reviews, dateRange);
|
||||
}, [reviews, dateRange]);
|
||||
|
||||
// Calculate statistics on date-filtered reviews
|
||||
const stats = useMemo(() => calculateReviewStats(dateFilteredReviews), [dateFilteredReviews]);
|
||||
|
||||
// Calculate timeline data for chart
|
||||
const timelineData = useMemo(() => calculateTimelineData(dateFilteredReviews), [dateFilteredReviews]);
|
||||
|
||||
// Filter reviews by selected ratings and sentiments (for table)
|
||||
const filteredReviews = useMemo(() => {
|
||||
return dateFilteredReviews.filter(r => {
|
||||
const matchesRating = selectedRatings.includes(r.rating);
|
||||
const sentiment = getSentimentLabel(r.rating);
|
||||
const matchesSentiment = selectedSentiments.includes(sentiment);
|
||||
const matchesSearch = !globalFilter ||
|
||||
r.author.toLowerCase().includes(globalFilter.toLowerCase()) ||
|
||||
r.text?.toLowerCase().includes(globalFilter.toLowerCase()) ||
|
||||
r.date_text.toLowerCase().includes(globalFilter.toLowerCase());
|
||||
|
||||
return matchesRating && matchesSentiment && matchesSearch;
|
||||
});
|
||||
}, [dateFilteredReviews, selectedRatings, selectedSentiments, globalFilter]);
|
||||
|
||||
const toggleRating = (rating: number) => {
|
||||
setSelectedRatings(prev =>
|
||||
prev.includes(rating) ? prev.filter(r => r !== rating) : [...prev, rating]
|
||||
);
|
||||
};
|
||||
|
||||
const toggleSentiment = (sentiment: 'positive' | 'neutral' | 'negative') => {
|
||||
setSelectedSentiments(prev =>
|
||||
prev.includes(sentiment) ? prev.filter(s => s !== sentiment) : [...prev, sentiment]
|
||||
);
|
||||
};
|
||||
|
||||
const clearAllFilters = () => {
|
||||
setDateRange('all');
|
||||
setSelectedRatings([1, 2, 3, 4, 5]);
|
||||
setSelectedSentiments(['positive', 'neutral', 'negative']);
|
||||
setGlobalFilter('');
|
||||
};
|
||||
|
||||
const hasActiveFilters = dateRange !== 'all' ||
|
||||
selectedRatings.length < 5 ||
|
||||
selectedSentiments.length < 3 ||
|
||||
globalFilter !== '';
|
||||
|
||||
const exportFilteredData = () => {
|
||||
const dataStr = JSON.stringify(filteredReviews, null, 2);
|
||||
const dataBlob = new Blob([dataStr], { type: 'application/json' });
|
||||
const url = URL.createObjectURL(dataBlob);
|
||||
const link = document.createElement('a');
|
||||
link.href = url;
|
||||
link.download = `reviews-filtered-${dateRange}-${new Date().toISOString().split('T')[0]}.json`;
|
||||
link.click();
|
||||
};
|
||||
|
||||
// Chart colors
|
||||
const COLORS = {
|
||||
positive: '#16a34a',
|
||||
neutral: '#ca8a04',
|
||||
negative: '#dc2626',
|
||||
};
|
||||
|
||||
// Table columns
|
||||
const columns = useMemo<ColumnDef<Review>[]>(
|
||||
() => [
|
||||
{
|
||||
accessorKey: 'author',
|
||||
header: ({ column }) => {
|
||||
return (
|
||||
<button
|
||||
onClick={() => column.toggleSorting(column.getIsSorted() === 'asc')}
|
||||
className="flex items-center gap-2 hover:text-blue-700 font-semibold"
|
||||
>
|
||||
Author
|
||||
{column.getIsSorted() === 'asc' ? <ArrowUp className="w-4 h-4" /> : column.getIsSorted() === 'desc' ? <ArrowDown className="w-4 h-4" /> : <ArrowUpDown className="w-4 h-4 opacity-50" />}
|
||||
</button>
|
||||
);
|
||||
},
|
||||
cell: ({ row }) => (
|
||||
<div className="flex items-center gap-2">
|
||||
{row.original.avatar_url && (
|
||||
<img src={row.original.avatar_url} alt={row.original.author} className="w-8 h-8 rounded-full" />
|
||||
)}
|
||||
<span className="font-medium text-gray-900">{row.original.author}</span>
|
||||
</div>
|
||||
),
|
||||
},
|
||||
{
|
||||
accessorKey: 'rating',
|
||||
header: ({ column }) => {
|
||||
return (
|
||||
<button
|
||||
onClick={() => column.toggleSorting(column.getIsSorted() === 'asc')}
|
||||
className="flex items-center gap-2 hover:text-blue-700 font-semibold"
|
||||
>
|
||||
Rating
|
||||
{column.getIsSorted() === 'asc' ? <ArrowUp className="w-4 h-4" /> : column.getIsSorted() === 'desc' ? <ArrowDown className="w-4 h-4" /> : <ArrowUpDown className="w-4 h-4 opacity-50" />}
|
||||
</button>
|
||||
);
|
||||
},
|
||||
cell: ({ row }) => (
|
||||
<div className="flex items-center gap-1">
|
||||
{[...Array(5)].map((_, i) => (
|
||||
<Star
|
||||
key={i}
|
||||
className={`w-4 h-4 ${i < row.original.rating ? 'text-yellow-500 fill-yellow-500' : 'text-gray-300'}`}
|
||||
/>
|
||||
))}
|
||||
<span className="ml-2 font-bold text-gray-900">{row.original.rating}</span>
|
||||
</div>
|
||||
),
|
||||
filterFn: (row, id, value) => {
|
||||
return value.includes(row.getValue(id));
|
||||
},
|
||||
},
|
||||
{
|
||||
accessorKey: 'centerDate',
|
||||
id: 'date',
|
||||
header: ({ column }) => {
|
||||
return (
|
||||
<button
|
||||
onClick={() => column.toggleSorting(column.getIsSorted() === 'asc')}
|
||||
className="flex items-center gap-2 hover:text-blue-700 font-semibold"
|
||||
>
|
||||
Date
|
||||
{column.getIsSorted() === 'asc' ? <ArrowUp className="w-4 h-4" /> : column.getIsSorted() === 'desc' ? <ArrowDown className="w-4 h-4" /> : <ArrowUpDown className="w-4 h-4 opacity-50" />}
|
||||
</button>
|
||||
);
|
||||
},
|
||||
sortingFn: (rowA, rowB) => {
|
||||
const dateA = rowA.original.centerDate?.getTime() || 0;
|
||||
const dateB = rowB.original.centerDate?.getTime() || 0;
|
||||
return dateA - dateB;
|
||||
},
|
||||
cell: ({ row }) => {
|
||||
const formatDate = (date: Date) => {
|
||||
return date.toLocaleDateString('en-US', { year: 'numeric', month: 'short', day: 'numeric' });
|
||||
};
|
||||
|
||||
const getUncertaintyDays = (minDate: Date, maxDate: Date) => {
|
||||
const diffMs = Math.abs(maxDate.getTime() - minDate.getTime());
|
||||
return Math.round(diffMs / (1000 * 60 * 60 * 24));
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="space-y-1">
|
||||
<div className="text-gray-900 font-medium">{row.original.date_text}</div>
|
||||
{row.original.minDate && row.original.maxDate && row.original.centerDate && (
|
||||
<div className="text-xs text-gray-500 space-y-0.5">
|
||||
<div>Range: {formatDate(row.original.maxDate)} - {formatDate(row.original.minDate)}</div>
|
||||
<div className="text-purple-700 font-semibold">
|
||||
Center: {formatDate(row.original.centerDate)}
|
||||
</div>
|
||||
<div className="text-blue-600">
|
||||
±{getUncertaintyDays(row.original.minDate, row.original.maxDate)} days uncertainty
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
},
|
||||
},
|
||||
{
|
||||
accessorKey: 'text',
|
||||
header: 'Review',
|
||||
cell: ({ row }) => {
|
||||
const [expanded, setExpanded] = useState(false);
|
||||
const text = row.original.text || 'No review text';
|
||||
const sentiment = getSentimentLabel(row.original.rating);
|
||||
|
||||
return (
|
||||
<div className="max-w-2xl">
|
||||
<div className={`inline-block px-2 py-1 rounded-md text-xs font-semibold mb-2 border ${getSentimentColor(sentiment)}`}>
|
||||
{sentiment.toUpperCase()}
|
||||
</div>
|
||||
<p className={`text-gray-800 ${!expanded && 'line-clamp-2'}`}>
|
||||
{text}
|
||||
</p>
|
||||
{text.length > 100 && (
|
||||
<button
|
||||
onClick={() => setExpanded(!expanded)}
|
||||
className="text-blue-700 hover:text-blue-800 text-sm font-semibold mt-1"
|
||||
>
|
||||
{expanded ? 'Show less' : 'Show more'}
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
},
|
||||
},
|
||||
],
|
||||
[]
|
||||
);
|
||||
|
||||
const table = useReactTable({
|
||||
data: filteredReviews,
|
||||
columns,
|
||||
state: {
|
||||
sorting,
|
||||
},
|
||||
onSortingChange: setSorting,
|
||||
getCoreRowModel: getCoreRowModel(),
|
||||
getSortedRowModel: getSortedRowModel(),
|
||||
getPaginationRowModel: getPaginationRowModel(),
|
||||
initialState: {
|
||||
pagination: {
|
||||
pageSize: 10,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h2 className="text-3xl font-bold text-gray-900">
|
||||
{businessName ? `${businessName} - Analytics` : 'Review Analytics'}
|
||||
</h2>
|
||||
<p className="text-gray-600 mt-1">Comprehensive insights from {reviews.length} total reviews</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Enhanced Filters */}
|
||||
<div className="bg-white border-2 border-gray-300 rounded-xl p-5 shadow-sm space-y-4">
|
||||
{/* Time Period Filter */}
|
||||
<div className="flex items-center gap-3 flex-wrap">
|
||||
<Filter className="w-5 h-5 text-gray-700" />
|
||||
<span className="font-semibold text-gray-900">Time Period:</span>
|
||||
{(['week', 'month', 'year', 'all'] as DateRange[]).map((range) => (
|
||||
<button
|
||||
key={range}
|
||||
onClick={() => setDateRange(range)}
|
||||
className={`px-4 py-2 rounded-lg font-semibold transition-all border-2 ${
|
||||
dateRange === range
|
||||
? 'bg-blue-600 text-white border-blue-700 shadow-md'
|
||||
: 'bg-white text-gray-700 border-gray-300 hover:border-blue-400 hover:bg-blue-50'
|
||||
}`}
|
||||
>
|
||||
{range === 'week' ? 'Last Week' : range === 'month' ? 'Last Month' : range === 'year' ? 'Last Year' : 'All Time'}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
{/* Sentiment Filter */}
|
||||
<div className="flex items-center gap-3 flex-wrap">
|
||||
<TrendingUp className="w-5 h-5 text-gray-700" />
|
||||
<span className="font-semibold text-gray-900">Sentiment:</span>
|
||||
{(['positive', 'neutral', 'negative'] as const).map((sentiment) => (
|
||||
<button
|
||||
key={sentiment}
|
||||
onClick={() => toggleSentiment(sentiment)}
|
||||
className={`px-4 py-2 rounded-lg font-semibold transition-all border-2 ${
|
||||
selectedSentiments.includes(sentiment)
|
||||
? sentiment === 'positive' ? 'bg-green-600 text-white border-green-700 shadow-md'
|
||||
: sentiment === 'neutral' ? 'bg-yellow-600 text-white border-yellow-700 shadow-md'
|
||||
: 'bg-red-600 text-white border-red-700 shadow-md'
|
||||
: 'bg-white text-gray-700 border-gray-300 hover:border-blue-400 hover:bg-blue-50'
|
||||
}`}
|
||||
>
|
||||
{sentiment === 'positive' ? '😊 Positive (4-5★)' : sentiment === 'neutral' ? '😐 Neutral (3★)' : '😞 Negative (1-2★)'}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
{/* Filter Summary */}
|
||||
<div className="flex items-center justify-between pt-2 border-t-2 border-gray-200">
|
||||
<span className="text-sm font-medium text-gray-600">
|
||||
Showing {filteredReviews.length} of {reviews.length} reviews
|
||||
{hasActiveFilters && <span className="text-blue-700 ml-1">(filtered)</span>}
|
||||
</span>
|
||||
{hasActiveFilters && (
|
||||
<button
|
||||
onClick={clearAllFilters}
|
||||
className="px-3 py-1.5 bg-gray-100 text-gray-700 rounded-lg hover:bg-gray-200 font-semibold border-2 border-gray-300 text-sm"
|
||||
>
|
||||
Clear All Filters
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* KPI Cards */}
|
||||
<div className="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-4 gap-4">
|
||||
{/* Average Rating */}
|
||||
<div className="bg-gradient-to-br from-yellow-100 to-yellow-200 border-2 border-yellow-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow">
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<Star className="w-5 h-5 text-yellow-700" />
|
||||
<span className="text-sm font-bold text-yellow-900">Avg Rating</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-yellow-900">{stats.averageRating.toFixed(1)}★</div>
|
||||
<div className="text-xs text-yellow-800 mt-1 font-medium">
|
||||
{stats.totalReviews} total reviews
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Positive Reviews */}
|
||||
<div className="bg-gradient-to-br from-green-100 to-green-200 border-2 border-green-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow cursor-pointer" onClick={() => { setSelectedSentiments(['positive']); setDateRange('all'); }}>
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<ThumbsUp className="w-5 h-5 text-green-700" />
|
||||
<span className="text-sm font-bold text-green-900">Positive</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-green-900">{stats.sentimentBreakdown.positive}</div>
|
||||
<div className="text-xs text-green-800 mt-1 font-medium">
|
||||
{stats.sentimentScore.toFixed(0)}% positive (4-5★)
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Neutral Reviews */}
|
||||
<div className="bg-gradient-to-br from-yellow-50 to-yellow-100 border-2 border-yellow-300 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow cursor-pointer" onClick={() => { setSelectedSentiments(['neutral']); setDateRange('all'); }}>
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<MessageSquare className="w-5 h-5 text-yellow-700" />
|
||||
<span className="text-sm font-bold text-yellow-800">Neutral</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-yellow-800">{stats.sentimentBreakdown.neutral}</div>
|
||||
<div className="text-xs text-yellow-700 mt-1 font-medium">
|
||||
{((stats.sentimentBreakdown.neutral / stats.totalReviews) * 100).toFixed(0)}% neutral (3★)
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Negative Reviews - Alert */}
|
||||
<div className="bg-gradient-to-br from-red-100 to-red-200 border-2 border-red-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow cursor-pointer" onClick={() => { setSelectedSentiments(['negative']); setDateRange('all'); }}>
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<AlertTriangle className="w-5 h-5 text-red-700" />
|
||||
<span className="text-sm font-bold text-red-900">Negative</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-red-900">{stats.negativeReviews}</div>
|
||||
<div className="text-xs text-red-800 mt-1 font-medium">
|
||||
{((stats.negativeReviews / stats.totalReviews) * 100).toFixed(0)}% negative (1-2★)
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Recent Activity */}
|
||||
<div className="bg-gradient-to-br from-blue-100 to-blue-200 border-2 border-blue-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow cursor-pointer" onClick={() => setDateRange('month')}>
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<Calendar className="w-5 h-5 text-blue-700" />
|
||||
<span className="text-sm font-bold text-blue-900">Recent</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-blue-900">{stats.recentReviews}</div>
|
||||
<div className="text-xs text-blue-800 mt-1 font-medium">last 30 days</div>
|
||||
</div>
|
||||
|
||||
{/* Review Length */}
|
||||
<div className="bg-gradient-to-br from-purple-100 to-purple-200 border-2 border-purple-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow">
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<FileText className="w-5 h-5 text-purple-700" />
|
||||
<span className="text-sm font-bold text-purple-900">Avg Length</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-purple-900">{stats.avgReviewLength}</div>
|
||||
<div className="text-xs text-purple-800 mt-1 font-medium">words per review</div>
|
||||
</div>
|
||||
|
||||
{/* Photos */}
|
||||
<div className="bg-gradient-to-br from-pink-100 to-pink-200 border-2 border-pink-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow">
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<Image className="w-5 h-5 text-pink-700" />
|
||||
<span className="text-sm font-bold text-pink-900">With Photos</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-pink-900">{stats.photoCount}</div>
|
||||
<div className="text-xs text-pink-800 mt-1 font-medium">
|
||||
{((stats.photoCount / stats.totalReviews) * 100).toFixed(0)}% have avatars
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Total Reviews */}
|
||||
<div className="bg-gradient-to-br from-indigo-100 to-indigo-200 border-2 border-indigo-400 rounded-xl p-4 shadow-md hover:shadow-lg transition-shadow">
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<MessageSquare className="w-5 h-5 text-indigo-700" />
|
||||
<span className="text-sm font-bold text-indigo-900">Total</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-indigo-900">{stats.totalReviews}</div>
|
||||
<div className="text-xs text-indigo-800 mt-1 font-medium">all time</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Rating Timeline with Rolling Average */}
|
||||
{timelineData.length > 0 && (
|
||||
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
|
||||
<h3 className="text-xl font-bold mb-4 text-gray-900">Rating Trend Over Time</h3>
|
||||
<ResponsiveContainer width="100%" height={300}>
|
||||
<LineChart data={timelineData}>
|
||||
<CartesianGrid strokeDasharray="3 3" stroke="#e5e7eb" />
|
||||
<XAxis
|
||||
dataKey="date"
|
||||
tick={{ fill: '#374151', fontWeight: 600 }}
|
||||
tickLine={{ stroke: '#9ca3af' }}
|
||||
/>
|
||||
<YAxis
|
||||
domain={[0, 5]}
|
||||
ticks={[0, 1, 2, 3, 4, 5]}
|
||||
tick={{ fill: '#374151', fontWeight: 600 }}
|
||||
tickLine={{ stroke: '#9ca3af' }}
|
||||
/>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
backgroundColor: '#ffffff',
|
||||
border: '2px solid #3b82f6',
|
||||
borderRadius: '8px',
|
||||
fontWeight: 600
|
||||
}}
|
||||
/>
|
||||
<Line
|
||||
type="monotone"
|
||||
dataKey="rating"
|
||||
stroke="#94a3b8"
|
||||
strokeWidth={2}
|
||||
name="Monthly Avg"
|
||||
dot={{ fill: '#64748b', r: 4 }}
|
||||
/>
|
||||
<Line
|
||||
type="monotone"
|
||||
dataKey="rollingAvg"
|
||||
stroke="#3b82f6"
|
||||
strokeWidth={3}
|
||||
name="3-Month Rolling Avg"
|
||||
dot={{ fill: '#2563eb', r: 5 }}
|
||||
/>
|
||||
</LineChart>
|
||||
</ResponsiveContainer>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Charts Grid */}
|
||||
<div className="grid md:grid-cols-3 gap-6">
|
||||
{/* Rating Distribution - Interactive */}
|
||||
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
|
||||
<h3 className="text-lg font-bold mb-4 text-gray-900">
|
||||
Rating Distribution
|
||||
<span className="text-xs font-normal text-gray-500 ml-2">(click to filter)</span>
|
||||
</h3>
|
||||
<ResponsiveContainer width="100%" height={250}>
|
||||
<BarChart
|
||||
data={stats.ratingDistribution}
|
||||
onClick={(data) => {
|
||||
if (data && data.activePayload && data.activePayload[0]) {
|
||||
const rating = data.activePayload[0].payload.rating;
|
||||
setSelectedRatings([rating]);
|
||||
setSelectedSentiments(['positive', 'neutral', 'negative']);
|
||||
}
|
||||
}}
|
||||
style={{ cursor: 'pointer' }}
|
||||
>
|
||||
<CartesianGrid strokeDasharray="3 3" stroke="#e5e7eb" />
|
||||
<XAxis
|
||||
dataKey="rating"
|
||||
tick={{ fill: '#374151', fontWeight: 600 }}
|
||||
tickLine={{ stroke: '#9ca3af' }}
|
||||
/>
|
||||
<YAxis
|
||||
tick={{ fill: '#374151', fontWeight: 600 }}
|
||||
tickLine={{ stroke: '#9ca3af' }}
|
||||
/>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
backgroundColor: '#ffffff',
|
||||
border: '2px solid #3b82f6',
|
||||
borderRadius: '8px',
|
||||
fontWeight: 600
|
||||
}}
|
||||
content={({ active, payload }) => {
|
||||
if (active && payload && payload.length) {
|
||||
return (
|
||||
<div className="bg-white border-2 border-blue-600 rounded-lg p-2 shadow-lg">
|
||||
<p className="font-bold text-gray-900">{payload[0].payload.rating}★</p>
|
||||
<p className="text-sm text-gray-600">{payload[0].value} reviews ({payload[0].payload.percentage.toFixed(1)}%)</p>
|
||||
<p className="text-xs text-blue-600 mt-1">Click to filter</p>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
return null;
|
||||
}}
|
||||
/>
|
||||
<Bar dataKey="count" fill="#3b82f6" radius={[8, 8, 0, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</div>
|
||||
|
||||
{/* Sentiment Breakdown - Interactive */}
|
||||
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
|
||||
<h3 className="text-lg font-bold mb-4 text-gray-900">
|
||||
Sentiment Breakdown
|
||||
<span className="text-xs font-normal text-gray-500 ml-2">(click to filter)</span>
|
||||
</h3>
|
||||
<ResponsiveContainer width="100%" height={250}>
|
||||
<PieChart>
|
||||
<Pie
|
||||
data={[
|
||||
{ name: 'Positive', value: stats.sentimentBreakdown.positive, sentiment: 'positive' },
|
||||
{ name: 'Neutral', value: stats.sentimentBreakdown.neutral, sentiment: 'neutral' },
|
||||
{ name: 'Negative', value: stats.sentimentBreakdown.negative, sentiment: 'negative' },
|
||||
]}
|
||||
cx="50%"
|
||||
cy="50%"
|
||||
labelLine={false}
|
||||
label={({ name, percent }) => `${name} ${(percent * 100).toFixed(0)}%`}
|
||||
outerRadius={80}
|
||||
fill="#8884d8"
|
||||
dataKey="value"
|
||||
style={{ fontWeight: 700, fontSize: '13px', cursor: 'pointer' }}
|
||||
onClick={(data) => {
|
||||
if (data && data.sentiment) {
|
||||
setSelectedSentiments([data.sentiment as 'positive' | 'neutral' | 'negative']);
|
||||
setSelectedRatings([1, 2, 3, 4, 5]);
|
||||
}
|
||||
}}
|
||||
>
|
||||
<Cell fill={COLORS.positive} />
|
||||
<Cell fill={COLORS.neutral} />
|
||||
<Cell fill={COLORS.negative} />
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
backgroundColor: '#ffffff',
|
||||
border: '2px solid #3b82f6',
|
||||
borderRadius: '8px',
|
||||
fontWeight: 600
|
||||
}}
|
||||
content={({ active, payload }) => {
|
||||
if (active && payload && payload.length) {
|
||||
return (
|
||||
<div className="bg-white border-2 border-blue-600 rounded-lg p-2 shadow-lg">
|
||||
<p className="font-bold text-gray-900">{payload[0].name}</p>
|
||||
<p className="text-sm text-gray-600">{payload[0].value} reviews</p>
|
||||
<p className="text-xs text-blue-600 mt-1">Click to filter</p>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
return null;
|
||||
}}
|
||||
/>
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</div>
|
||||
|
||||
{/* Top Keywords */}
|
||||
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
|
||||
<h3 className="text-lg font-bold mb-4 text-gray-900">Top Keywords</h3>
|
||||
<ResponsiveContainer width="100%" height={250}>
|
||||
<BarChart data={stats.topKeywords} layout="vertical">
|
||||
<CartesianGrid strokeDasharray="3 3" stroke="#e5e7eb" />
|
||||
<XAxis
|
||||
type="number"
|
||||
tick={{ fill: '#374151', fontWeight: 600 }}
|
||||
tickLine={{ stroke: '#9ca3af' }}
|
||||
/>
|
||||
<YAxis
|
||||
type="category"
|
||||
dataKey="word"
|
||||
width={80}
|
||||
tick={{ fill: '#374151', fontWeight: 600 }}
|
||||
tickLine={{ stroke: '#9ca3af' }}
|
||||
/>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
backgroundColor: '#ffffff',
|
||||
border: '2px solid #3b82f6',
|
||||
borderRadius: '8px',
|
||||
fontWeight: 600
|
||||
}}
|
||||
/>
|
||||
<Bar dataKey="count" fill="#8b5cf6" radius={[0, 8, 8, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Reviews Table */}
|
||||
<div className="bg-white border-2 border-gray-300 rounded-xl p-6 shadow-md">
|
||||
<div className="flex items-center justify-between mb-4">
|
||||
<h3 className="text-xl font-bold text-gray-900">Review Details</h3>
|
||||
<button
|
||||
onClick={exportFilteredData}
|
||||
className="flex items-center gap-2 px-4 py-2 bg-green-600 text-white rounded-lg hover:bg-green-700 font-semibold shadow-md border-2 border-green-700"
|
||||
>
|
||||
<Download className="w-4 h-4" />
|
||||
Export Filtered Data
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Search */}
|
||||
<div className="mb-6">
|
||||
<div className="relative">
|
||||
<Search className="absolute left-3 top-1/2 transform -translate-y-1/2 w-5 h-5 text-gray-500" />
|
||||
<input
|
||||
type="text"
|
||||
value={globalFilter}
|
||||
onChange={e => setGlobalFilter(e.target.value)}
|
||||
placeholder="Search by author, review text, or date..."
|
||||
className="w-full pl-10 pr-4 py-3 border-2 border-gray-300 rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500 font-medium"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Table */}
|
||||
<div className="overflow-x-auto border-2 border-gray-300 rounded-lg">
|
||||
<table className="w-full">
|
||||
<thead className="bg-gray-100 border-b-2 border-gray-300">
|
||||
{table.getHeaderGroups().map(headerGroup => (
|
||||
<tr key={headerGroup.id}>
|
||||
{headerGroup.headers.map(header => (
|
||||
<th key={header.id} className="px-6 py-4 text-left text-gray-900">
|
||||
{header.isPlaceholder
|
||||
? null
|
||||
: flexRender(header.column.columnDef.header, header.getContext())}
|
||||
</th>
|
||||
))}
|
||||
</tr>
|
||||
))}
|
||||
</thead>
|
||||
<tbody className="divide-y-2 divide-gray-200">
|
||||
{table.getRowModel().rows.map(row => (
|
||||
<tr key={row.id} className="hover:bg-gray-50">
|
||||
{row.getVisibleCells().map(cell => (
|
||||
<td key={cell.id} className="px-6 py-4">
|
||||
{flexRender(cell.column.columnDef.cell, cell.getContext())}
|
||||
</td>
|
||||
))}
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
{/* Pagination */}
|
||||
<div className="flex items-center justify-between mt-6">
|
||||
<div className="text-sm text-gray-700 font-medium">
|
||||
Showing {table.getState().pagination.pageIndex * table.getState().pagination.pageSize + 1} to{' '}
|
||||
{Math.min((table.getState().pagination.pageIndex + 1) * table.getState().pagination.pageSize, filteredReviews.length)} of{' '}
|
||||
{filteredReviews.length} reviews
|
||||
</div>
|
||||
<div className="flex gap-2">
|
||||
<button
|
||||
onClick={() => table.previousPage()}
|
||||
disabled={!table.getCanPreviousPage()}
|
||||
className="px-4 py-2 border-2 border-gray-300 rounded-lg disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50 font-semibold text-gray-900"
|
||||
>
|
||||
Previous
|
||||
</button>
|
||||
<button
|
||||
onClick={() => table.nextPage()}
|
||||
disabled={!table.getCanNextPage()}
|
||||
className="px-4 py-2 border-2 border-gray-300 rounded-lg disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50 font-semibold text-gray-900"
|
||||
>
|
||||
Next
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
909
web/components/ScraperTest.tsx
Normal file
909
web/components/ScraperTest.tsx
Normal file
@@ -0,0 +1,909 @@
|
||||
'use client';
|
||||
|
||||
import { useState, useEffect, useRef } from 'react';
|
||||
import ReviewAnalytics from './ReviewAnalytics';
|
||||
|
||||
interface Review {
|
||||
author: string;
|
||||
rating: number;
|
||||
text: string | null;
|
||||
date_text: string;
|
||||
avatar_url: string | null;
|
||||
profile_url: string | null;
|
||||
review_id: string;
|
||||
}
|
||||
|
||||
interface JobStatus {
|
||||
job_id: string;
|
||||
status: 'pending' | 'running' | 'completed' | 'failed';
|
||||
url: string;
|
||||
created_at: string;
|
||||
started_at: string | null;
|
||||
completed_at: string | null;
|
||||
updated_at: string | null; // Last update time for progress tracking
|
||||
reviews_count: number | null;
|
||||
total_reviews: number | null;
|
||||
scrape_time: number | null;
|
||||
error_message: string | null;
|
||||
}
|
||||
|
||||
export default function ScraperTest() {
|
||||
const [searchQuery, setSearchQuery] = useState('');
|
||||
const [searchedQuery, setSearchedQuery] = useState('');
|
||||
const [jobs, setJobs] = useState<Map<string, JobStatus>>(new Map());
|
||||
const [activeJobId, setActiveJobId] = useState<string | null>(null);
|
||||
const [reviews, setReviews] = useState<Review[]>([]);
|
||||
const [error, setError] = useState('');
|
||||
const [isSubmitting, setIsSubmitting] = useState(false);
|
||||
const [showAnalytics, setShowAnalytics] = useState(false);
|
||||
const [isLoadingReviews, setIsLoadingReviews] = useState(false);
|
||||
const [showConfirmModal, setShowConfirmModal] = useState(false);
|
||||
const [isCheckingReviews, setIsCheckingReviews] = useState(false);
|
||||
const [hasReviews, setHasReviews] = useState<boolean | null>(null);
|
||||
const [availableReviewCount, setAvailableReviewCount] = useState<number | null>(null);
|
||||
const [businessName, setBusinessName] = useState<string | null>(null);
|
||||
const [businessAddress, setBusinessAddress] = useState<string | null>(null);
|
||||
const [businessRating, setBusinessRating] = useState<number | null>(null);
|
||||
const debounceRef = useRef<NodeJS.Timeout | null>(null);
|
||||
const pollingIntervals = useRef<Map<string, NodeJS.Timeout>>(new Map());
|
||||
const abortControllerRef = useRef<AbortController | null>(null);
|
||||
|
||||
// Debounce: update map preview as user types (500ms after stopping)
|
||||
useEffect(() => {
|
||||
if (searchQuery.trim().length >= 2) {
|
||||
if (debounceRef.current) {
|
||||
clearTimeout(debounceRef.current);
|
||||
}
|
||||
|
||||
debounceRef.current = setTimeout(() => {
|
||||
setSearchedQuery(searchQuery.trim());
|
||||
}, 500);
|
||||
|
||||
return () => {
|
||||
if (debounceRef.current) {
|
||||
clearTimeout(debounceRef.current);
|
||||
}
|
||||
};
|
||||
}
|
||||
}, [searchQuery]);
|
||||
|
||||
// Clear validation results when user starts typing a new search
|
||||
useEffect(() => {
|
||||
// If searchQuery is different from searchedQuery, clear results
|
||||
if (searchQuery.trim() !== searchedQuery && searchedQuery) {
|
||||
// Abort any pending validation request
|
||||
if (abortControllerRef.current) {
|
||||
abortControllerRef.current.abort();
|
||||
}
|
||||
setHasReviews(null);
|
||||
setAvailableReviewCount(null);
|
||||
setBusinessName(null);
|
||||
setBusinessAddress(null);
|
||||
setBusinessRating(null);
|
||||
}
|
||||
}, [searchQuery, searchedQuery]);
|
||||
|
||||
// Check for reviews function (called manually when user clicks Validate)
|
||||
const checkReviews = async (query: string) => {
|
||||
// Abort any previous validation request
|
||||
if (abortControllerRef.current) {
|
||||
abortControllerRef.current.abort();
|
||||
}
|
||||
|
||||
setIsCheckingReviews(true);
|
||||
setHasReviews(null);
|
||||
setAvailableReviewCount(null);
|
||||
setBusinessName(null);
|
||||
setBusinessAddress(null);
|
||||
setBusinessRating(null);
|
||||
setError('');
|
||||
|
||||
// Create new abort controller with 30 second timeout
|
||||
const controller = new AbortController();
|
||||
abortControllerRef.current = controller;
|
||||
const timeoutId = setTimeout(() => controller.abort(), 30000);
|
||||
|
||||
try {
|
||||
const url = `https://www.google.com/maps/search/?api=1&query=${encodeURIComponent(query)}`;
|
||||
|
||||
const response = await fetch('/api/check-reviews', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ url }),
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
clearTimeout(timeoutId);
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (response.ok && data.success) {
|
||||
setHasReviews(data.has_reviews);
|
||||
setAvailableReviewCount(data.total_reviews || 0);
|
||||
setBusinessName(data.name);
|
||||
setBusinessAddress(data.address);
|
||||
setBusinessRating(data.rating);
|
||||
} else {
|
||||
console.error('Failed to get business info:', data.error);
|
||||
// Business not found
|
||||
setHasReviews(false);
|
||||
setAvailableReviewCount(0);
|
||||
}
|
||||
} catch (err) {
|
||||
clearTimeout(timeoutId);
|
||||
|
||||
// Ignore AbortError (happens when user starts a new validation)
|
||||
if (err instanceof Error && err.name === 'AbortError') {
|
||||
console.log('Validation cancelled (new validation started)');
|
||||
return;
|
||||
}
|
||||
|
||||
console.error('Error getting business info:', err);
|
||||
// Error occurred
|
||||
setHasReviews(false);
|
||||
setAvailableReviewCount(0);
|
||||
} finally {
|
||||
// Only clear loading state if this controller wasn't aborted
|
||||
if (!controller.signal.aborted) {
|
||||
setIsCheckingReviews(false);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// Poll job status for all active jobs
|
||||
const startPolling = (jobId: string) => {
|
||||
// Don't start if already polling this job
|
||||
if (pollingIntervals.current.has(jobId)) return;
|
||||
|
||||
const pollInterval = setInterval(async () => {
|
||||
try {
|
||||
const response = await fetch(`/api/jobs/${jobId}`);
|
||||
const data = await response.json();
|
||||
|
||||
// Update job in map
|
||||
setJobs(prev => {
|
||||
const newMap = new Map(prev);
|
||||
newMap.set(jobId, data);
|
||||
return newMap;
|
||||
});
|
||||
|
||||
// Stop polling if job is done
|
||||
if (data.status === 'completed' || data.status === 'failed') {
|
||||
const interval = pollingIntervals.current.get(jobId);
|
||||
if (interval) {
|
||||
clearInterval(interval);
|
||||
pollingIntervals.current.delete(jobId);
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
console.error('Poll error for job', jobId, err);
|
||||
}
|
||||
}, 2000); // Poll every 2 seconds
|
||||
|
||||
pollingIntervals.current.set(jobId, pollInterval);
|
||||
};
|
||||
|
||||
// Cleanup polling intervals and abort controllers on unmount
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
pollingIntervals.current.forEach(interval => clearInterval(interval));
|
||||
pollingIntervals.current.clear();
|
||||
if (abortControllerRef.current) {
|
||||
abortControllerRef.current.abort();
|
||||
}
|
||||
};
|
||||
}, []);
|
||||
|
||||
const handleSearch = () => {
|
||||
if (searchQuery.trim().length < 2) return;
|
||||
|
||||
const query = searchQuery.trim();
|
||||
|
||||
// Clear any pending debounce
|
||||
if (debounceRef.current) {
|
||||
clearTimeout(debounceRef.current);
|
||||
}
|
||||
|
||||
// Immediately update map preview and trigger validation
|
||||
setSearchedQuery(query);
|
||||
checkReviews(query);
|
||||
};
|
||||
|
||||
const handlePreviewBusiness = (e: React.FormEvent) => {
|
||||
e.preventDefault();
|
||||
setShowConfirmModal(true);
|
||||
};
|
||||
|
||||
const handleConfirmScrape = async () => {
|
||||
setError('');
|
||||
setIsSubmitting(true);
|
||||
setShowConfirmModal(false);
|
||||
|
||||
// Use the search query to create a Google Maps search URL
|
||||
const url = `https://www.google.com/maps/search/?api=1&query=${encodeURIComponent(searchedQuery)}`;
|
||||
|
||||
try {
|
||||
const response = await fetch('/api/scrape', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ url }),
|
||||
});
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(data.error || 'Failed to start scraping');
|
||||
}
|
||||
|
||||
// Add job to Map with initial status
|
||||
setJobs(prev => {
|
||||
const newMap = new Map(prev);
|
||||
newMap.set(data.job_id, {
|
||||
job_id: data.job_id,
|
||||
status: 'pending',
|
||||
url: url,
|
||||
created_at: new Date().toISOString(),
|
||||
started_at: null,
|
||||
completed_at: null,
|
||||
reviews_count: null,
|
||||
total_reviews: null,
|
||||
scrape_time: null,
|
||||
error_message: null,
|
||||
});
|
||||
return newMap;
|
||||
});
|
||||
|
||||
// Set as active job and start polling
|
||||
setActiveJobId(data.job_id);
|
||||
startPolling(data.job_id);
|
||||
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : 'Failed to submit job');
|
||||
} finally {
|
||||
setIsSubmitting(false);
|
||||
}
|
||||
};
|
||||
|
||||
const getStatusColor = (status: string) => {
|
||||
switch (status) {
|
||||
case 'completed': return 'text-green-700';
|
||||
case 'running': return 'text-blue-700';
|
||||
case 'failed': return 'text-red-700';
|
||||
default: return 'text-gray-800';
|
||||
}
|
||||
};
|
||||
|
||||
const getStatusIcon = (status: string) => {
|
||||
switch (status) {
|
||||
case 'completed':
|
||||
return (
|
||||
<svg className="w-5 h-5 text-green-500" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
|
||||
</svg>
|
||||
);
|
||||
case 'running':
|
||||
return <div className="w-5 h-5 border-2 border-blue-500 border-t-transparent rounded-full animate-spin" />;
|
||||
case 'failed':
|
||||
return (
|
||||
<svg className="w-5 h-5 text-red-500" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zM8.707 7.293a1 1 0 00-1.414 1.414L8.586 10l-1.293 1.293a1 1 0 101.414 1.414L10 11.414l1.293 1.293a1 1 0 001.414-1.414L11.414 10l1.293-1.293a1 1 0 00-1.414-1.414L10 8.586 8.707 7.293z" clipRule="evenodd" />
|
||||
</svg>
|
||||
);
|
||||
default:
|
||||
return (
|
||||
<svg className="w-5 h-5 text-gray-400" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm1-12a1 1 0 10-2 0v4a1 1 0 00.293.707l2.828 2.829a1 1 0 101.415-1.415L11 9.586V6z" clipRule="evenodd" />
|
||||
</svg>
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
const embedUrl = searchedQuery
|
||||
? `https://maps.google.com/maps?q=${encodeURIComponent(searchedQuery)}&output=embed&z=15`
|
||||
: '';
|
||||
|
||||
const [mapClicked, setMapClicked] = useState(false);
|
||||
const searchInputRef = useRef<HTMLInputElement>(null);
|
||||
|
||||
const handleMapClick = () => {
|
||||
setMapClicked(true);
|
||||
};
|
||||
|
||||
const closeModal = () => {
|
||||
setMapClicked(false);
|
||||
};
|
||||
|
||||
const focusSearchBar = () => {
|
||||
setMapClicked(false);
|
||||
searchInputRef.current?.focus();
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="w-full max-w-4xl mx-auto">
|
||||
{/* Search Interface */}
|
||||
<>
|
||||
<div className="mb-4 flex gap-2">
|
||||
<div className="relative flex-1">
|
||||
<div className="absolute left-4 top-1/2 -translate-y-1/2 text-gray-400">
|
||||
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z" />
|
||||
</svg>
|
||||
</div>
|
||||
<input
|
||||
ref={searchInputRef}
|
||||
type="text"
|
||||
value={searchQuery}
|
||||
onChange={(e) => setSearchQuery(e.target.value)}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter' && searchQuery.trim().length >= 2 && !isCheckingReviews) {
|
||||
e.preventDefault();
|
||||
handleSearch();
|
||||
}
|
||||
}}
|
||||
placeholder="Business name and location (e.g., Soho Club Vilnius)..."
|
||||
className="w-full pl-12 pr-4 py-3 text-gray-900 bg-white border-2 border-gray-200 rounded-xl focus:border-blue-500 focus:ring-4 focus:ring-blue-100 outline-none transition-all"
|
||||
/>
|
||||
</div>
|
||||
<button
|
||||
onClick={handleSearch}
|
||||
disabled={searchQuery.trim().length < 2 || isCheckingReviews}
|
||||
className="px-6 py-3 bg-blue-600 text-white font-semibold rounded-xl hover:bg-blue-700 disabled:bg-gray-300 disabled:cursor-not-allowed transition-colors flex items-center gap-2"
|
||||
>
|
||||
{isCheckingReviews ? (
|
||||
<>
|
||||
<div className="w-4 h-4 border-2 border-white border-t-transparent rounded-full animate-spin" />
|
||||
Validating...
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M9 12l2 2 4-4m6 2a9 9 0 11-18 0 9 9 0 0118 0z" />
|
||||
</svg>
|
||||
Validate
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Map Preview with Click Overlay */}
|
||||
<div className="mb-4 rounded-xl overflow-hidden border-2 border-gray-200 bg-gray-100 relative">
|
||||
{searchedQuery ? (
|
||||
<>
|
||||
<iframe
|
||||
src={embedUrl}
|
||||
width="100%"
|
||||
height="350"
|
||||
style={{ border: 0, pointerEvents: 'none' }}
|
||||
allowFullScreen
|
||||
loading="lazy"
|
||||
referrerPolicy="no-referrer-when-downgrade"
|
||||
title="Google Maps"
|
||||
/>
|
||||
{/* Click detection overlay - always present to capture clicks */}
|
||||
<div
|
||||
className="absolute inset-0 cursor-pointer"
|
||||
onClick={handleMapClick}
|
||||
/>
|
||||
{/* Modal centered on map card */}
|
||||
{mapClicked && (
|
||||
<div
|
||||
className="absolute inset-0 flex items-center justify-center backdrop-blur-md bg-gray-900/30 p-4"
|
||||
onClick={closeModal}
|
||||
>
|
||||
<div
|
||||
className="bg-white rounded-2xl p-6 sm:p-8 shadow-2xl w-full max-w-md border-2 border-blue-500 animate-fade-in"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<div className="text-center mb-4 sm:mb-6">
|
||||
<div className="text-4xl sm:text-5xl mb-2 sm:mb-3">🎯</div>
|
||||
<p className="text-xl sm:text-2xl font-bold text-gray-900 mb-2">Want a specific business?</p>
|
||||
<p className="text-xs sm:text-sm text-gray-600">
|
||||
Search for the <strong>exact business name</strong> to scrape its reviews
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="bg-blue-50 border-2 border-blue-200 rounded-lg p-3 mb-4">
|
||||
<p className="text-xs text-blue-900 font-medium mb-1">💡 Example:</p>
|
||||
<p className="text-sm font-semibold text-blue-800">"Starbucks Downtown Seattle"</p>
|
||||
<p className="text-xs text-gray-500 mt-1">instead of just "coffee"</p>
|
||||
</div>
|
||||
|
||||
<div className="flex gap-2">
|
||||
<button
|
||||
onClick={focusSearchBar}
|
||||
className="flex-1 py-3 bg-blue-600 hover:bg-blue-700 text-white rounded-lg font-bold transition-all flex items-center justify-center gap-2 shadow-md"
|
||||
>
|
||||
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z" />
|
||||
</svg>
|
||||
Search
|
||||
</button>
|
||||
<button
|
||||
onClick={closeModal}
|
||||
className="px-4 py-3 bg-gray-200 hover:bg-gray-300 text-gray-700 rounded-lg font-bold transition-all"
|
||||
>
|
||||
✕
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
) : (
|
||||
<div className="h-[350px] flex items-center justify-center text-gray-400">
|
||||
<div className="text-center">
|
||||
<svg className="w-12 h-12 mx-auto mb-3 opacity-50" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M17.657 16.657L13.414 20.9a1.998 1.998 0 01-2.827 0l-4.244-4.243a8 8 0 1111.314 0z" />
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M15 11a3 3 0 11-6 0 3 3 0 016 0z" />
|
||||
</svg>
|
||||
<p>Search for a business to see the map</p>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Business Card - Validation Results */}
|
||||
{searchedQuery && hasReviews !== null && (
|
||||
<div className="mb-6">
|
||||
{hasReviews ? (
|
||||
// Success - Show Business Card
|
||||
<div className="bg-white border-2 border-green-500 rounded-2xl shadow-lg overflow-hidden mb-4">
|
||||
{/* Header */}
|
||||
<div className="bg-gradient-to-r from-green-500 to-emerald-500 px-6 py-4">
|
||||
<div className="flex items-center gap-2 text-white">
|
||||
<svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M5 13l4 4L19 7" />
|
||||
</svg>
|
||||
<span className="font-bold text-lg">Business Found</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Business Info */}
|
||||
<div className="p-6">
|
||||
{/* Business Name */}
|
||||
<h3 className="text-2xl font-bold text-gray-900 mb-3">{businessName}</h3>
|
||||
|
||||
{/* Rating */}
|
||||
{businessRating && (
|
||||
<div className="flex items-center gap-1 mb-3">
|
||||
<span className="text-2xl font-bold text-gray-900">{businessRating.toFixed(1)}</span>
|
||||
<div className="flex items-center ml-1">
|
||||
{[...Array(5)].map((_, i) => (
|
||||
<svg
|
||||
key={i}
|
||||
className={`w-5 h-5 ${i < Math.floor(businessRating) ? 'text-yellow-400' : 'text-gray-300'}`}
|
||||
fill="currentColor"
|
||||
viewBox="0 0 20 20"
|
||||
>
|
||||
<path d="M9.049 2.927c.3-.921 1.603-.921 1.902 0l1.07 3.292a1 1 0 00.95.69h3.462c.969 0 1.371 1.24.588 1.81l-2.8 2.034a1 1 0 00-.364 1.118l1.07 3.292c.3.921-.755 1.688-1.54 1.118l-2.8-2.034a1 1 0 00-1.175 0l-2.8 2.034c-.784.57-1.838-.197-1.539-1.118l1.07-3.292a1 1 0 00-.364-1.118L2.98 8.72c-.783-.57-.38-1.81.588-1.81h3.461a1 1 0 00.951-.69l1.07-3.292z" />
|
||||
</svg>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Address */}
|
||||
{businessAddress && (
|
||||
<div className="flex items-start gap-2 text-gray-600 mb-4">
|
||||
<span className="text-lg">📍</span>
|
||||
<span className="text-sm">{businessAddress}</span>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Start Scraping Button */}
|
||||
<form onSubmit={handlePreviewBusiness}>
|
||||
<button
|
||||
type="submit"
|
||||
disabled={isSubmitting}
|
||||
className="w-full py-4 bg-gradient-to-r from-green-600 to-emerald-600 hover:from-green-700 hover:to-emerald-700 text-white rounded-xl font-bold transition-all flex items-center justify-center gap-2 shadow-lg text-lg disabled:opacity-50 disabled:cursor-not-allowed"
|
||||
>
|
||||
{isSubmitting ? (
|
||||
<>
|
||||
<div className="w-5 h-5 border-2 border-white border-t-transparent rounded-full animate-spin" />
|
||||
Starting scrape...
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M13 10V3L4 14h7v7l9-11h-7z" />
|
||||
</svg>
|
||||
Start Scraping Reviews
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
) : (
|
||||
// No Reviews - Show Warning
|
||||
<div className="p-4 bg-yellow-50 border-2 border-yellow-300 rounded-xl">
|
||||
<div className="flex items-start gap-3">
|
||||
<div className="w-10 h-10 bg-yellow-500 rounded-lg flex items-center justify-center flex-shrink-0">
|
||||
<svg className="w-6 h-6 text-white" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-3L13.732 4c-.77-1.333-2.694-1.333-3.464 0L3.34 16c-.77 1.333.192 3 1.732 3z" />
|
||||
</svg>
|
||||
</div>
|
||||
<div className="flex-1">
|
||||
<p className="font-bold text-yellow-900 text-lg">No reviews available</p>
|
||||
{businessName && (
|
||||
<p className="text-sm text-yellow-800 mt-1">
|
||||
Business: <strong>{businessName}</strong>
|
||||
</p>
|
||||
)}
|
||||
<p className="text-xs text-yellow-700 mt-1">
|
||||
This business has no reviews to scrape. Try a different search.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
|
||||
{/* Error */}
|
||||
{error && (
|
||||
<div className="mb-6 p-4 bg-red-100 border-2 border-red-300 rounded-xl">
|
||||
<div className="flex items-start gap-3">
|
||||
<svg className="w-6 h-6 text-red-700 flex-shrink-0 mt-0.5" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zM8.707 7.293a1 1 0 00-1.414 1.414L8.586 10l-1.293 1.293a1 1 0 101.414 1.414L10 11.414l1.293 1.293a1 1 0 001.414-1.414L11.414 10l1.293-1.293a1 1 0 00-1.414-1.414L10 8.586 8.707 7.293z" clipRule="evenodd" />
|
||||
</svg>
|
||||
<div>
|
||||
<p className="font-bold text-red-900 text-lg">Error</p>
|
||||
<p className="text-red-800 mt-1">{error}</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Jobs List */}
|
||||
{jobs.size > 0 && (
|
||||
<div className="mb-6 space-y-4">
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<h2 className="text-2xl font-bold text-gray-900">
|
||||
Scraping Jobs
|
||||
</h2>
|
||||
<span className="px-3 py-1 bg-blue-100 text-blue-800 font-semibold rounded-full text-sm">
|
||||
{jobs.size} {jobs.size === 1 ? 'Job' : 'Jobs'}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{Array.from(jobs.values())
|
||||
.sort((a, b) => new Date(b.created_at).getTime() - new Date(a.created_at).getTime())
|
||||
.map(job => (
|
||||
<div
|
||||
key={job.job_id}
|
||||
className={`p-6 rounded-xl transition-all shadow-md ${
|
||||
job.job_id === activeJobId
|
||||
? 'bg-blue-50 border-2 border-blue-500 shadow-lg'
|
||||
: 'bg-white border-2 border-gray-300'
|
||||
}`}
|
||||
>
|
||||
<div className="flex items-start justify-between mb-4">
|
||||
<div className="flex-1">
|
||||
<div className="flex items-center gap-2 mb-3">
|
||||
{getStatusIcon(job.status)}
|
||||
<h3 className="text-lg font-bold text-gray-900">
|
||||
Status: <span className={`${getStatusColor(job.status)} font-extrabold`}>{job.status.toUpperCase()}</span>
|
||||
</h3>
|
||||
</div>
|
||||
<p className="text-xs font-mono text-gray-600 mb-2 bg-gray-100 px-2 py-1 rounded inline-block">{job.job_id}</p>
|
||||
<p className="text-sm text-gray-700 truncate max-w-2xl font-medium">{job.url}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Progress Bar for Running Jobs */}
|
||||
{job.status === 'running' && job.total_reviews !== null && job.reviews_count !== null && (
|
||||
<div className="mb-4 p-4 bg-blue-50 border-2 border-blue-200 rounded-lg">
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<span className="text-sm font-bold text-blue-900">Extracting Reviews</span>
|
||||
<span className="text-sm font-bold text-blue-700">
|
||||
{job.reviews_count} / {job.total_reviews}
|
||||
</span>
|
||||
</div>
|
||||
<div className="w-full bg-blue-200 rounded-full h-3 overflow-hidden">
|
||||
<div
|
||||
className="bg-gradient-to-r from-blue-500 to-indigo-600 h-3 rounded-full transition-all duration-500 ease-out flex items-center justify-end pr-1"
|
||||
style={{ width: `${Math.min((job.reviews_count / job.total_reviews) * 100, 100)}%` }}
|
||||
>
|
||||
{job.reviews_count > 0 && (
|
||||
<span className="text-xs font-bold text-white drop-shadow">
|
||||
{Math.round((job.reviews_count / job.total_reviews) * 100)}%
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="grid grid-cols-2 md:grid-cols-4 gap-4 mb-4">
|
||||
{job.reviews_count !== null && (
|
||||
<div className="p-4 bg-blue-100 border-2 border-blue-200 rounded-lg">
|
||||
<div className="text-3xl font-bold text-blue-800">{job.reviews_count}</div>
|
||||
<div className="text-xs font-semibold text-blue-700 mt-1">Reviews</div>
|
||||
</div>
|
||||
)}
|
||||
{job.scrape_time !== null && (
|
||||
<div className="p-4 bg-green-100 border-2 border-green-200 rounded-lg">
|
||||
<div className="text-3xl font-bold text-green-800">{job.scrape_time.toFixed(1)}s</div>
|
||||
<div className="text-xs font-semibold text-green-700 mt-1">Time</div>
|
||||
</div>
|
||||
)}
|
||||
{job.scrape_time && job.reviews_count && (
|
||||
<div className="p-4 bg-purple-100 border-2 border-purple-200 rounded-lg">
|
||||
<div className="text-3xl font-bold text-purple-800">
|
||||
{(job.reviews_count / job.scrape_time).toFixed(1)}
|
||||
</div>
|
||||
<div className="text-xs font-semibold text-purple-700 mt-1">Reviews/sec</div>
|
||||
</div>
|
||||
)}
|
||||
{job.started_at && (
|
||||
<div className="p-4 bg-gray-100 border-2 border-gray-300 rounded-lg">
|
||||
<div className="text-lg font-bold text-gray-800">
|
||||
{new Date(job.started_at).toLocaleTimeString()}
|
||||
</div>
|
||||
<div className="text-xs font-semibold text-gray-700 mt-1">Started</div>
|
||||
</div>
|
||||
)}
|
||||
{job.status === 'running' && job.updated_at && (
|
||||
<div className="p-4 bg-blue-100 border-2 border-blue-200 rounded-lg">
|
||||
<div className="text-lg font-bold text-blue-800">
|
||||
{new Date(job.updated_at).toLocaleTimeString()}
|
||||
</div>
|
||||
<div className="text-xs font-semibold text-blue-700 mt-1">Last Update</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Action Buttons - Show when completed */}
|
||||
{job.status === 'completed' && (
|
||||
<div className="flex gap-3">
|
||||
<button
|
||||
onClick={async () => {
|
||||
setError('');
|
||||
setIsLoadingReviews(true);
|
||||
|
||||
try {
|
||||
console.log('Fetching reviews for job:', job.job_id);
|
||||
const reviewsResponse = await fetch(`/api/jobs/${job.job_id}/reviews?limit=10000`);
|
||||
|
||||
if (!reviewsResponse.ok) {
|
||||
throw new Error(`Failed to fetch reviews: ${reviewsResponse.status}`);
|
||||
}
|
||||
|
||||
const reviewsData = await reviewsResponse.json();
|
||||
console.log('Reviews fetched:', reviewsData);
|
||||
|
||||
if (!reviewsData.reviews || reviewsData.reviews.length === 0) {
|
||||
setError('No reviews found for this job');
|
||||
setIsLoadingReviews(false);
|
||||
return;
|
||||
}
|
||||
|
||||
setReviews(reviewsData.reviews);
|
||||
setActiveJobId(job.job_id);
|
||||
setShowAnalytics(true);
|
||||
} catch (err) {
|
||||
console.error('Failed to fetch reviews:', err);
|
||||
setError(err instanceof Error ? err.message : 'Failed to load reviews for analysis');
|
||||
} finally {
|
||||
setIsLoadingReviews(false);
|
||||
}
|
||||
}}
|
||||
disabled={isLoadingReviews}
|
||||
className="flex-1 py-4 bg-gradient-to-r from-blue-600 to-indigo-700 text-white rounded-xl font-bold hover:from-blue-700 hover:to-indigo-800 transition-all flex items-center justify-center gap-2 shadow-lg disabled:opacity-50 disabled:cursor-not-allowed text-lg border-2 border-blue-500"
|
||||
>
|
||||
{isLoadingReviews ? (
|
||||
<>
|
||||
<div className="w-5 h-5 border-2 border-white border-t-transparent rounded-full animate-spin" />
|
||||
Loading Reviews...
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M9 19v-6a2 2 0 00-2-2H5a2 2 0 00-2 2v6a2 2 0 002 2h2a2 2 0 002-2zm0 0V9a2 2 0 012-2h2a2 2 0 012 2v10m-6 0a2 2 0 002 2h2a2 2 0 002-2m0 0V5a2 2 0 012-2h2a2 2 0 012 2v14a2 2 0 01-2 2h-2a2 2 0 01-2-2z" />
|
||||
</svg>
|
||||
📊 Open Analytics Dashboard
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
<button
|
||||
onClick={async () => {
|
||||
try {
|
||||
const reviewsResponse = await fetch(`/api/jobs/${job.job_id}/reviews?limit=10000`);
|
||||
if (reviewsResponse.ok) {
|
||||
const reviewsData = await reviewsResponse.json();
|
||||
const data = JSON.stringify(reviewsData.reviews, null, 2);
|
||||
const blob = new Blob([data], { type: 'application/json' });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = `reviews-${job.job_id}.json`;
|
||||
a.click();
|
||||
}
|
||||
} catch (err) {
|
||||
console.error('Failed to export reviews:', err);
|
||||
}
|
||||
}}
|
||||
className="px-6 py-4 bg-gray-700 hover:bg-gray-800 text-white border-2 border-gray-600 rounded-xl font-bold transition-colors flex items-center justify-center gap-2 shadow-md"
|
||||
>
|
||||
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 16v1a3 3 0 003 3h10a3 3 0 003-3v-1m-4-4l-4 4m0 0l-4-4m4 4V4" />
|
||||
</svg>
|
||||
Export JSON
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Error Message */}
|
||||
{job.status === 'failed' && job.error_message && (
|
||||
<div className="mt-4 p-4 bg-red-100 border-2 border-red-300 rounded-lg">
|
||||
<div className="flex items-start gap-2">
|
||||
<svg className="w-5 h-5 text-red-700 flex-shrink-0 mt-0.5" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zM8.707 7.293a1 1 0 00-1.414 1.414L8.586 10l-1.293 1.293a1 1 0 101.414 1.414L10 11.414l1.293 1.293a1 1 0 001.414-1.414L11.414 10l1.293-1.293a1 1 0 00-1.414-1.414L10 8.586 8.707 7.293z" clipRule="evenodd" />
|
||||
</svg>
|
||||
<div>
|
||||
<p className="font-bold text-red-900">Error</p>
|
||||
<p className="text-sm text-red-800 mt-1">{job.error_message}</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Analytics Dashboard or Simple Review List */}
|
||||
{reviews.length > 0 && (
|
||||
<>
|
||||
{showAnalytics ? (
|
||||
<div>
|
||||
<div className="mb-4">
|
||||
<button
|
||||
onClick={() => setShowAnalytics(false)}
|
||||
className="flex items-center gap-2 text-blue-600 hover:text-blue-700 font-medium"
|
||||
>
|
||||
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M10 19l-7-7m0 0l7-7m-7 7h18" />
|
||||
</svg>
|
||||
Back to Simple View
|
||||
</button>
|
||||
</div>
|
||||
<ReviewAnalytics reviews={reviews} businessName={searchedQuery || 'Business'} />
|
||||
</div>
|
||||
) : (
|
||||
<div>
|
||||
<div className="flex items-center justify-between mb-4">
|
||||
<h3 className="text-xl font-bold text-gray-900">
|
||||
Reviews ({reviews.length})
|
||||
</h3>
|
||||
<button
|
||||
onClick={() => {
|
||||
const data = JSON.stringify(reviews, null, 2);
|
||||
const blob = new Blob([data], { type: 'application/json' });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = `reviews-${activeJobId || 'export'}.json`;
|
||||
a.click();
|
||||
}}
|
||||
className="px-4 py-2 text-sm bg-gray-700 hover:bg-gray-800 text-white border-2 border-gray-600 rounded-lg font-bold transition-colors shadow-md"
|
||||
>
|
||||
Export JSON
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="space-y-3 max-h-[600px] overflow-y-auto pr-2">
|
||||
{reviews.map((review, index) => (
|
||||
<div key={`${index}-${review.review_id}`} className="p-4 bg-white border border-gray-200 rounded-xl hover:border-gray-300 transition-colors">
|
||||
<div className="flex items-start gap-3">
|
||||
{review.avatar_url && (
|
||||
<img
|
||||
src={review.avatar_url}
|
||||
alt={review.author}
|
||||
className="w-10 h-10 rounded-full"
|
||||
/>
|
||||
)}
|
||||
<div className="flex-1 min-w-0">
|
||||
<div className="flex items-center justify-between mb-1">
|
||||
<span className="font-medium text-gray-900">{review.author}</span>
|
||||
<div className="flex items-center gap-1">
|
||||
{[...Array(5)].map((_, i) => (
|
||||
<svg
|
||||
key={i}
|
||||
className={`w-4 h-4 ${i < review.rating ? 'text-yellow-400' : 'text-gray-300'}`}
|
||||
fill="currentColor"
|
||||
viewBox="0 0 20 20"
|
||||
>
|
||||
<path d="M9.049 2.927c.3-.921 1.603-.921 1.902 0l1.07 3.292a1 1 0 00.95.69h3.462c.969 0 1.371 1.24.588 1.81l-2.8 2.034a1 1 0 00-.364 1.118l1.07 3.292c.3.921-.755 1.688-1.54 1.118l-2.8-2.034a1 1 0 00-1.175 0l-2.8 2.034c-.784.57-1.838-.197-1.539-1.118l1.07-3.292a1 1 0 00-.364-1.118L2.98 8.72c-.783-.57-.38-1.81.588-1.81h3.461a1 1 0 00.951-.69l1.07-3.292z" />
|
||||
</svg>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-xs text-gray-500 mb-2">{review.date_text}</p>
|
||||
{review.text && (
|
||||
<p className="text-sm text-gray-700 leading-relaxed">{review.text}</p>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
|
||||
{/* Confirmation Modal */}
|
||||
{showConfirmModal && (
|
||||
<div
|
||||
className="fixed inset-0 z-50 flex items-center justify-center bg-black/50 backdrop-blur-sm p-4"
|
||||
onClick={() => setShowConfirmModal(false)}
|
||||
>
|
||||
<div
|
||||
className="bg-white rounded-2xl shadow-2xl w-full max-w-md border-2 border-green-500 animate-fade-in"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
{/* Header */}
|
||||
<div className="bg-gradient-to-r from-green-600 to-emerald-600 text-white px-6 py-5 rounded-t-xl">
|
||||
<div className="flex items-center gap-3">
|
||||
<div className="w-10 h-10 bg-white/20 rounded-lg flex items-center justify-center">
|
||||
<svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M13 10V3L4 14h7v7l9-11h-7z" />
|
||||
</svg>
|
||||
</div>
|
||||
<h2 className="text-xl font-bold">Start Scraping?</h2>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Content */}
|
||||
<div className="px-6 py-5">
|
||||
<p className="text-gray-700 mb-4">
|
||||
This will start scraping reviews for:
|
||||
</p>
|
||||
<div className="bg-green-50 border-2 border-green-200 rounded-lg p-4 mb-4">
|
||||
<p className="font-bold text-green-900 text-lg">{businessName}</p>
|
||||
{businessAddress && (
|
||||
<p className="text-sm text-green-700 mt-1">{businessAddress}</p>
|
||||
)}
|
||||
</div>
|
||||
<p className="text-sm text-gray-600">
|
||||
The scraping job will run in the background. You can monitor progress below.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* Actions */}
|
||||
<div className="px-6 py-4 bg-gray-50 rounded-b-xl border-t-2 border-gray-200 flex gap-3">
|
||||
<button
|
||||
onClick={() => setShowConfirmModal(false)}
|
||||
className="flex-1 py-3 px-4 bg-gray-200 hover:bg-gray-300 text-gray-800 rounded-lg font-semibold transition-all"
|
||||
>
|
||||
Cancel
|
||||
</button>
|
||||
<button
|
||||
onClick={handleConfirmScrape}
|
||||
disabled={isSubmitting}
|
||||
className="flex-1 py-3 px-4 bg-gradient-to-r from-green-600 to-emerald-600 hover:from-green-700 hover:to-emerald-700 text-white rounded-lg font-semibold transition-all flex items-center justify-center gap-2 disabled:opacity-50"
|
||||
>
|
||||
{isSubmitting ? (
|
||||
<>
|
||||
<div className="w-4 h-4 border-2 border-white border-t-transparent rounded-full animate-spin" />
|
||||
Starting...
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M5 13l4 4L19 7" />
|
||||
</svg>
|
||||
Confirm
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
18
web/eslint.config.mjs
Normal file
18
web/eslint.config.mjs
Normal file
@@ -0,0 +1,18 @@
|
||||
import { defineConfig, globalIgnores } from "eslint/config";
|
||||
import nextVitals from "eslint-config-next/core-web-vitals";
|
||||
import nextTs from "eslint-config-next/typescript";
|
||||
|
||||
const eslintConfig = defineConfig([
|
||||
...nextVitals,
|
||||
...nextTs,
|
||||
// Override default ignores of eslint-config-next.
|
||||
globalIgnores([
|
||||
// Default ignores of eslint-config-next:
|
||||
".next/**",
|
||||
"out/**",
|
||||
"build/**",
|
||||
"next-env.d.ts",
|
||||
]),
|
||||
]);
|
||||
|
||||
export default eslintConfig;
|
||||
398
web/lib/analytics.ts
Normal file
398
web/lib/analytics.ts
Normal file
@@ -0,0 +1,398 @@
|
||||
// Analytics utility functions
|
||||
|
||||
export interface Review {
|
||||
author: string;
|
||||
rating: number;
|
||||
text: string | null;
|
||||
date_text: string;
|
||||
avatar_url: string | null;
|
||||
profile_url: string | null;
|
||||
review_id: string;
|
||||
// Derived fields (computed on load)
|
||||
parsedDate?: Date;
|
||||
dateCategory?: 'recent' | 'month' | 'year' | 'older'; // Time range category
|
||||
minDate?: Date; // Earliest possible date (accounting for Google's uncertainty)
|
||||
maxDate?: Date; // Latest possible date (accounting for Google's uncertainty)
|
||||
centerDate?: Date; // Midpoint of the range (used for calculations and sorting)
|
||||
}
|
||||
|
||||
export type DateRange = 'week' | 'month' | 'year' | 'all';
|
||||
|
||||
export interface TimelineDataPoint {
|
||||
date: string;
|
||||
rating: number;
|
||||
rollingAvg: number;
|
||||
}
|
||||
|
||||
export interface ReviewStats {
|
||||
totalReviews: number;
|
||||
averageRating: number;
|
||||
sentimentScore: number;
|
||||
photoCount: number;
|
||||
avgReviewLength: number;
|
||||
recentReviews: number;
|
||||
ratingDistribution: { rating: number; count: number; percentage: number }[];
|
||||
topKeywords: { word: string; count: number }[];
|
||||
sentimentBreakdown: { positive: number; neutral: number; negative: number };
|
||||
negativeReviews: number;
|
||||
responseRate: number;
|
||||
averageResponseTime: string;
|
||||
}
|
||||
|
||||
export function calculateReviewStats(reviews: Review[]): ReviewStats {
|
||||
// Populate minDate/maxDate/centerDate on reviews for display
|
||||
reviews.forEach(r => {
|
||||
if (!r.minDate || !r.maxDate || !r.centerDate) {
|
||||
const range = parseDateTextToRange(r.date_text);
|
||||
r.minDate = range.minDate;
|
||||
r.maxDate = range.maxDate;
|
||||
// Calculate centerDate as midpoint
|
||||
const midpointTime = (range.minDate.getTime() + range.maxDate.getTime()) / 2;
|
||||
r.centerDate = new Date(midpointTime);
|
||||
}
|
||||
});
|
||||
|
||||
const totalReviews = reviews.length;
|
||||
|
||||
// Average rating
|
||||
const averageRating = reviews.reduce((sum, r) => sum + r.rating, 0) / totalReviews;
|
||||
|
||||
// Sentiment score (% of 4-5 star reviews)
|
||||
const positiveReviews = reviews.filter(r => r.rating >= 4).length;
|
||||
const sentimentScore = (positiveReviews / totalReviews) * 100;
|
||||
|
||||
// Photo count (reviews with avatars as proxy)
|
||||
const photoCount = reviews.filter(r => r.avatar_url).length;
|
||||
|
||||
// Average review length
|
||||
const avgReviewLength = Math.round(
|
||||
reviews.reduce((sum, r) => sum + (r.text?.split(' ').length || 0), 0) / totalReviews
|
||||
);
|
||||
|
||||
// Recent reviews (last 30 days - simplified check)
|
||||
const recentReviews = reviews.filter(r => {
|
||||
const text = r.date_text.toLowerCase();
|
||||
return text.includes('day') || text.includes('week') || text.includes('hour');
|
||||
}).length;
|
||||
|
||||
// Rating distribution
|
||||
const ratingCounts: Record<number, number> = { 1: 0, 2: 0, 3: 0, 4: 0, 5: 0 };
|
||||
reviews.forEach(r => {
|
||||
ratingCounts[r.rating] = (ratingCounts[r.rating] || 0) + 1;
|
||||
});
|
||||
|
||||
const ratingDistribution = [5, 4, 3, 2, 1].map(rating => ({
|
||||
rating,
|
||||
count: ratingCounts[rating] || 0,
|
||||
percentage: ((ratingCounts[rating] || 0) / totalReviews) * 100,
|
||||
}));
|
||||
|
||||
// Extract keywords from review text
|
||||
const allWords = reviews
|
||||
.filter(r => r.text)
|
||||
.flatMap(r =>
|
||||
r.text!
|
||||
.toLowerCase()
|
||||
.replace(/[^\w\s]/g, '')
|
||||
.split(/\s+/)
|
||||
.filter(w => w.length > 3)
|
||||
);
|
||||
|
||||
const stopWords = new Set(['this', 'that', 'with', 'from', 'have', 'been', 'were', 'very', 'great', 'good', 'best', 'nice', 'here', 'there', 'they', 'their', 'about', 'would', 'could', 'should', 'place', 'really']);
|
||||
|
||||
const wordCounts: Record<string, number> = {};
|
||||
allWords.forEach(word => {
|
||||
if (!stopWords.has(word)) {
|
||||
wordCounts[word] = (wordCounts[word] || 0) + 1;
|
||||
}
|
||||
});
|
||||
|
||||
const topKeywords = Object.entries(wordCounts)
|
||||
.sort(([, a], [, b]) => b - a)
|
||||
.slice(0, 10)
|
||||
.map(([word, count]) => ({ word, count }));
|
||||
|
||||
// Sentiment breakdown
|
||||
const sentimentBreakdown = {
|
||||
positive: reviews.filter(r => r.rating >= 4).length,
|
||||
neutral: reviews.filter(r => r.rating === 3).length,
|
||||
negative: reviews.filter(r => r.rating <= 2).length,
|
||||
};
|
||||
|
||||
// Negative reviews count
|
||||
const negativeReviews = reviews.filter(r => r.rating <= 2).length;
|
||||
|
||||
// Response rate (placeholder - would need owner_response field)
|
||||
const responseRate = 0; // TODO: Calculate when owner responses are available
|
||||
|
||||
// Average response time (placeholder)
|
||||
const averageResponseTime = 'N/A'; // TODO: Calculate when response data is available
|
||||
|
||||
return {
|
||||
totalReviews,
|
||||
averageRating,
|
||||
sentimentScore,
|
||||
photoCount,
|
||||
avgReviewLength,
|
||||
recentReviews,
|
||||
ratingDistribution,
|
||||
topKeywords,
|
||||
sentimentBreakdown,
|
||||
negativeReviews,
|
||||
responseRate,
|
||||
averageResponseTime,
|
||||
};
|
||||
}
|
||||
|
||||
export function getSentimentLabel(rating: number): 'positive' | 'neutral' | 'negative' {
|
||||
if (rating >= 4) return 'positive';
|
||||
if (rating === 3) return 'neutral';
|
||||
return 'negative';
|
||||
}
|
||||
|
||||
// Helper function to get date range boundaries for preset buttons
|
||||
export function getDateRangeBoundaries(range: DateRange): { from: Date | null; to: Date | null } {
|
||||
if (range === 'all') return { from: null, to: null };
|
||||
|
||||
const now = new Date();
|
||||
const to = new Date(now); // Today as end date
|
||||
const from = new Date();
|
||||
|
||||
switch (range) {
|
||||
case 'week':
|
||||
from.setDate(now.getDate() - 7);
|
||||
break;
|
||||
case 'month':
|
||||
from.setMonth(now.getMonth() - 1);
|
||||
break;
|
||||
case 'year':
|
||||
from.setFullYear(now.getFullYear() - 1);
|
||||
break;
|
||||
}
|
||||
|
||||
// Set to start of day for from, end of day for to
|
||||
from.setHours(0, 0, 0, 0);
|
||||
to.setHours(23, 59, 59, 999);
|
||||
|
||||
return { from, to };
|
||||
}
|
||||
|
||||
export function getSentimentColor(sentiment: 'positive' | 'neutral' | 'negative'): string {
|
||||
switch (sentiment) {
|
||||
case 'positive': return 'text-green-700 bg-green-50 border-green-300';
|
||||
case 'neutral': return 'text-yellow-700 bg-yellow-50 border-yellow-300';
|
||||
case 'negative': return 'text-red-700 bg-red-50 border-red-300';
|
||||
}
|
||||
}
|
||||
|
||||
function extractNumber(text: string): number {
|
||||
// Extract first number from text (e.g., "2 weeks ago" -> 2, "Hace 2 semanas" -> 2)
|
||||
const match = text.match(/\d+/);
|
||||
if (match) {
|
||||
return parseInt(match[0]);
|
||||
}
|
||||
// Handle singular: "a month ago", "un mes", "una semana"
|
||||
if (text.match(/^a\s+\w+\s+ago/) || text.includes('un ') || text.includes('una ')) {
|
||||
return 1;
|
||||
}
|
||||
return 1; // Default to 1 if no number found
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse date_text into time range boundaries (min/max dates)
|
||||
*
|
||||
* This accounts for Google's inherent uncertainty in relative dates.
|
||||
* Based on reverse-engineered patterns from 244 reviews.
|
||||
*
|
||||
* Examples:
|
||||
* - "a month ago" → { min: 30 days ago, max: 59 days ago }
|
||||
* - "2 months ago" → { min: 60 days ago, max: 89 days ago }
|
||||
* - "a year ago" → { min: 365 days ago, max: 729 days ago }
|
||||
*/
|
||||
export function parseDateTextToRange(dateText: string): { minDate: Date; maxDate: Date } {
|
||||
const now = new Date();
|
||||
const text = dateText.toLowerCase();
|
||||
|
||||
// Remove "Edited " prefix if present
|
||||
const cleaned = text.replace(/^edited\s+/i, '');
|
||||
|
||||
// Helper to create date from days ago
|
||||
const daysAgo = (days: number) => new Date(now.getTime() - days * 24 * 60 * 60 * 1000);
|
||||
|
||||
// Seconds: 1-59 seconds
|
||||
if (cleaned.includes('second')) {
|
||||
const seconds = extractNumber(cleaned);
|
||||
const minDate = new Date(now.getTime() - seconds * 1000);
|
||||
const maxDate = new Date(now.getTime() - seconds * 1000);
|
||||
return { minDate, maxDate };
|
||||
}
|
||||
|
||||
// Minutes: 1-59 minutes
|
||||
if (cleaned.includes('minute')) {
|
||||
const minutes = extractNumber(cleaned);
|
||||
const minDate = new Date(now.getTime() - minutes * 60 * 1000);
|
||||
const maxDate = new Date(now.getTime() - minutes * 60 * 1000);
|
||||
return { minDate, maxDate };
|
||||
}
|
||||
|
||||
// Hours: 1-23 hours
|
||||
if (cleaned.includes('hora') || cleaned.includes('hour')) {
|
||||
const hours = extractNumber(cleaned);
|
||||
const minDate = new Date(now.getTime() - hours * 60 * 60 * 1000);
|
||||
const maxDate = new Date(now.getTime() - hours * 60 * 60 * 1000);
|
||||
return { minDate, maxDate };
|
||||
}
|
||||
|
||||
// Days: 1-6 days
|
||||
if (cleaned.includes('día') || cleaned.includes('day')) {
|
||||
const days = extractNumber(cleaned);
|
||||
const minDate = daysAgo(days);
|
||||
const maxDate = daysAgo(days);
|
||||
return { minDate, maxDate };
|
||||
}
|
||||
|
||||
// Weeks: 2-3 weeks (Google never shows "1 week ago" or "4 weeks ago")
|
||||
if (cleaned.includes('semana') || cleaned.includes('week')) {
|
||||
const weeks = extractNumber(cleaned);
|
||||
// Each week pattern represents a 7-day range
|
||||
const minDays = weeks * 7;
|
||||
const maxDays = weeks * 7 + 6; // Up to 6 extra days before switching to next week
|
||||
return { minDate: daysAgo(maxDays), maxDate: daysAgo(minDays) };
|
||||
}
|
||||
|
||||
// Months: Singular "a month ago" or plural "2-11 months ago"
|
||||
if (cleaned.includes('mes') || cleaned.includes('month')) {
|
||||
const months = extractNumber(cleaned);
|
||||
|
||||
// "a month ago" = 30-59 days (before switching to "2 months ago")
|
||||
if (months === 1) {
|
||||
return { minDate: daysAgo(59), maxDate: daysAgo(30) };
|
||||
}
|
||||
|
||||
// "2 months ago" = 60-89 days
|
||||
// "3 months ago" = 90-119 days
|
||||
// Pattern: N months = (N*30) to ((N+1)*30 - 1) days
|
||||
const minDays = months * 30;
|
||||
const maxDays = (months + 1) * 30 - 1;
|
||||
return { minDate: daysAgo(maxDays), maxDate: daysAgo(minDays) };
|
||||
}
|
||||
|
||||
// Years: Singular "a year ago" or plural "2-11 years ago"
|
||||
if (cleaned.includes('año') || cleaned.includes('year')) {
|
||||
const years = extractNumber(cleaned);
|
||||
|
||||
// "a year ago" = 365-729 days (12-24 months before switching to "2 years ago")
|
||||
if (years === 1) {
|
||||
return { minDate: daysAgo(729), maxDate: daysAgo(365) };
|
||||
}
|
||||
|
||||
// "2 years ago" = 730-1094 days (24-36 months)
|
||||
// Pattern: N years = (N*365) to ((N+1)*365 - 1) days
|
||||
const minDays = years * 365;
|
||||
const maxDays = (years + 1) * 365 - 1;
|
||||
return { minDate: daysAgo(maxDays), maxDate: daysAgo(minDays) };
|
||||
}
|
||||
|
||||
// Default: very old (10+ years)
|
||||
return { minDate: daysAgo(3650 + 365), maxDate: daysAgo(3650) };
|
||||
}
|
||||
|
||||
export function parseDateText(dateText: string): Date {
|
||||
// Get the time range and return the midpoint
|
||||
const { minDate, maxDate } = parseDateTextToRange(dateText);
|
||||
const midpointTime = (minDate.getTime() + maxDate.getTime()) / 2;
|
||||
return new Date(midpointTime);
|
||||
}
|
||||
|
||||
export function filterReviewsByDateRange(reviews: Review[], range: DateRange): Review[] {
|
||||
if (range === 'all') return reviews;
|
||||
|
||||
const now = new Date();
|
||||
const filterStart = new Date();
|
||||
|
||||
switch (range) {
|
||||
case 'week':
|
||||
filterStart.setDate(now.getDate() - 7);
|
||||
break;
|
||||
case 'month':
|
||||
filterStart.setMonth(now.getMonth() - 1);
|
||||
break;
|
||||
case 'year':
|
||||
filterStart.setFullYear(now.getFullYear() - 1);
|
||||
break;
|
||||
}
|
||||
|
||||
const filterEnd = now;
|
||||
|
||||
// Use range overlap logic: Include review if its time range overlaps with filter range
|
||||
// Review range: [minDate, maxDate]
|
||||
// Filter range: [filterStart, filterEnd]
|
||||
// Overlap occurs when: minDate <= filterEnd AND maxDate >= filterStart
|
||||
return reviews.filter(r => {
|
||||
const { minDate, maxDate } = parseDateTextToRange(r.date_text);
|
||||
return minDate <= filterEnd && maxDate >= filterStart;
|
||||
});
|
||||
}
|
||||
|
||||
export function filterReviewsByCustomDateRange(reviews: Review[], fromDate: Date | null, toDate: Date | null): Review[] {
|
||||
if (!fromDate && !toDate) return reviews;
|
||||
|
||||
return reviews.filter(r => {
|
||||
const reviewDate = parseDateText(r.date_text);
|
||||
|
||||
// If only fromDate is set, filter reviews >= fromDate
|
||||
if (fromDate && !toDate) {
|
||||
return reviewDate >= fromDate;
|
||||
}
|
||||
|
||||
// If only toDate is set, filter reviews <= toDate (end of day)
|
||||
if (!fromDate && toDate) {
|
||||
const endOfDay = new Date(toDate);
|
||||
endOfDay.setHours(23, 59, 59, 999);
|
||||
return reviewDate <= endOfDay;
|
||||
}
|
||||
|
||||
// Both dates set - filter reviews within range
|
||||
const endOfDay = new Date(toDate!);
|
||||
endOfDay.setHours(23, 59, 59, 999);
|
||||
return reviewDate >= fromDate! && reviewDate <= endOfDay;
|
||||
});
|
||||
}
|
||||
|
||||
export function calculateTimelineData(reviews: Review[]): TimelineDataPoint[] {
|
||||
// Sort reviews by date (newest first)
|
||||
const sortedReviews = [...reviews]
|
||||
.map(r => ({ ...r, parsedDate: parseDateText(r.date_text) }))
|
||||
.sort((a, b) => b.parsedDate.getTime() - a.parsedDate.getTime());
|
||||
|
||||
// Group by month
|
||||
const monthlyData: Record<string, { ratings: number[]; date: Date }> = {};
|
||||
|
||||
sortedReviews.forEach(review => {
|
||||
const monthKey = `${review.parsedDate.getFullYear()}-${String(review.parsedDate.getMonth() + 1).padStart(2, '0')}`;
|
||||
|
||||
if (!monthlyData[monthKey]) {
|
||||
monthlyData[monthKey] = { ratings: [], date: review.parsedDate };
|
||||
}
|
||||
monthlyData[monthKey].ratings.push(review.rating);
|
||||
});
|
||||
|
||||
// Calculate averages and rolling average
|
||||
const dataPoints: TimelineDataPoint[] = Object.entries(monthlyData)
|
||||
.map(([monthKey, data]) => ({
|
||||
date: monthKey,
|
||||
rating: data.ratings.reduce((a, b) => a + b, 0) / data.ratings.length,
|
||||
rollingAvg: 0, // Will calculate below
|
||||
}))
|
||||
.sort((a, b) => a.date.localeCompare(b.date));
|
||||
|
||||
// Calculate 3-month rolling average
|
||||
dataPoints.forEach((point, idx) => {
|
||||
const start = Math.max(0, idx - 2);
|
||||
const end = idx + 1;
|
||||
const window = dataPoints.slice(start, end);
|
||||
point.rollingAvg = window.reduce((sum, p) => sum + p.rating, 0) / window.length;
|
||||
});
|
||||
|
||||
return dataPoints;
|
||||
}
|
||||
7
web/next.config.ts
Normal file
7
web/next.config.ts
Normal file
@@ -0,0 +1,7 @@
|
||||
import type { NextConfig } from "next";
|
||||
|
||||
const nextConfig: NextConfig = {
|
||||
/* config options here */
|
||||
};
|
||||
|
||||
export default nextConfig;
|
||||
7
web/postcss.config.mjs
Normal file
7
web/postcss.config.mjs
Normal file
@@ -0,0 +1,7 @@
|
||||
const config = {
|
||||
plugins: {
|
||||
"@tailwindcss/postcss": {},
|
||||
},
|
||||
};
|
||||
|
||||
export default config;
|
||||
1
web/public/file.svg
Normal file
1
web/public/file.svg
Normal file
@@ -0,0 +1 @@
|
||||
<svg fill="none" viewBox="0 0 16 16" xmlns="http://www.w3.org/2000/svg"><path d="M14.5 13.5V5.41a1 1 0 0 0-.3-.7L9.8.29A1 1 0 0 0 9.08 0H1.5v13.5A2.5 2.5 0 0 0 4 16h8a2.5 2.5 0 0 0 2.5-2.5m-1.5 0v-7H8v-5H3v12a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1M9.5 5V2.12L12.38 5zM5.13 5h-.62v1.25h2.12V5zm-.62 3h7.12v1.25H4.5zm.62 3h-.62v1.25h7.12V11z" clip-rule="evenodd" fill="#666" fill-rule="evenodd"/></svg>
|
||||
|
After Width: | Height: | Size: 391 B |
1
web/public/globe.svg
Normal file
1
web/public/globe.svg
Normal file
@@ -0,0 +1 @@
|
||||
<svg fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16"><g clip-path="url(#a)"><path fill-rule="evenodd" clip-rule="evenodd" d="M10.27 14.1a6.5 6.5 0 0 0 3.67-3.45q-1.24.21-2.7.34-.31 1.83-.97 3.1M8 16A8 8 0 1 0 8 0a8 8 0 0 0 0 16m.48-1.52a7 7 0 0 1-.96 0H7.5a4 4 0 0 1-.84-1.32q-.38-.89-.63-2.08a40 40 0 0 0 3.92 0q-.25 1.2-.63 2.08a4 4 0 0 1-.84 1.31zm2.94-4.76q1.66-.15 2.95-.43a7 7 0 0 0 0-2.58q-1.3-.27-2.95-.43a18 18 0 0 1 0 3.44m-1.27-3.54a17 17 0 0 1 0 3.64 39 39 0 0 1-4.3 0 17 17 0 0 1 0-3.64 39 39 0 0 1 4.3 0m1.1-1.17q1.45.13 2.69.34a6.5 6.5 0 0 0-3.67-3.44q.65 1.26.98 3.1M8.48 1.5l.01.02q.41.37.84 1.31.38.89.63 2.08a40 40 0 0 0-3.92 0q.25-1.2.63-2.08a4 4 0 0 1 .85-1.32 7 7 0 0 1 .96 0m-2.75.4a6.5 6.5 0 0 0-3.67 3.44 29 29 0 0 1 2.7-.34q.31-1.83.97-3.1M4.58 6.28q-1.66.16-2.95.43a7 7 0 0 0 0 2.58q1.3.27 2.95.43a18 18 0 0 1 0-3.44m.17 4.71q-1.45-.12-2.69-.34a6.5 6.5 0 0 0 3.67 3.44q-.65-1.27-.98-3.1" fill="#666"/></g><defs><clipPath id="a"><path fill="#fff" d="M0 0h16v16H0z"/></clipPath></defs></svg>
|
||||
|
After Width: | Height: | Size: 1.0 KiB |
1
web/public/next.svg
Normal file
1
web/public/next.svg
Normal file
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 394 80"><path fill="#000" d="M262 0h68.5v12.7h-27.2v66.6h-13.6V12.7H262V0ZM149 0v12.7H94v20.4h44.3v12.6H94v21h55v12.6H80.5V0h68.7zm34.3 0h-17.8l63.8 79.4h17.9l-32-39.7 32-39.6h-17.9l-23 28.6-23-28.6zm18.3 56.7-9-11-27.1 33.7h17.8l18.3-22.7z"/><path fill="#000" d="M81 79.3 17 0H0v79.3h13.6V17l50.2 62.3H81Zm252.6-.4c-1 0-1.8-.4-2.5-1s-1.1-1.6-1.1-2.6.3-1.8 1-2.5 1.6-1 2.6-1 1.8.3 2.5 1a3.4 3.4 0 0 1 .6 4.3 3.7 3.7 0 0 1-3 1.8zm23.2-33.5h6v23.3c0 2.1-.4 4-1.3 5.5a9.1 9.1 0 0 1-3.8 3.5c-1.6.8-3.5 1.3-5.7 1.3-2 0-3.7-.4-5.3-1s-2.8-1.8-3.7-3.2c-.9-1.3-1.4-3-1.4-5h6c.1.8.3 1.6.7 2.2s1 1.2 1.6 1.5c.7.4 1.5.5 2.4.5 1 0 1.8-.2 2.4-.6a4 4 0 0 0 1.6-1.8c.3-.8.5-1.8.5-3V45.5zm30.9 9.1a4.4 4.4 0 0 0-2-3.3 7.5 7.5 0 0 0-4.3-1.1c-1.3 0-2.4.2-3.3.5-.9.4-1.6 1-2 1.6a3.5 3.5 0 0 0-.3 4c.3.5.7.9 1.3 1.2l1.8 1 2 .5 3.2.8c1.3.3 2.5.7 3.7 1.2a13 13 0 0 1 3.2 1.8 8.1 8.1 0 0 1 3 6.5c0 2-.5 3.7-1.5 5.1a10 10 0 0 1-4.4 3.5c-1.8.8-4.1 1.2-6.8 1.2-2.6 0-4.9-.4-6.8-1.2-2-.8-3.4-2-4.5-3.5a10 10 0 0 1-1.7-5.6h6a5 5 0 0 0 3.5 4.6c1 .4 2.2.6 3.4.6 1.3 0 2.5-.2 3.5-.6 1-.4 1.8-1 2.4-1.7a4 4 0 0 0 .8-2.4c0-.9-.2-1.6-.7-2.2a11 11 0 0 0-2.1-1.4l-3.2-1-3.8-1c-2.8-.7-5-1.7-6.6-3.2a7.2 7.2 0 0 1-2.4-5.7 8 8 0 0 1 1.7-5 10 10 0 0 1 4.3-3.5c2-.8 4-1.2 6.4-1.2 2.3 0 4.4.4 6.2 1.2 1.8.8 3.2 2 4.3 3.4 1 1.4 1.5 3 1.5 5h-5.8z"/></svg>
|
||||
|
After Width: | Height: | Size: 1.3 KiB |
1
web/public/vercel.svg
Normal file
1
web/public/vercel.svg
Normal file
@@ -0,0 +1 @@
|
||||
<svg fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1155 1000"><path d="m577.3 0 577.4 1000H0z" fill="#fff"/></svg>
|
||||
|
After Width: | Height: | Size: 128 B |
1
web/public/window.svg
Normal file
1
web/public/window.svg
Normal file
@@ -0,0 +1 @@
|
||||
<svg fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16"><path fill-rule="evenodd" clip-rule="evenodd" d="M1.5 2.5h13v10a1 1 0 0 1-1 1h-11a1 1 0 0 1-1-1zM0 1h16v11.5a2.5 2.5 0 0 1-2.5 2.5h-11A2.5 2.5 0 0 1 0 12.5zm3.75 4.5a.75.75 0 1 0 0-1.5.75.75 0 0 0 0 1.5M7 4.75a.75.75 0 1 1-1.5 0 .75.75 0 0 1 1.5 0m1.75.75a.75.75 0 1 0 0-1.5.75.75 0 0 0 0 1.5" fill="#666"/></svg>
|
||||
|
After Width: | Height: | Size: 385 B |
Reference in New Issue
Block a user