Phases 5-7: Dashboard UI, Admin API, and Auth middleware
Phase 5 - Main Dashboard: - Dashboard overview page with system health stats - Jobs by status breakdown, success rates, top clients - Dashboard API (/api/dashboard/overview, by-client, problems, by-version) Phase 6 - Admin/Scraper Management: - Scrapers management page with traffic allocation UI - Admin API for scraper CRUD operations - Traffic percentage updates for A/B testing - Promote/deprecate scraper versions Phase 7 - Authentication: - API key authentication middleware - SHA-256 key hashing (keys never stored in plain text) - Scope-based authorization (jobs:read, jobs:write, admin) - Rate limiting per API key Also: - Updated api_server_production.py to include new routers - Extended core/database.py with dashboard query methods - Added dashboard link to sidebar navigation - Updated CONTEXT-KEEPER.md to mark all phases complete Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -12,16 +12,23 @@
|
||||
|
||||
---
|
||||
|
||||
## Current State (as of 2025-01-24)
|
||||
## Current State (as of 2026-01-24)
|
||||
|
||||
### Working Features
|
||||
- Google Reviews scraper (`modules/scraper_clean.py`) - fully functional
|
||||
- Google Reviews scraper (`scrapers/google_reviews/v1_0_0.py`) - fully functional
|
||||
- Job queue with PostgreSQL storage
|
||||
- Real-time SSE streaming of logs/progress
|
||||
- Web UI for job management and analytics
|
||||
- Chrome pool for browser management
|
||||
- Crash detection and analysis
|
||||
- JobDevTools observability panel
|
||||
- **NEW**: Requester tracking (client_id, source, purpose)
|
||||
- **NEW**: Batch job submission API
|
||||
- **NEW**: Webhook/callback delivery with retries
|
||||
- **NEW**: Scraper versioning with A/B traffic routing
|
||||
- **NEW**: Main dashboard with system health stats
|
||||
- **NEW**: Admin API for scraper management
|
||||
- **NEW**: API key authentication middleware
|
||||
|
||||
### Repository
|
||||
- **Location**: `/Users/agutierrez/Desktop/google-reviews-scraper-pro`
|
||||
@@ -105,14 +112,14 @@ reviewiq/ # Will rename from google-reviews-scraper-pro
|
||||
|-------|-------------|--------|
|
||||
| 0 | Project restructure (move files to new locations) | ✅ COMPLETE |
|
||||
| 1 | Database migrations (new fields + tables) | ✅ COMPLETE |
|
||||
| 2 | Requester & batch support | Not started |
|
||||
| 3 | Webhooks | Not started |
|
||||
| 4 | Scraper versioning & registry | Not started |
|
||||
| 5 | Main dashboard UI | Not started |
|
||||
| 6 | A/B traffic management | Not started |
|
||||
| 7 | Authentication (API keys) | Not started |
|
||||
| 2 | Requester & batch support | ✅ COMPLETE |
|
||||
| 3 | Webhooks | ✅ COMPLETE |
|
||||
| 4 | Scraper versioning & registry | ✅ COMPLETE |
|
||||
| 5 | Main dashboard UI | ✅ COMPLETE |
|
||||
| 6 | A/B traffic management (Admin API) | ✅ COMPLETE |
|
||||
| 7 | Authentication middleware | ✅ COMPLETE |
|
||||
|
||||
**Phase 0 must complete first.** Then phases 1-5 can parallelize.
|
||||
**All phases complete!** Core platform ready for integration testing.
|
||||
|
||||
---
|
||||
|
||||
@@ -137,12 +144,20 @@ reviewiq/ # Will rename from google-reviews-scraper-pro
|
||||
|
||||
## Files to Know
|
||||
|
||||
| Current Location | Purpose |
|
||||
|------------------|---------|
|
||||
| `modules/scraper_clean.py` | Main Google Reviews scraper (96KB) |
|
||||
| `modules/database.py` | PostgreSQL database manager |
|
||||
| `api_server_production.py` | FastAPI server (will be split into api/) |
|
||||
| Location | Purpose |
|
||||
|----------|---------|
|
||||
| `scrapers/google_reviews/v1_0_0.py` | Main Google Reviews scraper (migrated) |
|
||||
| `scrapers/registry.py` | Scraper version registry with A/B routing |
|
||||
| `core/database.py` | PostgreSQL database manager |
|
||||
| `api_server_production.py` | FastAPI server with all routers |
|
||||
| `api/routes/dashboard.py` | Dashboard API endpoints |
|
||||
| `api/routes/admin.py` | Admin/scraper management API |
|
||||
| `api/routes/batches.py` | Batch job submission API |
|
||||
| `api/middleware/auth.py` | API key authentication middleware |
|
||||
| `web/app/dashboard/page.tsx` | Main dashboard UI |
|
||||
| `web/app/dashboard/scrapers/page.tsx` | Scraper management UI |
|
||||
| `web/app/jobs/[id]/page.tsx` | Job detail page with DevTools |
|
||||
| `migrations/versions/` | SQL migration files (001-004) |
|
||||
| `.artifacts/ReviewIQ-Platform-Spec.md` | Full specification document |
|
||||
|
||||
---
|
||||
@@ -177,4 +192,4 @@ When resuming after context compaction:
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-01-24*
|
||||
*Last updated: 2026-01-24*
|
||||
|
||||
Reference in New Issue
Block a user