Alejandro Gutiérrez
|
03ed7029e2
|
feat: Add decoupled pipeline schema with separate PostgreSQL namespace
- Create consolidated migration (005_create_pipeline_schema.sql) with
'pipeline' schema for all classification tables
- Update pipeline repositories to use schema prefix (pipeline.*)
- Add run_migrations() method to DatabaseManager
- Add CLI tool for running versioned migrations
Tables created in pipeline schema:
- reviews_raw, reviews_enriched (Stage 1)
- review_spans (Stage 2)
- issues, issue_spans, issue_events (Stage 3)
- fact_timeseries (Stage 4)
- urt_domains, urt_categories (taxonomy lookup)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
2026-01-24 18:17:20 +00:00 |
|
Alejandro Gutiérrez
|
2412996c54
|
Phase 1: Database migrations for platform features
Migrations created:
- 001_add_job_platform_fields.sql: Add 15 new columns to jobs table
- Requester tracking (client_id, source, purpose, metadata)
- Batch support (batch_id, batch_index)
- Execution tracking (job_type, scraper_version, variant, priority)
- Webhook callbacks (url, status, sent_at, attempts)
- Result summary (JSONB for cross-type dashboard)
- 7 indexes for query performance
- 5 CHECK constraints for data validation
- 002_create_batches_table.sql: Batch job grouping
- Tracks batch progress (total/completed/failed)
- Batch-level callbacks
- Requester association
- 003_create_scraper_registry.sql: Scraper version management
- Version routing (stable/beta/canary variants)
- A/B traffic splitting (traffic_pct)
- Priority-based routing
- Seeds google_reviews v1.0.0 as stable default
- 004_create_api_keys.sql: API authentication
- Secure key storage (SHA-256 hashes, not plaintext)
- Scopes-based permissions
- Rate limiting support
- Key lifecycle (expiry, active status)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
2026-01-24 15:24:28 +00:00 |
|