3317553658
Dashboard page:
- Fetch top clients from /api/dashboard/by-client
- Show loading state while fetching
- Display empty state when no client data
- Show real client_id, job count, and success rate
Scrapers page:
- Fetch versions from /api/admin/scrapers
- Wire promote/deprecate buttons to real API calls
- Wire add version form to POST /api/admin/scrapers
- Wire traffic allocation to PUT /api/admin/scrapers/{id}/traffic
- Add loading and error states
Dockerfile:
- Add COPY commands for new directories (api/, core/, scrapers/, etc.)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
110 lines
2.9 KiB
Docker
110 lines
2.9 KiB
Docker
FROM python:3.11-slim
|
|
|
|
# Install system dependencies for Chrome, Selenium, and Xvfb (virtual display)
|
|
RUN apt-get update && apt-get install -y \
|
|
# Basic utilities
|
|
wget \
|
|
gnupg \
|
|
unzip \
|
|
curl \
|
|
# Xvfb for virtual display (allows non-headless Chrome in container)
|
|
xvfb \
|
|
# Chrome dependencies
|
|
fonts-liberation \
|
|
libasound2 \
|
|
libatk-bridge2.0-0 \
|
|
libatk1.0-0 \
|
|
libatspi2.0-0 \
|
|
libcups2 \
|
|
libdbus-1-3 \
|
|
libdrm2 \
|
|
libgbm1 \
|
|
libgtk-3-0 \
|
|
libnspr4 \
|
|
libnss3 \
|
|
libwayland-client0 \
|
|
libxcomposite1 \
|
|
libxdamage1 \
|
|
libxfixes3 \
|
|
libxkbcommon0 \
|
|
libxrandr2 \
|
|
xdg-utils \
|
|
# Additional dependencies
|
|
libu2f-udev \
|
|
libvulkan1 \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Install Chromium (works on all architectures)
|
|
RUN apt-get update \
|
|
&& apt-get install -y chromium chromium-driver \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Install VNC server and noVNC (browser-based VNC viewer)
|
|
RUN apt-get update && apt-get install -y \
|
|
x11vnc \
|
|
novnc \
|
|
websockify \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Set working directory
|
|
WORKDIR /app
|
|
|
|
# Copy requirements and install Python dependencies
|
|
COPY requirements-production.txt .
|
|
RUN pip install --no-cache-dir -r requirements-production.txt
|
|
|
|
# Copy application code
|
|
COPY modules/ ./modules/
|
|
COPY api/ ./api/
|
|
COPY core/ ./core/
|
|
COPY scrapers/ ./scrapers/
|
|
COPY services/ ./services/
|
|
COPY utils/ ./utils/
|
|
COPY workers/ ./workers/
|
|
COPY api_server_production.py .
|
|
COPY config.yaml .
|
|
|
|
# Create startup script for Xvfb + VNC + API server
|
|
RUN echo '#!/bin/bash\n\
|
|
# Start Xvfb (virtual display) in background\n\
|
|
Xvfb :99 -screen 0 1920x1080x24 -ac +extension GLX +render -noreset &\n\
|
|
export DISPLAY=:99\n\
|
|
\n\
|
|
# Wait for Xvfb to start\n\
|
|
sleep 2\n\
|
|
\n\
|
|
# Start VNC server (no password for local dev, binds to all interfaces)\n\
|
|
x11vnc -display :99 -forever -shared -rfbport 5900 -nopw -bg\n\
|
|
\n\
|
|
# Start noVNC websocket proxy (browser access at http://localhost:6080/vnc.html)\n\
|
|
websockify --web=/usr/share/novnc/ 6080 localhost:5900 &\n\
|
|
\n\
|
|
echo "VNC server running on port 5900"\n\
|
|
echo "noVNC web interface at http://localhost:6080/vnc.html"\n\
|
|
\n\
|
|
# Start API server\n\
|
|
exec python api_server_production.py\n\
|
|
' > /app/start.sh && chmod +x /app/start.sh
|
|
|
|
# Create non-root user and give SeleniumBase write permissions
|
|
RUN useradd -m -u 1000 scraper && \
|
|
chown -R scraper:scraper /app && \
|
|
chown -R scraper:scraper /usr/local/lib/python3.11/site-packages/seleniumbase
|
|
|
|
USER scraper
|
|
|
|
# Expose ports: API (8000), VNC (5900), noVNC web (6080)
|
|
EXPOSE 8000 5900 6080
|
|
|
|
# Environment variables for Chromium in container
|
|
ENV DISPLAY=:99
|
|
ENV CHROME_BIN=/usr/bin/chromium
|
|
ENV CHROME_PATH=/usr/bin/chromium
|
|
|
|
# Health check
|
|
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
|
|
CMD curl -f http://localhost:8000/health/live || exit 1
|
|
|
|
# Run startup script (starts Xvfb + API server)
|
|
CMD ["/app/start.sh"]
|