# Pipeline Core

Extensible multi-pipeline framework with dynamic dashboards.

## Overview

Pipeline Core provides the base abstractions for building pipelines that can be:
- Discovered and registered dynamically
- Executed with status tracking
- Rendered with auto-generated dashboards

## Features

- **BasePipeline** - Abstract base class all pipelines implement
- **PipelineRegistry** - Database-backed pipeline discovery and management
- **PipelineRunner** - Execution with status tracking
- **Dashboard Contracts** - TypedDicts for widget configuration

## Installation

```bash
pip install -e packages/pipeline-core
```

## Usage

### Implementing a Pipeline

```python
from pipeline_core import BasePipeline, PipelineMetadata, DashboardConfig

class MyPipeline(BasePipeline):
    @property
    def metadata(self) -> PipelineMetadata:
        return {
            "id": "my-pipeline",
            "name": "My Pipeline",
            "description": "Does something useful",
            "version": "1.0.0",
            "stages": ["stage1", "stage2"],
            "input_type": "MyInputType",
        }

    async def initialize(self) -> None:
        # Set up connections
        pass

    async def close(self) -> None:
        # Clean up
        pass

    async def process(self, input_data, stages=None):
        # Run the pipeline
        pass

    def get_dashboard_config(self) -> DashboardConfig:
        return {
            "pipeline_id": "my-pipeline",
            "title": "My Dashboard",
            "sections": [...]
        }

    async def get_widget_data(self, widget_id, params):
        # Return widget data
        pass
```

### Registering a Pipeline

```python
from pipeline_core import PipelineRegistry
import asyncpg

pool = await asyncpg.create_pool(database_url)
registry = PipelineRegistry(pool)

await registry.register(
    pipeline_id="my-pipeline",
    name="My Pipeline",
    description="Does something useful",
    version="1.0.0",
    module_path="my_package.pipeline:MyPipeline",
    stages=["stage1", "stage2"],
    input_type="MyInputType",
)
```

### Executing a Pipeline

```python
from pipeline_core import PipelineRunner

runner = PipelineRunner(pool, registry)

execution_id, result = await runner.execute(
    pipeline_id="my-pipeline",
    request={
        "input_data": {"key": "value"},
        "stages": ["stage1"],
    }
)
```

## Dashboard Widgets

Pipelines declare dashboard widgets via `get_dashboard_config()`. Available widget types:

- `stat_card` - KPI stat card with value and trend
- `line_chart` - Time series line chart
- `bar_chart` - Bar chart (horizontal or vertical)
- `pie_chart` - Pie/donut chart
- `table` - Data table with columns
- `heatmap` - Heatmap grid visualization
- `area_chart` - Stacked area chart
- `gauge` - Gauge/meter visualization

## License

MIT