> ## Documentation Index
> Fetch the complete documentation index at: https://artie.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Pipelines

> Learn how Artie pipelines replicate data from your source to your destination in real time.

A pipeline connects a source database to a destination and continuously streams changes. Once created, a pipeline handles initial backfills, schema evolution, and ongoing replication without manual intervention.

```mermaid theme={null}
graph LR
    subgraph shared [Shared Source Reader]
        SR[Source Reader]
        DB[(Source DB)]
        DB --> SR
    end

    subgraph pipelineA [Pipeline A]
        DA[Destination Connector A]
    end

    subgraph pipelineB [Pipeline B]
        DB2[Destination Connector B]
    end

    SR --> DA
    SR --> DB2
```

Each pipeline consists of:

* **Source reader** - Connects to your source database and captures changes via CDC. A source reader can be shared across multiple pipelines so your database is only read once.
* **Destination connector** - Writes changes to a specific destination (e.g. Snowflake, BigQuery, S3).

You can monitor pipeline health, throughput, and latency from the [analytics portal](/monitoring/analytics-portal).

## Writing to multiple destinations

Artie supports reading from a single source and writing to multiple destinations in parallel - for example, streaming changes to both S3 and Snowflake simultaneously. This is configured by creating multiple pipelines that share the same source reader, so your source database is only read once regardless of how many destinations you have.

This feature is currently available through [Terraform](https://registry.terraform.io/providers/artie-labs/artie/latest/docs/resources/source_reader).

To set it up, create `artie_source_reader` resource and reference it across multiple `artie_pipeline` resources.

## Pausing and resuming

You can pause a pipeline at any time from the pipeline overview page. Artie offers two pause modes:

* **Pause writing only** - Artie stops writing to the destination but continues capturing changes from the source. Events accumulate in Kafka, so no data is lost. If you resume within 14 days, all queued changes are applied in order and no backfill is needed.
* **Pause reading & writing** - Artie stops both reading from the source and writing to the destination entirely. No new changes are captured while paused.

<Frame>
  <img src="https://mintcdn.com/artie/qZV6ijhhUTveyBnA/assets/pipelines/pause.png?fit=max&auto=format&n=qZV6ijhhUTveyBnA&q=85&s=c08c61875b8f20fb38bd316d4e1f7f70" alt="Pause replication dialog showing the two pause modes" width="1597" height="892" data-path="assets/pipelines/pause.png" />
</Frame>

You can also pause individual tables within a pipeline from the tables tab in the pipeline editor.

<Frame>
  <img src="https://mintcdn.com/artie/qZV6ijhhUTveyBnA/assets/pipelines/pause_table.png?fit=max&auto=format&n=qZV6ijhhUTveyBnA&q=85&s=84e254c69a32d216bb4f16b6bd374b3e" alt="Pausing a table within a pipeline" width="1643" height="273" data-path="assets/pipelines/pause_table.png" />
</Frame>

## Autopilot

Autopilot is a feature that will automatically replicate new tables from your source database.

To enable autopilot:

1. Go to the pipeline editor
2. Click on the `Destination` tab
3. Click on the `Advanced settings` tab
4. Enable `Auto-replicate new tables`

<Frame>
  <img src="https://mintcdn.com/artie/Osg_R5VD67fPl1UB/assets/pipelines/autopilot.png?fit=max&auto=format&n=Osg_R5VD67fPl1UB&q=85&s=293a632324c23dd2b14738b6d9d5bf72" alt="Autopilot settings" width="858" height="369" data-path="assets/pipelines/autopilot.png" />
</Frame>
