Backfills
Going over how backfills work in Artie
Overview
The following circumstances may trigger a :
Deploying for the first time
When you launch a new Artie deployment, we will backfill all the tables.
Adding new tables
We will backfill the new tables you have added.
Adhoc backfills
Need to backfill a specific table on demand? Just let us know. We are working on making this feature self-serve in the future.
How it works
Read replica support
You can specify a read replica for backfills to minimize any impact on your primary database.
Parallel processing
Artie backfills by having parallel processes per table (default is 5). This speeds up the copying process without adding too much load to your database.
Real-time sync
While backfilling, Artie will continue to read your database logs, capturing all new changes in real-time into our Kafka topics. Once the backfill is complete, Artie process will then switch to consume from the Kafka topics.
Was this page helpful?