> ## Documentation Index
> Fetch the complete documentation index at: https://artie.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Control how often data is loaded into your destination with Artie's flush rules. Balance between performance, latency and resource usage to optimize your data pipeline.

## How it works

<Tooltip tip="Flushing is the process of loading buffered data from memory into your destination database.">Flushing</Tooltip> is a critical part of Artie's data pipeline that determines when and how data gets written to your destination.

<Steps>
  <Step title="Data buffering">
    * Artie's reading process will read changes from your source database and publish them to Kafka
    * Artie's writing process will read messages from Kafka and write them to your destination
    * Messages are temporarily stored in memory and deduplicated based on primary key(s) or unique index
    * Multiple changes to the same record are merged to reduce write volume
  </Step>

  <Step title="Flush trigger evaluation">
    * Artie continuously monitors three flush conditions
    * When **any** condition is met, a flush is triggered
    * Reading from Kafka pauses during the flush operation
  </Step>

  <Step title="Data loading">
    * Buffered data is written to your destination in an optimized batch
    * After completion, Artie will commit the offset and resume reading from Kafka
    * The cycle repeats for continuous data flow
  </Step>
</Steps>

## Conditions

Artie evaluates three conditions to determine when to flush data. **Any one** of these conditions will trigger a flush:

<Card title="Time elapsed" icon="clock" horizontal>
  **Maximum time in seconds** - Ensures data freshness even during low-volume periods
</Card>

<Card title="Message count" icon="envelope" horizontal>
  **Number of deduplicated messages** - Based on unique primary keys or unique index.
</Card>

<Card title="Byte size" icon="hard-drive" horizontal>
  **Total bytes of deduplicated data** - Actual payload size after deduplication
</Card>