Skip to main content

Documentation Index

Fetch the complete documentation index at: https://artie.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

​
Required settings

  • Host name
  • Port (default is 5432)
  • Service account
  • Database with logical replication enabled

​
Provider specific instructions

Amazon Aurora or RDS

Google CloudSQL

Supabase

​
Additional features

​
How does Artie support partitioned tables?

Artie will automatically detect partitioned tables and automatically fan data into a single table in your destination.

​
Can I use logical replication on standby replicas?

Yes, you can use logical replication on standby replicas with some caveats:
  1. This is only available in PostgreSQL 16 and above.
  2. This is not available for Amazon Aurora
Steps to enable this:
1
Create dbz_publication publication on the primary database
CREATE PUBLICATION dbz_publication FOR ALL TABLES;
2
Enable hot_standby_feedback on the standby server
postgres> SHOW hot_standby_feedback;
 hot_standby_feedback
----------------------
 on
3
WAL_LEVEL is set to logical on the standby server
postgres> SHOW WAL_LEVEL;
 wal_level
----------------------
 logical

​
Enabling heartbeats

You’ll want to enable heartbeats to prevent your replication slot from overflowing if you have an idle database. To get started, select “Enable heartbeats” under the advanced settings on the source tab in the pipeline editor. Then run this command:
heartbeats.sql
CREATE TABLE test_heartbeat_table (id text PRIMARY KEY, ts timestamp);
-- Grant access to the heartbeat table
GRANT SELECT, UPDATE ON TABLE test_heartbeat_table TO artie;
-- Then insert one row into this table.
-- Artie's periodic pings will be this:
-- UPDATE test_heartbeat_table set ts = now() WHERE id = '1';
-- Such that we never end up adding additional rows.
INSERT INTO test_heartbeat_table (id, ts) VALUES (1, NOW());
Don’t want to enable publications to capture all the tables? You can change this behavior by selecting Filtered under the Pipeline advanced settings.If you change this to be filtered, you are responsible for adding and removing any captured tables from the publication. Our validation will check that the publications is configured correctly.
TOAST (The Oversized-Attribute Storage Technique) is a mechanism used by Postgres to store large column values to alleviate the page size limit. TOAST columns may be different data types such as text, jsonb, bytea, array, etc.One thing that is special about TOAST columns is that if the column value has not changed, the column value will not be included in WAL if the table’s replica identity is set to default. Artie has automatic detection built-in to detect an unchanged TOAST column and will conditionally replicate this into the target database.Our built-in process will detect if an incoming CDC event has an unchanged TOAST column and will not update the column value in the target database. We are able to do all of this without having to increase your table replica identity.
We have regular monitors that provide additional guardrails around your database replication, and will do the following:
  • Regularly check and monitor your replication slot size in 15-minute intervals and notify if the slot exceeds a certain threshold.
  • Heartbeats verification. For folks that are leveraging heartbeats, we will alert you if a heartbeat query fails.
  • Terminate any idle queries that are lingering for more than a day. You can avoid having this problem by setting idle_in_transaction_session_timeout.

​
Questions

We recommend enabling publications for all tables. This will ensure that you are capturing all the changes to your database and you will not need to manually add and remove tables from the publication.In the case where you only want to capture a small subset of tables in your database, you may want to enable publications for the specified tables. This will require you to manually manage the publication.
We can connect to the read-only endpoint for backfills. However, we will need to connect to the write endpoint to consume from change data capture.