Changelog

October 2026

1. RELIABILITYOCTOBER 26, 2026

   Delta Lake commits survive ambiguous S3 responses

   Artie no longer deletes data files referenced by a Delta commit that landed despite a failed response, and checkpoints are named after the version whose state they actually hold.

   Read more: Delta Lake commits survive ambiguous S3 responses

   When the write of a Delta commit to S3 failed ambiguously, for example a lost response or a retry that found Artie's own commit already in place, Artie treated the commit as failed and deleted the Parquet files it had uploaded. If the commit had in fact landed, readers then hit missing files. Separately, when two writes raced, a checkpoint could be named after the newer table version while holding the older file list, so readers starting from that checkpoint missed the newer commit's files.

   Artie now reads the commit back after a failed write. If the stored bytes are its own, the commit counts as landed; if the outcome cannot be confirmed, the uploaded files are kept rather than deleted. Checkpoints are now written with the version captured together with their state, so a checkpoint's name always matches its contents.

   This is for teams using Delta Lake destinations on S3.

   Why this matters

   • A landed commit never loses the files it references
   • Checkpoints always point readers at the right set of files
   • An orphaned file is the worst case, not missing data

2. PERFORMANCEOCTOBER 26, 2026

   Faster backfills for MySQL tables with composite primary keys

   Snapshot batches on MySQL tables with multi-column primary keys now range-scan the primary key instead of re-reading the index from the start, so large backfills no longer slow down as they go.

   Read more: Faster backfills for MySQL tables with composite primary keys

   For a MySQL table with a multi-column primary key, each backfill batch used a row comparison such as (id, connected_at) > (?, ?). MySQL does not range-scan the primary key for that form, so every batch read the index from the beginning. Each batch was slower than the last, so a large backfill lost most of its throughput over the hours it ran.

   Artie now spells out the batch bounds in a form MySQL does range-scan. On a table with tens of millions of rows, a 25,000-row batch that had been reading 22 million index entries in 58 seconds reads 25,000 in under a tenth of a second. The change also corrects the upper and lower bounds that SQL Server and Oracle composite-key scans share, which could include rows past a custom start or end key. Tables with a single-column primary key produce the same query as before.

   This is for teams backfilling large MySQL tables with composite primary keys, such as tables partitioned by date.

   Why this matters

   • Backfill time stays flat instead of growing with every batch
   • Custom start and end keys are honored exactly on MySQL, SQL Server, and Oracle
   • No configuration change needed

3. RELIABILITYOCTOBER 26, 2026

   Fewer duplicate MotherDuck rows after a partial append

   An append that partially committed before failing is no longer retried as a full stream, so rows that already landed are not sent again.

   Read more: Fewer duplicate MotherDuck rows after a partial append

   MotherDuck commits an append in chunks as the stream runs. When a stream failed partway, Artie retried the whole stream, up to ten times, so chunks that had already committed were sent again and could appear as duplicate rows.

   Artie now retries only appends into a temporary staging table, and clears that table before each retry so a failed attempt's committed rows are gone. Appends straight into a target table are not retried inside this path, and a value that cannot be converted returns its error immediately instead of being retried.

   This is for teams using MotherDuck destinations.

   Why this matters

   • Fewer duplicate rows after a transient failure mid-append
   • Retries start clean instead of stacking on previously written rows
   • Conversion problems surface right away

4. CONNECTORSOCTOBER 26, 2026

   Load into SQL Server tables with IDENTITY columns

   Artie now writes source identity values into SQL Server destination tables that define IDENTITY columns, keeps those columns out of updates, and survives concurrent table creation.

   Read more: Load into SQL Server tables with IDENTITY columns

   A load into a SQL Server destination table with an IDENTITY column failed with "Cannot insert explicit value for identity column ... when IDENTITY_INSERT is set to OFF", because Artie copies the source's values, identity values included. Separately, two writers creating the same destination table at once could deadlock.

   Artie now detects a table's identity columns and, when the source supplies values for them, inserts those values with IDENTITY_INSERT on inside a single transaction, while leaving identity columns out of every update. An identity column the source does not have, such as a surrogate key you added on the destination, is still filled by SQL Server. Setting IDENTITY_INSERT requires ALTER permission on those tables. Artie also retries when parallel writers race to create the same table. Append-only and history tables do not yet handle identity columns.

   This is for teams replicating into SQL Server tables that use IDENTITY columns.

   Why this matters

   • Replicate into tables you designed with identity primary keys
   • Keep your own surrogate identity columns working alongside replicated data
   • Parallel snapshot loads no longer trip over table creation

5. RELIABILITYOCTOBER 26, 2026

   Long ClickHouse loads survive a closed pooled connection

   When a pooled ClickHouse connection has been closed by the server or a load balancer, Artie now retries preparing the insert on a fresh connection instead of failing the batch and the backfill around it.

   Read more: Long ClickHouse loads survive a closed pooled connection

   ClickHouse inserts reuse pooled connections. If the server or a load balancer had closed one after an idle timeout, the next batch failed with "driver: bad connection" before any rows were sent, and nothing retried it. During a parallel backfill that cancelled every other shard too, so one closed connection could end a backfill that had been running for hours.

   Artie now retries preparing the insert on a new connection, up to six times, when it finds a closed one. Preparation sends no rows, so a retry cannot write anything twice. Commits are not retried, because a connection that breaks mid-commit may already have stored some rows.

   This is for teams loading into ClickHouse, especially with long backfills.

   Why this matters

   • Idle timeouts on the ClickHouse side no longer end a backfill
   • Retries happen only where they cannot duplicate rows
   • No change to how commit-time failures are reported

6. CONNECTORSOCTOBER 26, 2026

   Manage SMB shares for SQL Server log backups as connectors

   SMB shares that hold SQL Server transaction-log backups are now connectors you create once, test, and reuse across pipelines, instead of settings typed into each pipeline.

   Read more: Manage SMB shares for SQL Server log backups as connectors

   Artie already reads SQL Server transaction-log backups from SMB shares, including over PrivateLink. But the share's path and credentials were entered inside each pipeline's source settings, so there was no way to test the share on its own or reuse it across pipelines.

   SMB share is now a connector in its own right. Add it from the Connectors page with the share's UNC path, a username, and a password, optionally routed over a PrivateLink connection, and test the connection before saving. Then pick it as the archive location in any SQL Server transaction-log source. Pipelines that already had a share configured inline were moved to a connector automatically.

   This is for SQL Server teams using transaction-log backup CDC with backups stored on a file share.

   Why this matters

   • Configure a share once and reuse it across pipelines
   • Rotate the share's credentials in one place
   • Check connectivity before a pipeline depends on it

7. PERFORMANCEOCTOBER 26, 2026

   Merge-on-read for Iceberg tables in Amazon S3 Tables

   S3 Tables pipelines can now create and maintain Iceberg tables as merge-on-read, sorted by primary key, so each batch of changes writes far less data.

   Read more: Merge-on-read for Iceberg tables in Amazon S3 Tables

   Iceberg tables in S3 Tables were written copy-on-write, so every batch of changes rewrote the data files it touched, even when only a handful of rows in each file changed.

   Artie is enabling merge-on-read for S3 Tables pipelines one pipeline at a time. Once a pipeline is on it, new tables are created merge-on-read and sorted by their primary keys, and existing tables are switched the same way on their next merge. In a benchmark on a 10-million-row table with updates spread across it, a merge took 2.0 seconds instead of 5.2 and wrote roughly 200 times fewer bytes. Tables stay on Iceberg format v2, so engines such as Athena and StarRocks keep reading them. If a table must stay copy-on-write, set its write.delete.mode, write.update.mode, and write.merge.mode properties to copy-on-write yourself: Artie never changes a table whose write.merge.mode is already set, and setting the three properties after a switch puts the table back. Pipelines that have not been enabled are unchanged; contact Artie to enable yours.

   This is for teams using Amazon S3 Tables as an Iceberg destination.

   Why this matters

   • Merges finish faster and write far less to S3
   • No format upgrade, so your query engines keep working
   • Return any table to copy-on-write with its own table properties

8. CONNECTORSOCTOBER 26, 2026

   Publish change events to your own Kafka cluster

   Kafka is now an Artie destination. Artie publishes each table's changes straight to your Apache Kafka or Amazon MSK brokers, one topic per table, with SASL, AWS MSK IAM, TLS, and mutual TLS.

   Read more: Publish change events to your own Kafka cluster

   Teams that already run Kafka wanted Artie's change events in their own topics, under their cluster's authentication and topic rules, instead of only landing in a warehouse.

   Kafka is now a destination connector. Artie publishes each table's inserts, updates, and deletes directly to your brokers as Debezium-format JSON, one topic per table named <topic prefix>.<table identifier>. Authenticate with SCRAM-SHA-512, SCRAM-SHA-256, PLAIN, or AWS MSK IAM (with an optional external ID on the role). TLS is on by default, and you can supply a custom CA certificate or a client certificate and key for mutual TLS. If your cluster does not create topics automatically, turn on Create missing topics and Artie creates each table's topic before the first write (Kafka 2.4 or later). Artie connects to every advertised broker directly, so allow Artie's IP addresses; SSH tunnels and PrivateLink are not supported for Kafka.

   Kafka destinations are turned on per account. Contact Artie to enable them for yours.

   This is for teams that operate their own Kafka or Amazon MSK cluster and want CDC events in it.

   Why this matters

   • Feed your own stream consumers without a warehouse in between
   • Match authentication and TLS to how your cluster is already set up
   • Let Artie create topics, or keep topic management entirely on your side

   Kafka destination docs

9. CONNECTORSOCTOBER 26, 2026

   Replicate collections from several MongoDB databases in one pipeline

   A new cluster-wide change stream option lets a MongoDB pipeline include collections from any database in the deployment, with each database optionally mapped to its own destination schema.

   Read more: Replicate collections from several MongoDB databases in one pipeline

   A MongoDB pipeline opened its change stream at the database level, so collections that lived in other databases of the same deployment each needed a pipeline of their own.

   Turn on Use one change stream for the whole cluster in the source's advanced settings. Artie then opens a single deployment-wide change stream and lists collections from every database, so one pipeline can replicate collections from across the deployment. Destination tables are named <database>_<collection>. If you would rather keep databases apart, choose Multiple schemas in the destination's schema settings and each collection's database becomes its destination schema; that choice is offered only when every collection in the pipeline was added after the cluster-wide option was turned on. The option needs MongoDB 4.0 or later and a user with the readAnyDatabase role, can only be changed while the pipeline is in draft or paused, and turns off per-collection change streams. It is not available for DocumentDB. In Terraform, set mongodb_deployment_scoped_mode on the source reader resource (provider v2.3.53 or later).

   This is for MongoDB teams whose collections span more than one database in a deployment.

   Why this matters

   • One pipeline instead of one per database
   • Keep databases separate downstream by mapping each to its own schema
   • Manage the setting in the Dashboard or in Terraform

10. CONNECTORSOCTOBER 26, 2026

    S3 CSV ingestion handles files with a byte order mark

    A leading UTF-8 byte order mark no longer corrupts the first header or value of a CSV file ingested from S3, so primary keys match on files exported by spreadsheet tools.

    Read more: S3 CSV ingestion handles files with a byte order mark

    Excel and other tools write a UTF-8 byte order mark at the start of "CSV UTF-8" exports. Artie read it as part of the first header, so a column named id arrived with an invisible leading character. Mixing those files with plain ones split one column into two, and a primary key configured as id failed on every row of the exported file.

    Artie now drops a leading byte order mark before parsing a CSV file from an S3 source. Files with a header row and header-less files that use configured static headers are both covered, and a byte order mark that appears inside a field value is kept as data.

    This is for teams ingesting CSV files from S3, especially exports from spreadsheet tools.

    Why this matters

    • Spreadsheet exports load without a manual cleanup step
    • Primary keys match the parsed header as configured
    • Mixed exports no longer fan out into extra columns