From a DBA’s perspective this is a silent killer, and PostgreSQL 17 finally gave us a clean way to disarm it. Let me show you exactly why it breaks, what the old workarounds cost, and how native failover slots<\/strong> fix it, with a small Patroni lab you can run on Docker Desktop to see both the failure and the cure.<\/p>\n\n\n\n If you only run Debezium:<\/strong> your half is one line, A logical replication slot is a bookmark. It records how far a consumer has read the write-ahead log (the WAL is PostgreSQL’s running log of every change; an LSN is just a position in it, like a byte offset), and it keeps PostgreSQL from recycling WAL the consumer still needs. Debezium, a native The issue is that a logical slot lives on only one instance, the one where it was created. Physical streaming replication, the mechanism Patroni uses to keep a standby in sync, copies the data blocks and the WAL. It does not copy the logical slots. For years that was a deliberate design decision, not a bug: a slot carries decoding state and a So the picture before failover looks healthy:<\/p>\n\n\n\n <\/p>\n\n\n\n The slot exists only on the primary. Promote the standby, and the new primary has all your data and none of your slots. Debezium reconnects through your load balancer, asks for This is the first question I get, and it is a good one. The instinct is reasonable : I already have HAProxy in front of the cluster, or a VIP that floats with the leader on keepalived, or a cloud load balancer, so when the primary moves my consumer simply reconnects to the new one. Doesn’t that cover it?<\/p>\n\n\n\n It does not, and understanding why tells you what class of problem this really is.<\/p>\n\n\n\n A load balancer solves routing<\/strong>: how the consumer finds whoever is currently the primary. A floating VIP does the same thing one layer down. Both get the client to the right door, and you do need one of them (in the lab that is exactly HAProxy’s job). But the slot is not a routing concern, it is state<\/strong>. It is a file in the primary’s data directory. When the consumer reconnects through the VIP to the new primary and asks for There is exactly one architecture where the slot survives without any slot synchronization at all, and it is a different design rather than a different load balancer: shared-storage failover<\/strong>. If your HA is built on a block volume that detaches from the failed node and reattaches to the new one (DRBD, a SAN, a cloud disk, a Kubernetes PVC), then failover moves the whole data directory, The next idea people reach for is to pre-create the slot on every node, primary and standbys, and let the consumer use whichever one is currently primary. The intuition is right. In fact it is almost exactly what failover slots do, it just misses the one piece that makes it safe.<\/p>\n\n\n\n A slot is not only a name. It carries moving state: That is worse than losing the slot, not better. After failover the consumer finds a slot called Two more walls. Before PostgreSQL 16 you cannot even create a logical slot on a standby (you get “logical decoding cannot be used while in recovery”); version 16 added logical decoding on standbys<\/a>, and 17 added the synchronization that makes a standby slot actually track the primary. And idle slots are not free: every slot, active or not, pins WAL and holds back vacuum on its node, so leaving un-advanced slots on every node invites WAL growth.<\/p>\n\n\n\n So the correct version of “a slot on every node, one active and the rest idle” is precisely a failover slot: present on the standby, not consumable until promotion, with its position kept in lockstep with the primary. The synchronization is the whole feature.<\/p>\n\n\n\n This problem is not new, and the community built two workarounds.<\/p>\n\n\n\n The first was the The second, if you run Patroni, is built in. Patroni has its own concept of permanent replication slots<\/a><\/strong>. You declare a slot in the cluster configurationand Patroni makes sure it exists on every eligible node, copying the slot to thestandbys and advancing its position on a schedule. In Patroni terms it looks like this:<\/p>\n\n\n This genuinely survives a switchover, and for a lot of clusters it has been good enough for years. But notice the words “on a schedule”. Patroni advances the copy of the slot on the standby every That is the gap PostgreSQL 17 closes.<\/p>\n\n\n\n PostgreSQL 17 introduced native failover slots<\/strong><\/a>: logical slots that the database itself keeps synchronized to a physical standby as part of replication itself, with a guarantee that the consumer cannot read past what the standby has (the There are three moving parts.<\/p>\n\n\n\n 1. Mark the slot as a failover slot.<\/strong> If you drive logical replication with a subscription, the same flag exists there, 2. Let the standby synchronize slots.<\/strong> On the standby you enable the slot sync worker (both are replication settings<\/a>):<\/p>\n\n\n 3. Hold the logical slot back until the standby has the data.<\/strong> On the primary you set This is the subtle, important one, a leash on the consumer. It tells the primary’s logical WAL senders never to hand a change to the CDC consumer until the physical standby has also received it, so the consumer can never get ahead of the standby. Without it, after failover the new primary cannot give the consumer what it already saw; with it, promotion is always safe.<\/p>\n\n\n\n With those in place, the slot is mirrored in real time. On the standby, There is a prerequisite that is easy to miss and fails in the quietest possible way: the standby’s A physical replication connection does not need a database name, so historically nobody put one there. But the slot sync worker connects to the primary as a normal backend to read slot state, and it needs a database to connect to. If Recent Patroni adds It must contain a dbname=\u2026 ; if not, slot sync is silently doing nothing<\/em><\/p>\n\n\n\n Let me walk both cases on a small two-node Patroni cluster: PostgreSQL 18 nodes The commands below are the real thing, run them on your own cluster if you like. If you would rather watch it happen in a few minutes, there is a ready-to-run Docker version at the end of this post.<\/p>\n\n\n\n A normal logical slot has It lives on the primary and nowhere else, because nothing mirrors it:<\/p>\n\n\n On the standby, The new primary never had the slot. The consumer reconnects through HAProxy and, here is the trap, stays perfectly green:<\/p>\n\n\n because Debezium found its slot missing and silently recreated it at the current<\/em> LSN, straight from the Connect log:<\/p>\n\n\n Everything committed between the switchover and that recreation is skipped. Draining the topic and comparing it to the source table shows the hole:<\/p>\n\n\n Five committed rows, gone, and nothing anywhere raised an error. Whether Debezium recreates the slot quietly (as here) or fails loudly is a matter of configuration; the lab shows the quiet case because it is the one that hurts. That is the silent version that costs you a reseed three days later when someone notices the warehouse is short.<\/p>\n\n\n\n Same scenario, two changes. First, create the slot with Second, turn on synchronization. On the standby:<\/p>\n\n\n and on the primary, hold logical slots back until the standby has the data, naming the standby’s physical slot:<\/p>\n\n\n Now the slot is mirrored to the standby. Look at it there:<\/p>\n\n\n Only once On the new primary the slot is already there and live, and the consumer resumes on its own. Re-point Every committed row reached Kafka. Same cluster, same switchover, same consumer; the only difference is one boolean on the slot and the synchronization around it. The duplicates are at-least-once redelivery (you may see a handful or none on a given run), so the consumer must be idempotent either way, but not one committed row was lost. That boolean is the gap between “we lost four hours of CDC” and “nobody noticed”.<\/p>\n\n\n\n I want to be honest about the rough edges, because this is where the real-world write-ups tend to go quiet.<\/p>\n\n\n\n PostgreSQL 17 gives you the mechanism. Patroni, as of the 4.1 series I tested, does not<\/strong> yet fully wire it for you, and there are two manual pieces.<\/p>\n\n\n\n First, Patroni does not manage Second, the Neither of these is a reason to avoid failover slots. They are a reason to treat “failover slots on Patroni” as a thing you configure and test deliberately, not a checkbox you assume.<\/p>\n\n\n\n If you run logical replication or CDC on a Patroni cluster, here is what I would verify :<\/p>\n\n\n\n The lab runs on PostgreSQL 18, which keeps the PG17 mechanism intact and adds a few conveniences around it: PostgreSQL 19 is in development as I write this, so treat this as a direction, not a promise (check the final release notes). The theme is telling: most of the 19 cycle’s work here makes the failure I hit easier to see<\/em> rather than changing the mechanism, slot-sync skip counters in \n
slot.failover=true<\/code> on the connector (Debezium \u2265 3.0.5 against a PostgreSQL 17+ primary). The other half is server-side, two settings and the failover wiring, and it is your DBA’s job; the rest of this post is what to ask them for.<\/p>\n<\/blockquote>\n\n\n\nWhy the slot does not survive<\/h2>\n\n\n\n
CREATE SUBSCRIPTION<\/code>, pg_recvlogical<\/code> all rely on a named slot to guarantee they never miss a change and never have to replay from the beginning.<\/p>\n\n\n\ncatalog_xmin<\/code> (a do-not-garbage-collect marker for the old system-catalog rows it still needs to decode the WAL), and replaying that safely onto a standby is genuinely hard.<\/p>\n\n\n\n![\"\"](\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2026\/05\/image-82.png\")
<\/figure>\n\n\n\ncdc_slot<\/code>, and PostgreSQL answers, correctly, that no such slot exists. The pipeline is broken, and depending on how the connector is configured it either fails loudly or, worse, quietly recreates a fresh slot at the current position and skips everything in between.<\/p>\n\n\n\nCan’t my load balancer or VIP just handle this?<\/h2>\n\n\n\n
cdc_slot<\/code>, that instance still has to physically own the slot. The IP moving does not move the slot, because the new primary is a different PostgreSQL instance with its own pg_replslot<\/code> directory. So whether HAProxy sits on the database nodes or on its own tier, and whether you use HAProxy or a VIP or a cloud LB, the effect on the slot is identical: none.<\/p>\n\n\n\npg_replslot<\/code> included, and the slot comes along because it never logically moved. That works, but it buys a different set of trade-offs (the storage becomes a hard dependency and a fencing problem, and you give up streaming read replicas off the same volume), and it is not what a streaming-replication cluster like Patroni does. On streaming replication the slot has to be synchronized deliberately, which is the rest of this post.<\/p>\n\n\n\nWhy not just create the slot on every node?<\/h2>\n\n\n\n
restart_lsn<\/code>, confirmed_flush_lsn<\/code> and catalog_xmin<\/code>. If you create independent slots on each node, each one starts at that node’s position and then sits there, because nothing advances the standby’s copy as the consumer acknowledges changes on the primary. The names match, the positions drift.<\/p>\n\n\n\ncdc_slot<\/code> (so no loud error) and resumes from its stale position. If the copy is behind the consumer, you re-deliver changes it already processed, which is duplicates. If it is ahead, the consumer asks for an LSN the slot has already discarded, which is a gap or an error. You have turned a loud, obvious failure(“replication slot does not exist”, which at least pages someone) into a silent data-integrity problem, and for a pipeline feeding a data lake, silent is the worst outcome.<\/p>\n\n\n\nThe years before PostgreSQL 17<\/h2>\n\n\n\n
pg_failover_slots<\/code><\/a><\/strong> extension (from EDB), which runs on theprimary and copies logical slot positions out to physical standbys. It supportsPostgreSQL 11 and up, but from 17 on you would reach for the native feature instead,which is what EDB now recommends too. Either way it is an external extension you haveto install, load viashared_preload_libraries<\/code>, and keep compatible with your major version.<\/p>\n\n\n\n\nslots:\n cdc_slot:\n type: logical\n database: cdcdemo\n plugin: pgoutput\n<\/pre><\/div>\n\n\n
loop_wait<\/code> seconds, by default every 10 seconds. That periodic copy is the weak point. The standby’s idea of “how far the consumer has read” can lag the real primary by up to a loop interval, so a failover at the wrong moment can leave the new primary slightly behind what the consumer already acknowledged, which is exactly the window where you get duplicates or, with the wrong settings, a gap. It is a copy advanced on a timer, not a hard guarantee, so itis stale between refreshes and nothing stops the consumer from running ahead of it.<\/p>\n\n\n\nPostgreSQL 17 failover slots<\/h2>\n\n\n\n
synchronized_standby_slots<\/code> part below), rather than an external copy on a timer. (The feature landed in 17, so everything here applies to PostgreSQL 17 and later. I built the lab on PostgreSQL18, which adds a couple of conveniences I will mention at the end.)<\/p>\n\n\n\npg_create_logical_replication_slot<\/code><\/a> grew a fifth argument,failover<\/code>:<\/p>\n\n\n\nSELECT * FROM pg_create_logical_replication_slot('cdc_slot', 'pgoutput',\n false, false, true);\n-- temporary --^ ^ ^\n-- twophase ------+ |\n-- failover -------------+\n<\/pre><\/div>\n\n\nCREATE SUBSCRIPTION ... WITH (failover = true)<\/code><\/a>, and the slot and the subscription must agree. If you use Debezium 3.x, you set slot.failover = true<\/code> and the connector creates the slot with the flag for you, as long as it is talking to a PostgreSQL 17+ primary.<\/p>\n\n\n\n\nsync_replication_slots = on\nhot_standby_feedback = on\n<\/pre><\/div>\n\n\n
hot_standby_feedback<\/code> is not optional here. The standby has to tell the primary which catalog rows it still needs so the primary does not vacuum away rows the synchronized slot depends on.<\/p>\n\n\n\nsynchronized_standby_slots<\/code><\/a>:<\/p>\n\n\n\nsynchronized_standby_slots = 'physical_slot_of_the_standby'\n<\/pre><\/div>\n\n\n
pg_replication_slots<\/code><\/a> shows it as synced, and it cannot be consumed until the standby is promoted:<\/p>\n\n\n\n slot_name | slot_type | active | failover | synced | confirmed_flush_lsn\n-----------+-----------+--------+----------+--------+---------------------\n cdc_slot | logical | f | t | t | 0\/3A1240\n<\/pre><\/div>\n\n\n
failover = t<\/code>, synced = t<\/code>, active = f<\/code>. That row is the whole point. When this standby becomes primary, cdc_slot<\/code> is already there, at the right LSN, and the consumer reconnects as if nothing happened.<\/p>\n\n\n\nThe gotcha that bites everyone, silently<\/h3>\n\n\n\n
primary_conninfo<\/code> must contain a dbname<\/code> (the logical replication failover docs<\/a> call this out, but it is easy to skip past).<\/p>\n\n\n\ndbname<\/code> is missing, the worker does not error in your face. It just does not sync anything. Your slots silently fail to appear on the standby, and you find out at failover, which is the worst possible time.<\/p>\n\n\n\ndbname<\/code> to primary_conninfo<\/code> for you when it is managing logical slots, but I check it explicitly, every time, and so should you, on the standby:<\/p>\n\n\n\nSHOW primary_conninfo; \n<\/pre><\/div>\n\n\n
Seeing it break, then survive<\/h2>\n\n\n\n
patroni1<\/code> and patroni2<\/code>, a Debezium connector reading through HAProxy (so it always follows the current primary), and a tiny generator inserting rows with a strictly increasing counter n<\/code>. The proof is simple: compare the n<\/code> values that reached Kafka against the rows actually in the table. Every committed row should be in Kafka; duplicates are fine, because CDC delivery is at-least-once.<\/p>\n\n\n\nFirst, the break: a plain slot<\/h3>\n\n\n\n
failover<\/code> off, the default. Debezium creates one for you when you register a connector with slot.failover=false<\/code>; the bare SQL equivalent is:<\/p>\n\n\n\nSELECT pg_create_logical_replication_slot('dbz_slot', 'pgoutput');\n<\/pre><\/div>\n\n\n\n-- on the primary (patroni1)\nSELECT slot_name, slot_type, active, failover, synced FROM pg_replication_slots;\n\n slot_name | slot_type | active | failover | synced\n-----------+-----------+--------+----------+--------\n dbz_slot | logical | t | f | f\n patroni2 | physical | t | f | f (the standby's streaming slot)\n<\/pre><\/div>\n\n\n
dbz_slot<\/code> is simply not there. Now fail over with patronictl<\/code>:<\/p>\n\n\n\npatronictl -c \/etc\/patroni\/patroni.yml switchover --leader patroni1 --candidate patroni2 --force cdc-demo\n\nSuccessfully switched over to "patroni2"\n<\/pre><\/div>\n\n\n
\ncurl -s localhost:8083\/connectors\/orders-plain\/status | jq '{state: .connector.state, task: .tasks[0].state}'\n\n{ "state": "RUNNING", "task": "RUNNING" }\n<\/pre><\/div>\n\n\n\nCreating replication slot with command CREATE_REPLICATION_SLOT "dbz_slot" LOGICAL pgoutput\n<\/pre><\/div>\n\n\n
\ntable rows: 255 distinct rows in Kafka: 250 missing: 5 (n = 100..104)\n<\/pre><\/div>\n\n\n
Then, the cure: a failover slot<\/h3>\n\n\n\n
failover<\/code> on (or set Debezium’s slot.failover=true<\/code>, which does it for you):<\/p>\n\n\n\nSELECT pg_create_logical_replication_slot('cdc_slot', 'pgoutput', false, false, true);\n<\/pre><\/div>\n\n\n\nsync_replication_slots = on\nhot_standby_feedback = on\n<\/pre><\/div>\n\n\n
\nALTER SYSTEM SET synchronized_standby_slots = 'patroni2';\nSELECT pg_reload_conf();\n<\/pre><\/div>\n\n\n
\n-- on the standby (patroni2)\nSELECT slot_name, active, failover, synced, temporary, confirmed_flush_lsn\n FROM pg_replication_slots WHERE slot_name = 'cdc_slot';\n\n slot_name | active | failover | synced | temporary | confirmed_flush_lsn\n-----------+--------+----------+--------+-----------+---------------------\n cdc_slot | t | t | t | f | 0\/34B0A10\n<\/pre><\/div>\n\n\n
failover = t<\/code>, synced = t<\/code>. One gotcha worth watching: a freshly synced slot starts temporary = t<\/code> and is dropped on promotion until it persists. The standby log shows the transition, a transient refusal (its catalog_xmin<\/code> is briefly ahead of the primary slot’s) and then sync-ready:<\/p>\n\n\n\nLOG: could not synchronize replication slot "cdc_slot"\nDETAIL: ... the remote slot needs catalog xmin 760, but the standby has xmin 761.\nLOG: newly created replication slot "cdc_slot" is sync-ready now\n<\/pre><\/div>\n\n\n
temporary = f<\/code> is it safe to fail over. So do it:<\/p>\n\n\n\npatronictl -c \/etc\/patroni\/patroni.yml switchover --leader patroni1 --candidate patroni2 --force cdc-demo\n<\/pre><\/div>\n\n\n
synchronized_standby_slots<\/code> at the new standby (the correct value flips after a role change), then reconcile:<\/p>\n\n\n\n-- on the new primary\nALTER SYSTEM SET synchronized_standby_slots = 'patroni1';\nSELECT pg_reload_conf();\n\ntable rows: 616 distinct rows in Kafka: 616 missing: 0 (74 duplicates)\n<\/pre><\/div>\n\n\n
Patroni does its part, but not all of it<\/h2>\n\n\n\n
synchronized_standby_slots<\/code> (this is tracked as Patroni issue #3431). You have to set it yourself, on the primary, to the physical slot name of the current standby. And in a two-node cluster that value is not symmetric: when patroni1<\/code> leads it must name patroni2<\/code>‘s slot, and after a switchover to patroni2<\/code> it must name patroni1<\/code>‘s slot. So you cannot just bake a single static value into the cluster config, you have to reset it after a role change. The lab’s enable-failover-config.sh<\/code> is role-aware and does this, and the walkthrough deliberately runs it again after the switchover so you can see the value flip. In production you would automate it with a Patroni callback on role change.<\/p>\n\n\n\ndbname<\/code> in primary_conninfo<\/code> requirement I mentioned. Recent Patroni handles it, but it is exactly the kind of thing that is true on the version you tested and silently false on the version you deployed, so verify it rather than trust it.<\/p>\n\n\n\nThe checklist<\/h2>\n\n\n\n
Check<\/th> Where<\/th> Why<\/th><\/tr><\/thead> wal_level = logical<\/code><\/td>all nodes<\/td> required for logical decoding at all<\/td><\/tr> slot created with failover = true<\/code><\/td>primary<\/td> the slot is eligible to be synchronized<\/td><\/tr> sync_replication_slots = on<\/code><\/td>standbys<\/td> the standby actually runs the sync worker<\/td><\/tr> hot_standby_feedback = on<\/code><\/td>standbys<\/td> protects the catalog rows the slot needs<\/td><\/tr> synchronized_standby_slots<\/code> names the standby slot<\/td>primary<\/td> consumer never gets ahead of the standby<\/td><\/tr> dbname=<\/code> present in primary_conninfo<\/code><\/td>standbys<\/td> without it, sync fails silently<\/td><\/tr> synced = t<\/code> and<\/strong> temporary = f<\/code> on the standby<\/td>standbys<\/td> mirrored and<\/strong> persisted: a still- temporary<\/code> synced slot is dropped on promotion (“sync-ready now”)<\/td><\/tr>re-set synchronized_standby_slots<\/code> after switchover<\/td>primary<\/td> the correct value flips in a 2-node cluster<\/td><\/tr> your CDC tool sets the slot failover flag<\/td> consumer<\/td> e.g. Debezium slot.failover = true<\/code>, needs \u2265 3.0.5 and PG 17+<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\nWhere this is going: PG18 today, PG19 next<\/h2>\n\n\n\n
pg_recvlogical --enable-failover<\/code><\/a> lets you create a failover slot straight from the command-line consumer, and --enable-two-phase<\/code> (plus pg_createsubscriber --enable-two-phase<\/code>) rounds out the tooling. The configuration is identical to 17, so everything above applies unchanged.<\/p>\n\n\n\npg_stat_replication_slots<\/code>, a pg_sync_replication_slots()<\/code>that no longer fails silently, starting logical replication without a wal_level<\/code> restart (a new effective_wal_level<\/code>), and logical replication of sequences.<\/p>\n\n\n\n