Why Exactly-Once Delivery Is a Myth

Reference · Interrupted execution

In interviews and in design meetings, someone always asks for "exactly-once delivery." It sounds like the safe choice. It is also impossible, and knowing why is a good way to show that you understand distributed systems.

The problem is the acknowledgement

When A sends a message to B, B does the work and sends back an "ok". But this "ok" can be lost on the way, exactly like any other message.

Now A is stuck. It did not receive the "ok". It cannot tell the difference between two cases:

• B never received the message.
• B received it, did the work, but the "ok" was lost.

A has only two choices, and both can be wrong:

• send again (risk: B does the work two times).
• do not send again (risk: B never did the work).

There is no third option. The network cannot promise that a message arrives exactly one time, because it cannot even tell you if it arrived at all.

The three guarantees

So real systems pick one of these:

• At-most-once. Send, never retry. Fast, but you can lose messages.
• At-least-once. Retry until you get an "ok". You never lose a message, but you can get duplicates.
• Exactly-once. The one everybody wants. It does not exist at the delivery level.

Almost every queue and webhook system you will use is at-least-once. Duplicates are not a bug; they are the price of never losing data.

The trick: move the problem

You cannot get exactly-once delivery. But you can get exactly-once effect.

The idea is simple: accept that messages arrive one or more times, and make processing them many times the same as processing them one time. In other words, make the consumer idempotent.

You do this with an idempotency key: the message carries an id, the consumer remembers the ids it already handled, and it ignores repeats.

Delivery is at-least-once. Processing is idempotent. The result, seen from outside, looks exactly-once. That is the honest version of the promise, and it is the one that actually works.

This is the same reason that reconciliation between two systems is built on idempotent writes: it repeats the same writes again and again, and that has to be safe.