Can you deliver a message exactly once?
“A product manager asks for "exactly-once" processing of payment events. Is that possible? What do you actually build?”
What this tests
- Understanding that at-least-once plus idempotency is the honest form of exactly-once
- Where duplicates come from: producer retries, consumer crashes after processing but before ack, rebalances
- Mechanisms: idempotency keys, processed-message table, transactional outbox, upserts
- Ability to explain the guarantee to a non-engineer without lying
Answers by level
Read the beginner answer first and notice what is missing.
Exactly-once delivery across a network is not achievable in general: the consumer can process a message and crash before acknowledging, and the broker has no way to know whether the effect happened. What is achievable is exactly-once *effect*: deliver at least once and make processing idempotent, so duplicates are harmless. Duplicates arise from producer retries after a timeout, consumer redelivery after a missed ack, and consumer-group rebalances that reassign in-flight partitions.
The build: each message carries a unique id (the payment intent id or a client-generated key). The consumer, in one local transaction, checks a processed-messages table for that id, applies the effect, and inserts the id. If the transaction commits, the message is acked; if the consumer crashes before the ack, the redelivery hits the table and is skipped. Where the effect is an upsert keyed by the id, the table is unnecessary because the write is naturally idempotent.
Brokers' "exactly-once" features cover their own boundary — a consumer that reads, transforms and writes back to the same broker within a transaction. The moment the effect is an external side effect, such as charging a card, idempotency in the consumer is the only mechanism.
Green flags · Red flags
- States that exactly-once delivery is impossible and exactly-once effect is the goal
- Names the duplicate sources: retries, missed acks, rebalances
- Describes the processed-message table in the same transaction as the effect
- Passes the idempotency key through to external providers
- Scopes broker exactly-once features correctly
- Uses the outbox on the producer side
- "Just turn on exactly-once in Kafka."
- Acks before processing to avoid duplicates (now loses messages)
- Deduplicates in memory across a fleet of consumers
- Does not mention the external provider's idempotency key