Delivery Guarantees: At-Most-Once, At-Least-Once, and Effectively-Once
Delivery guarantees are not broker features you toggle on — they are contracts that require coordinated design on the producer, broker, and consumer sides simultaneously. This article derives all three delivery modes from first principles, implements idempotency key deduplication, and shows why at-least-once transport plus idempotent consumer equals effectively-once business outcome.
Distributed Messaging Systems
Delivery Guarantees: At-Most-Once, At-Least-Once, and Effectively-Once
The payment service publishes a payment.charged event. The Kafka broker receives it, writes it to the leader's log, and then the broker crashes before replicating to followers. On broker recovery with acks=1, the message is gone — the consumer never sees it. The payment was charged, but the downstream fulfillment service has no record of it. The order is in limbo.
The inverse: the broker receives the event, replicates it, but crashes before sending the producer acknowledgement. The producer's send() call times out, triggers a retry, and the consumer processes payment.charged twice for the same transaction. The customer is charged twice.
Both scenarios are not edge cases. They are the inherent ambiguity of distributed systems — the two-generals problem applied to message delivery. Delivery guarantees are not broker settings you enable. They are contracts that require coordinated design across all three participants: producer, broker, and consumer.
Series positioning: This is Part 5 of Distributed Messaging Systems — the convergence point where the Kafka (Part 3) and RabbitMQ (Part 4) paths meet. The three delivery modes apply identically to both brokers; the mechanism differs. The prerequisite mental model is consistency trade-offs from Consistency Models and CAP Theorem. Follow-up: Retry Engineering: Exponential Backoff, Jitter, and Idempotency.
1. The Three Delivery Modes
1.1 At-Most-Once: Fire and Forget
At-most-once means a message is delivered zero or one time — never more. Duplicates are impossible. Data loss is possible.
enable.auto.commit=true (the Kafka default) is at-most-once, not at-least-once. The auto-commit interval fires on a timer — it commits the offset of the last fetched message, not the last processed message. A crash between auto-commit and processing completes = silent data loss. The naming is counterintuitive but the behavior is unambiguous.
Use at-most-once when: loss is acceptable and recoverability is not required — analytics sampling, metrics aggregation, telemetry where approximate counts are sufficient.
1.2 At-Least-Once: Commit After Processing
At-least-once means a message is delivered one or more times — loss is impossible, duplicates are possible.
The at-least-once crash window:
This is the exactly-once illusion most systems need to handle: at-least-once delivery with idempotent processing.
1.3 Effectively-Once: At-Least-Once + Idempotent Consumer
"Effectively-once" is not a delivery guarantee — it is a business outcome achieved by combining at-least-once transport with idempotent consumer logic. The message may arrive twice; the effect on the system is identical to arriving once.
Idempotency keys must be stored durably — in a database row or Redis with persistence enabled. An in-memory Set<string> of processed IDs resets on every crash or restart, making it useless for exactly the scenario it is meant to protect against: post-crash redelivery.
2. True Exactly-Once: Kafka Transactions
Kafka's transactional producer provides genuine exactly-once semantics within the Kafka ecosystem — a message is written to the topic and the consumer offset is committed atomically, with no duplicates possible even across producer retries.
When exactly-once is and is not justified:
| Scenario | Recommendation |
|---|---|
| Financial transactions (charges, refunds) | At-least-once + DB idempotency key (more portable than Kafka-specific transactions) |
| Read model projections (CQRS) | At-least-once + ON CONFLICT DO NOTHING |
| Kafka Streams aggregations | Exactly-once via processing.guarantee=exactly_once_v2 |
| Cross-broker pipelines (Kafka → RabbitMQ) | At-least-once + idempotent consumer — Kafka transactions do not span brokers |
| Analytics / metrics | At-most-once — approximate is acceptable, simplicity wins |
Kafka transactional exactly-once adds latency — the transaction coordinator must write the transaction markers and wait for quorum before the consumer can read committed messages (transactions use isolation.level=read_committed). For most business applications, at-least-once + application-level idempotency is simpler, more portable, and fast enough.
3. The Idempotency Key Contract
3.1 Who Generates It
The idempotency key must be generated by the event source (the system that first observed the business fact), not by the producer or consumer:
3.2 Idempotency Key TTL and Storage
Set Redis idempotency TTL longer than your broker's message retention period. If messages can be retained for 7 days, your idempotency TTL must be at least 7 days + maximum expected consumer lag. An expired idempotency key before the message expires means a late-replayed message will bypass the deduplication gate.
4. Delivery Mode Decision Framework
Summary
| Concept | Rule |
|---|---|
| Effectively-once outcome | At-least-once + idempotent consumer = effectively-once business outcome; true exactly-once requires transactional producers and is rarely justified. |
| Idempotency key storage | Idempotency keys must be stored durably (DB row or Redis) — in-memory deduplication resets on every crash. |
| Auto-commit is at-most-once | Auto-commit (enable.auto.commit=true) is at-most-once, not at-least-once — it commits offsets before processing is confirmed complete. |
What's Next
Part 6: Consumer Patterns — Groups, Lag, Backpressure, and Rebalancing moves from delivery guarantees to the consumer lifecycle: how partition assignment works under load, measuring and responding to consumer lag, and the cooperative rebalancing strategies that prevent the stop-the-world pauses that plagued Kafka consumer groups before 2.4.
This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.