8. Exactly-Once Semantics

Source: Kafka: The Definitive Guide, 2nd Ed., Ch. 8 The chapter's own summary, which is the best one-liner in the book: "Exactly-once semantics in Kafka is the opposite of chess: it is challenging to understand but easy to use."

Two mechanisms, two different problems:

Idempotent producer	Transactions
“help avoid duplicates caused by PRODUCER RETRIES”	“guarantee exactly-once processing in STREAM PROCESSING applications”
Simpler, more generally useful.	Built for consume-process-produce.

Sections

8.11
Why at-least-once isn't always enoughCh. 7 delivered at-least-once — "the guarantee that Kafka will not lose messages that it acknowledged as committed.
8.27
The idempotent producerThe setup: producer writes to topic A partition 0; leader on broker 5, follower on broker 3.
8.35
TransactionsTransactions provide both.
8.42
3.6 ⚠️ What transactions do NOT solve
8.52
3.7 Transactional IDs and fencing
8.42
How to use transactions(This is the practical payoff of KIP-447's group-metadata fencing.)
8.52
How transactions work internallyThree new producers per second is nothing — that's a modest serverless workload or a badly-written client that constructs a producer per request.
8.63
Performance of transactionsNote that the producer's natural unit is "one transaction per poll() batch," which conveniently ties transaction size to max.poll.records — giving you one knob that moves both.
8.7Failure catalog
What actually breaks in production — Ch. 8 consolidated
8.82
Deploy / monitor / configure
8.9Self-test
Self-test