2. Installing Kafka · Learn Labs

Source: Kafka: The Definitive Guide, 2nd Ed., Ch. 2 Learning goal: this is the operations chapter disguised as an install guide. The install is 3 commands; the value is in why each config exists, what it trades away, and which defaults will hurt you in production.

Sections

2.01
The shape of a Kafka deploymentEverything else is tuning.
2.1
What problem does this chapter solve?The default server.properties shipped with Kafka is "sufficient to run a standalone server as a proof of concept, but most likely will not be sufficient for large installations."
2.22
Environment setupKafka is a Java application and runs on Windows, macOS, Linux, and others.
2.3
Installing a Kafka broker
2.49
Configuring the broker — every knob, and its tradeoffInteger identifier, default 0, must be unique per broker within a cluster.
2.55
Selecting hardwareOther factors: specific drive technology (SAS vs SATA) and the quality of the drive controller.
2.64
Configuring Kafka clustersCluster size is bound by four things:
2.71
OS tuningThis is a great example of a config whose semantics changed underneath a widely-copied best practice.
2.84
Production concernsNote -Xmx6g -Xms6g — min and max heap set equal, the standard practice to avoid heap resizing pauses.
2.9Failure catalog
What actually breaks in production — Ch. 2 consolidated
2.101
Deploy / monitor / scale / back up — Chapter 2's contributionChapter 2 has no backup section, and that absence is informative.
2.11Self-test
Self-test