15. Appendices A & B
15.5 B.2 Cluster deployment and management
"When running Kafka outside of a managed platform, you will need several things to assist you with running the cluster properly. This includes help with provisioning and deployment, balancing data, and visualizing your clusters."
| Tool | What it does |
|---|---|
Strimzi — strimzi.io | "provides KUBERNETES OPERATORS for deploying Kafka clusters... It does NOT provide managed services but instead makes it easy for you to get up and running in a cloud, whether public or private. It also provides the Strimzi Kafka Bridge, a REST proxy implementation supported under the Apache 2.0 license. ⚠️ At this time, Strimzi does NOT have support for a schema registry, DUE TO CONCERNS ABOUT LICENSES." |
AKHQ — akhq.io | "a GUI for managing and interacting with Kafka clusters. It supports configuration management, INCLUDING USERS AND ACLs, and provides some support for components like the Schema Registry and Kafka Connect. It also provides tools for working with data in the cluster AS AN ALTERNATIVE TO THE CONSOLE TOOLS." |
| JulieOps (formerly Kafka Topology Builder) | "automated management of topics and ACLs using a GITOPS MODEL. More than viewing the state of the current configuration, JulieOps provides a means for DECLARATIVE CONFIGURATION AND CHANGE CONTROL of topics, schemas, ACLs, and more OVER TIME." |
| Cruise Control — LinkedIn | ⭐ *"LinkedIn's answer to how to manage hundreds of clusters with thousands of brokers. This tool began as a solution to automated rebalancing of data but has evolved to include ANOMALY DETECTION and ADMINISTRATIVE OPERATIONS, such as adding and removing brokers. FOR ANYTHING MORE THAN A TESTING CLUSTER, IT IS A MUST-HAVE FOR ANY KAFKA OPERATOR." |
Conduktor — conduktor.io | "not open source, [but] a popular DESKTOP tool... supports many of the managed platforms (Confluent, Aiven, MSK) and many different components (Connect, kSQL, Streams). Also allows you to interact with data in the clusters, as opposed to using the console tools. A FREE LICENSE is provided for development use that works with a SINGLE CLUSTER." |
Cruise Control is the one tool this book calls a “must-have.” It appears in five chapters:
| Chapter | What it is there for |
|---|---|
| Ch. 2 | maintaining rack awareness after reassignments |
| Ch. 10 | broker “demotion” before decommissioning |
| Ch. 12 | keeping leadership balanced; broker add/remove |
| Ch. 13 | “highly recommended… for keeping the cluster balanced at all times”; hot-partition detection Kafka can't do itself |
| App. B | “a must-have for any Kafka operator” |
► Kafka deliberately does not auto-rebalance data. This is the gap.