12.11 Operational quick reference

DAILY HEALTH SWEEP — no --topic; scans the whole cluster.

kafka-topics.sh --bootstrap-server $B --describe --unavailable-partitions
kafka-topics.sh --bootstrap-server $B --describe --under-min-isr-partitions
kafka-topics.sh --bootstrap-server $B --describe --at-min-isr-partitions
kafka-topics.sh --bootstrap-server $B --describe --under-replicated-partitions
kafka-consumer-groups.sh --bootstrap-server $B --list  (then --describe)

AFTER EVERY ROLLING RESTART

kafka-leader-election.sh --bootstrap-server $B \
  --election-type PREFERRED --all-topic-partitions
(or rely on auto.leader.rebalance.enable / Cruise Control)

SAFE OFFSET RESET

1. STOP all consumers in the group
2. --reset-offsets --to-current --dry-run > offsets.csv (EXPORT)
3. cp offsets.csv offsets.csv.bak (BACKUP)
4. edit offsets.csv
5. --reset-offsets --from-file offsets.csv --execute
6. START consumers

SAFE PARTITION REASSIGNMENT

0. (optional) move leadership OFF the source broker — bounce it, with auto-rebalance temporarily disabled
1. --topics-to-move-json-file topics.json --broker-list X,Y --generate
2. SAVE BOTH outputs: revert-*.json AND expand-*.json
3. --reassignment-json-file expand-*.json --execute --throttle <B/s>
4. --reassignment-json-file expand-*.json --verify (repeat until done)
5. preferred leader election to rebalance leadership

ROLLBACK: --reassignment-json-file revert-*.json --execute · ABORT: --cancel (⚠ reverts to prior set; order not guaranteed)

ESCALATION ORDER FOR UNSAFE OPS — each step riskier than the last.

1. Dynamic config (kafka-configs.sh) — e.g. flip unclean.leader.election.enable to unstick, THEN FLIP IT BACK
2. Preferred / unclean leader election (kafka-leader-election.sh)
3. Force controller move: delete /admin/controller znode
4. Unstick deletion: delete /admin/delete_topic/<topic> znode
5. Manual topic deletion: FULL CLUSTER SHUTDOWN + ZK edit + disk edit

The chapter's own closing advice

"As you begin to scale your Kafka clusters larger, EVEN THE USE OF THESE TOOLS MAY BECOME ARDUOUS AND DIFFICULT TO MANAGE. IT IS HIGHLY RECOMMENDED TO ENGAGE WITH THE OPEN SOURCE KAFKA COMMUNITY and take advantage of the many other open source projects in the ecosystem TO HELP AUTOMATE many of the tasks outlined in this chapter."

---