Learn Labs
10. Cross-Cluster Data Mirroring

10.11 Decision guide

START: do you need more than one cluster?

  • Clusters are fully independent (different depts / SLAs / security) ► “the same as running a single cluster multiple times” — done.
  • Need RPO = 0 (synchronous, often a LEGAL requirement)?
    • ► STRETCH CLUSTER across ≥3 DCs / AZs (or 2.5 DC), with broker.rack + acks=all + min.insync.replicas. ⚠ protects ONLY against DC failure, not Kafka/app failure
    • ► or Confluent MRC (≤50 ms, observers, auto-promotion)
  • Producing in many regions, need a global view but only in one place? ► HUB-AND-SPOKE. Simple. One-directional. Monitorable. ⚠ regional apps cannot see other regions’ data
  • Need every region to serve every user, locally? ► ACTIVE-ACTIVE. “Most scalable, resilient, flexible, and cost-effective.” You must solve:
    • replication cycles (namespaces / alias prefixes)
    • user stickiness
    • conflict resolution (consistent, DC-independent rules)
  • Only need DR (often a legal box to tick)? ► ACTIVE-STANDBY. Simplest to set up, wasteful, and “not possible to failover without losing data or having duplicates. Often both.” Choose a failover-offset strategy:
    • simplest → auto.offset.reset (latest)
    • explainable → TIME-BASED ◄ recommended default
    • most precise → OFFSET TRANSLATION (MM2 built-in)

Plan the scrape-and-reverse for failback. PRACTICE QUARTERLY.

The chapter's closing warning

*"remember that multicluster configuration and mirroring pipelines SHOULD BE MONITORED AND TESTED JUST LIKE EVERYTHING ELSE you take into production. BECAUSE MULTICLUSTER MANAGEMENT IN KAFKA CAN BE EASIER THAN IT IS WITH RELATIONAL DATABASES, SOME ORGANIZATIONS TREAT IT AS AN AFTERTHOUGHT and neglect to apply proper design, planning, testing, deployment automation, monitoring, and maintenance.

By taking multicluster management seriously, preferably as part of a HOLISTIC disaster or geodiversity plan for the ENTIRE ORGANIZATION that involves MULTIPLE APPLICATIONS AND DATA STORES, you will greatly increase the chances of successfully managing multiple Kafka clusters."*


On this page