15.8 B.5 Stream processing frameworks

"While the Apache Kafka project includes Kafka Streams for building applications, it's not the only choice out there."

Framework	Runs on	Character
Apache Samza	YARN	"specifically designed for Kafka. While it PREDATES Kafka Streams, it was developed by MANY OF THE SAME PEOPLE, and as a result the two SHARE MANY CONCEPTS. However, unlike Kafka Streams, Samza runs on Yarn and provides A FULL FRAMEWORK for applications to run in."
Apache Spark	—	"oriented toward BATCH processing. It handles streams by considering them to be FAST MICROBATCHES. This means the LATENCY IS A LITTLE HIGHER, BUT FAULT TOLERANCE IS SIMPLY HANDLED THROUGH REPROCESSING BATCHES, and LAMBDA ARCHITECTURE IS EASY. It also has the benefit of wide community support."
Apache Flink	YARN, Mesos, Kubernetes, standalone	"specifically oriented toward stream processing and operates with VERY LOW LATENCY.* ... It also supports Python and R with provided high-level APIs."*
Apache Beam	Samza, Spark, Flink as runners	"doesn't provide stream processing DIRECTLY but instead promotes itself as A UNIFIED PROGRAMMING MODEL FOR BOTH BATCH AND STREAM PROCESSING. It utilizes platforms like Samza, Spark, and Flink as RUNNERS for components in an overall processing pipeline."

Mapping these onto Ch. 14 §7's selection criteria:

What you need	What to pick
“low milliseconds actions”	Flink (very low latency); avoid Spark (“focuses on microbatches” — Ch. 14 says opt for event-by-event instead)
“near real-time analytics”	Flink, Spark, or Kafka Streams
“asynchronous microservices”	Kafka Streams — library, local state, no cluster manager
“ingest”	Kafka Connect, not any of these (Ch. 9)
want one model for batch + stream	Beam — as an abstraction over the others
already run YARN / want a full framework	Samza
already run Spark for batch	Spark — community support, easy Lambda architecture

Kafka Streams	Samza / Spark / Flink
A library. Your app is the cluster (Ch. 14 §4.1).	Frameworks. You deploy to a cluster manager.

Ch. 1 stated this as a deliberate design stance: Connect and Streams are “APIs and libraries, not full platforms… a solid foundation to build on and flexibility as to where they can be run.”