14. Stream Processing

Source: Kafka: The Definitive Guide, 2nd Ed., Ch. 14 The historical framing: "Kafka was traditionally seen as a powerful message bus, capable of delivering streams of events but without processing or transformation capabilities. ... many companies had a system containing many streams of interesting data, stored for long amounts of time and perfectly ordered, JUST WAITING FOR SOME STREAM PROCESSING FRAMEWORK TO SHOW UP AND PROCESS THEM. In other words, in the same way that data processing was significantly more difficult BEFORE DATABASES WERE INVENTED, STREAM PROCESSING WAS HELD BACK BY THE LACK OF A STREAM PROCESSING PLATFORM."

Since 0.10.0, Kafka ships Kafka Streams — "a powerful stream processing library as part of its collection of client libraries... This allows developers to consume, process, and produce events IN THEIR OWN APPS, WITHOUT RELYING ON AN EXTERNAL PROCESSING FRAMEWORK."

---