Learn Labs
1. Meet Kafka

1.8 Historical / trivia worth knowing

  • Created at LinkedIn; team led by Jay Kreps (who had previously built and open-sourced Voldemort, a distributed key-value store), with Neha Narkhede and later Jun Rao.
  • Open source on GitHub late 2010; Apache incubator July 2011; graduated October 2012.
  • Used at Netflix, Uber, and many others in the largest data pipelines in the world.
  • LinkedIn maintains Cruise Control, Kafka Monitor, Burrow.
  • Confluent (founded fall 2014 by Kreps, Narkhede, Rao) released ksqlDB, a schema registry, and a REST proxy under a community license — note: not strictly open source, as it includes use restrictions. Confluent also runs Kafka Summit (since 2016).
  • The name: Jay Kreps — "since Kafka was a system optimized for writing, using a writer's name would make sense. I had taken a lot of lit classes in college and liked Franz Kafka. Plus the name sounded cool for an open source project. So basically there is not much of a relationship."