Kafka

Apache Kafka is a framework for streaming data between internal systems.

We’re dealing with a lot of data these days – Big Data™ – and recording, storing, and moving it around is hard and expensive. Kafka helps stream that data throughout your company and distribute it to the systems that want to use it. The Kafka architecture works through a publish-subscribe pattern: you stream data in, and any system (your data warehouse, a billing system, etc.) can choose to subscribe to that data and consume it.