Ask Sendai

Apache Kafka is a beautiful system. It scales well, it is stable and it provides phenomenal system architecture flexibility. After 5 years of running production Kafka clusters, I have collected a list of tips and pitfalls. Some of them were learnt the hard way. If you work in a small team rolling out Kafka to production, those might prove useful. The article assume familiarity with basic Kafka concepts, such as brokers, topics, producers and consumers.

Scalability is often a key issue for many growing organizations. That’s why many organizations use Apache Kafka, a popular messaging and streaming platform. It is horizontally scalable, cloud-native, and versatile. It can serve as a traditional publish-and-subscribe messaging system, as a streaming platform, or as a distributed state store. Companies around the world use Apache Kafka to build real-time streaming applications, streaming data pipelines, and event-driven architectures. Source: redhat.com