All Series
Series

Apache Kafka from Zero to Production

Kafka sits at the center of modern data architectures, but it can feel like a lot to take in. This series starts with the core ideas and builds up, one practical post at a time, to the producers, consumers, replication, and operational habits you need to run Kafka with confidence.
4 articles Created Oct 2026
Apache Kafka from Zero to Production

About this series

Apache Kafka is an open-source, distributed event streaming platform used to collect, store, process, and route real-time data feeds. It powers everything from payment pipelines to live dashboards, yet its vocabulary of topics, partitions, offsets, and consumer groups can be a hurdle at first.

This series walks through Kafka step by step. Each part is short enough to read in one sitting, includes hands-on examples, and builds on the one before it.

Who it's for

Developers, data engineers, and architects who are new to Kafka or want a clearer mental model of how it works. No prior streaming experience is needed.

What you'll learn

  1. Apache Kafka Explained: what Kafka is, its core concepts, and when to use it

  2. Producers: keys and partitioning, batching, compression, acks, and idempotent writes

  3. Consumers and consumer groups: offsets, commits, rebalancing, and monitoring lag

  4. Inside the cluster: brokers, replication, in-sync replicas, leader election, and KRaft

  5. Delivery guarantees: at-most-once, at-least-once, and exactly-once with transactions

  6. Connect and Streams: moving data in and out of Kafka and processing it in flight

  7. Kafka in production: partition sizing, retention, security, monitoring, and managed options

How to read it

Start with Part 1 and work through in order, or jump to the topic you need. Every part ends with a short pointer to what comes next.

Articles in this Series

4 articles · curriculum order

Part 1

Apache Kafka Explained: The Backbone of Real-Time Data

Kafka turns a tangle of point-to-point integrations into one shared, durable log of events. This first post covers the core ideas, why Kafka is so fast, how delivery guarantees work, and the quickest way to run it on your own machine.

Samod AlexSamod Alex Oct 11 5 min read
Apache Kafka Explained: The Backbone of Real-Time Data
Part 2

Kafka Producers: Getting Events In

Every Kafka pipeline starts with a producer. Learn how message keys decide where events land and keep them in order, how batching and compression raise throughput, and which settings protect you from lost or duplicated writes.

Samod AlexSamod Alex Oct 12 5 min read
Kafka Producers: Getting Events In
Part 3

How Kafka Consumers Work: Offsets, Groups, and Rebalancing

Reading from Kafka is more than a poll loop. This post explains offsets and commits, how consumer groups split partitions and scale out, what happens during a rebalance, and how to monitor lag before it becomes an incident.

Samod AlexSamod Alex Oct 13 5 min read
How Kafka Consumers Work: Offsets, Groups, and Rebalancing
LatestPart 4

Inside the Kafka Cluster: Brokers, Replication, and KRaft

Kafka stays available when brokers fail because of how it stores and replicates data. See how partitions are copied across brokers, how leaders are elected, and how KRaft now handles cluster metadata without ZooKeeper.

Samod AlexSamod Alex Oct 14 5 min read
Inside the Kafka Cluster: Brokers, Replication, and KRaft

Actions

Overview

Total Articles4
CreatedOct 2026
Last UpdatedOct 2026

Contributors

Samod Alex
Samod Alex
@samod