Running Kafka Locally

A local Kafka broker is the fastest way to learn the platform, prototype producers and consumers, and reproduce production behaviour on your laptop. Modern Kafka runs in KRaft mode, which folds metadata management into the broker itself — there is no separate ZooKeeper process to install, configure, or keep alive. This page shows two equally valid setups: a single Docker container for zero-install convenience, and a binary download when you want CLI tools and full control. Either way you end up with a broker on localhost:9092 ready to accept your first topic.

Option A: Docker in KRaft mode

The official apache/kafka image ships a single-process broker that acts as both a broker and a controller — perfect for local development. Running it through Docker Compose keeps the configuration in version control and makes start/stop a one-liner.

Create a docker-compose.yml:

services:
  kafka:
    image: apache/kafka:3.9.0
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      # Single node acting as both broker and controller
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@localhost:9093
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      # Listeners: clients connect on 9092, controller traffic on 9093
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
      # Single-broker safe defaults
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1

Start it in the background and confirm it is healthy:

docker compose up -d
docker compose logs -f kafka

Output:

kafka  | Kafka Server started (kafka.server.KafkaRaftServer)
kafka  | [KafkaRaftServer nodeId=1] Kafka Server started

The KAFKA_ADVERTISED_LISTENERS value is the address Kafka hands back to clients. If it advertises the wrong host, producers connect once, then fail on the second hop. For a laptop, localhost:9092 is correct; for other containers on the same network, advertise the service name instead.

The image already runs kafka-storage.sh format for you on first boot, so there is nothing else to initialise.

Option B: Binary download

When you want the bundled CLI tools (kafka-topics.sh, kafka-console-producer.sh, and friends) directly on your machine, download a release tarball. KRaft requires that storage be formatted with a cluster ID before the broker will start — this writes the metadata log that replaces ZooKeeper.

curl -O https://downloads.apache.org/kafka/3.9.0/kafka_2.13-3.9.0.tgz
tar -xzf kafka_2.13-3.9.0.tgz
cd kafka_2.13-3.9.0

# 1. Generate a unique cluster ID
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"

# 2. Format the log directories for KRaft
bin/kafka-storage.sh format \
  --standalone \
  --cluster-id "$KAFKA_CLUSTER_ID" \
  --config config/kraft/reconfig-server.properties

# 3. Start the broker
bin/kafka-server-start.sh config/kraft/reconfig-server.properties

The --standalone flag wires up a single-node controller quorum automatically. Leave this terminal running; the broker logs to stdout and listens on localhost:9092.

Run kafka-storage.sh format exactly once per data directory. Re-formatting an existing log directory will refuse to proceed unless you pass --ignore-formatted, and forcing it discards your data.

Docker vs binary

Aspect	Docker (Option A)	Binary (Option B)
Setup time	Seconds, no JDK needed	Requires Java 17+ installed
CLI tools on host	No (exec into container)	Yes, in bin/
Cleanup	docker compose down -v	Delete the extracted folder
Best for	Quick demos, CI, app dev	Learning the CLI, ops practice

Create your first topic

With a broker running, create a topic named orders. On a single broker you must use --replication-factor 1, because there is only one broker to hold a copy — asking for more replicas than brokers fails. Three partitions give you room to experiment with parallel consumers later.

If you used Docker, run the CLI inside the container:

docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create --topic orders \
  --partitions 3 --replication-factor 1

If you used the binary, run it from the extracted directory:

bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create --topic orders \
  --partitions 3 --replication-factor 1

Output:

Created topic orders.

List and describe topics

List every topic to confirm orders exists:

kafka-topics.sh --bootstrap-server localhost:9092 --list

Output:

orders

Then describe it to inspect partitions, the leader broker, and the in-sync replica (ISR) set:

kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic orders

Output:

Topic: orders   TopicId: 7Qk2... PartitionCount: 3  ReplicationFactor: 1  Configs:
        Topic: orders   Partition: 0    Leader: 1    Replicas: 1    Isr: 1
        Topic: orders   Partition: 1    Leader: 1    Replicas: 1    Isr: 1
        Topic: orders   Partition: 2    Leader: 1    Replicas: 1    Isr: 1

Every partition shows Leader: 1, Replicas: 1, and Isr: 1 — all three live on your single broker (node ID 1). This is exactly why local setups use a replication factor of 1: with one broker there is nowhere to put a second copy. The moment you move to a real cluster you would raise this to 3 so the loss of one broker never costs you data.

Best Practices

• Use KRaft, not ZooKeeper — every modern release defaults to KRaft, and ZooKeeper is removed in Kafka 4.x; never stand up a new ZooKeeper cluster.
• Pin the image or release version (apache/kafka:3.9.0) so your local broker matches CI and production rather than silently tracking latest.
• Keep --replication-factor 1 only for single-broker local setups and bump it to 3 in any environment where durability matters.
• Advertise the address clients can actually reach — mismatched advertised.listeners is the single most common “it connects then hangs” failure.
• Reset state cleanly with docker compose down -v (or by deleting the formatted log dirs) when you want a fresh cluster, instead of fighting stale topics.
• Pick more partitions than you need today — you can add partitions later, but you cannot remove them, and partition count caps consumer parallelism.

Related Topics

• What is Apache Kafka? — the log model and core building blocks.
• Kafka architecture overview — brokers, controllers, and replication in depth.
• Kafka CLI tools — the full toolbox for topics, producers, and consumers.
• Your first application — produce and consume events against this local broker.
• Core concepts glossary — quick definitions for offsets, ISR, and more.