Managing Configurations

Kafka configuration is layered. Some settings are baked into server.properties and only take effect on restart, while many others can be changed live across the whole cluster — or for one broker, one topic, or one client — without bouncing a single process. Knowing which layer “wins” and which knobs are dynamic is the difference between a clean tuning change and an unplanned rolling restart in production. This page explains the config levels, how precedence resolves, and how to alter live configs with kafka-configs.sh.

Config levels and precedence

A given setting can be defined in several places. Kafka resolves them in a fixed order, with the most specific scope winning. For broker settings the order is:

Per-broker dynamic config — set on a single broker by ID, stored in the cluster metadata.
Cluster-wide dynamic config — a default entity applied to every broker.
Static server.properties — read once at startup.
Hard-coded Kafka default — the built-in fallback.

Topic-level settings follow a parallel rule: a per-topic override beats the broker-level default of the same property. For example, a topic with no retention.ms override inherits the broker’s log.retention.ms; set the topic override and that topic alone changes.

Scope	Where it lives	Restart needed	Use case
Static broker	`server.properties`	Yes	Identity, ports, log dirs, listeners
Cluster-wide dynamic	metadata (`--entity-default`)	No	Fleet-wide tuning (e.g. thread pools)
Per-broker dynamic	metadata (`--entity-name <id>`)	No	One broker (e.g. throttle a noisy node)
Topic	metadata (`--entity-type topics`)	No	Per-topic retention, cleanup, segments
Client / user	metadata (`--entity-type clients`)	No	Quotas for a producer/consumer

Not every broker property is dynamic. Settings like broker.id, log.dirs, and listener definitions are read-only and only change via server.properties plus a restart. The kafka-configs.sh tool rejects an --alter on a read-only key with a clear error.

Inspecting current configs

Before changing anything, read the effective values. The --describe action shows configs and, crucially, their source — so you can tell a dynamic override from a static default.

# All dynamic + static configs for broker 1
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-name 1 --describe

# Topic-level configs for the orders topic
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type topics --entity-name orders --describe

Output:

Dynamic configs for broker 1 are:
  num.io.threads=16 sensitive=false synonyms={DYNAMIC_BROKER_CONFIG:num.io.threads=16, STATIC_BROKER_CONFIG:num.io.threads=8, DEFAULT_CONFIG:num.io.threads=8}

The synonyms list is the precedence chain, top to bottom. Here num.io.threads resolves to 16 from the dynamic broker config, overriding the 8 in server.properties.

Changing topic configs at runtime

Topic configs are the most common live change. Use --alter --add-config to set or update keys and --delete-config to remove an override (reverting to the broker default). Changes propagate to all partitions of the topic immediately.

# Shorten retention to 3 days and shrink segments for the orders topic
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type topics --entity-name orders \
  --alter --add-config retention.ms=259200000,segment.bytes=536870912

# Revert retention to the broker default
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type topics --entity-name orders \
  --alter --delete-config retention.ms

Changing broker configs cluster-wide

To tune the whole fleet at once, target the default broker entity. This writes a cluster-wide dynamic config that every broker picks up without a restart — ideal for adjusting thread pools, replication throttles, or compression under load.

# Apply to every broker in the cluster
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter --add-config log.cleaner.threads=4

Changing one broker

Sometimes you need to adjust a single node — for instance, throttling outbound replication on a broker that is catching up after a disk replacement. Target it by ID with --entity-name.

# Cap leader/follower replication throughput on broker 2 (bytes/sec)
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-name 2 \
  --alter --add-config leader.replication.throttled.rate=52428800,follower.replication.throttled.rate=52428800

# Remove the throttle once the broker has caught up
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-name 2 \
  --alter --delete-config leader.replication.throttled.rate,follower.replication.throttled.rate

A per-broker value overrides the cluster-wide default for that node only; deleting it falls back to the cluster-wide setting (then to server.properties, then to the built-in default).

Sensitive configs and the AdminClient equivalent

Sensitive values such as ssl.keystore.password are accepted by --add-config but never returned in plaintext by --describe (they show as sensitive=true). Programmatically, the same operations run through the AdminClient using incrementalAlterConfigs with a ConfigResource of type BROKER or TOPIC. Prefer the incremental API there — it is additive and will not wipe configs you did not name.

Static and dynamic configs can disagree. After a dynamic override, server.properties still holds the old value. To make a live change permanent across a full cluster rebuild, mirror it into server.properties (or your config-management repo) as well — otherwise a clean restart silently loses it only if the dynamic store is also cleared.

Best Practices

Always --describe first and read the synonyms chain so you know which layer you are actually overriding.
Use --entity-default for fleet-wide changes and reserve per-broker overrides for genuinely node-specific needs like throttling.
Treat dynamic config changes as deployments: record them in version control or runbooks, since metadata-stored values are easy to forget.
Reserve server.properties for truly static, identity-level settings (broker id, listeners, log dirs); tune everything dynamic at runtime to avoid restarts.
Delete overrides with --delete-config rather than re-setting defaults, so the value cleanly inherits future default changes.
Validate that a key is dynamic before relying on a live change; read-only keys will error and require a rolling restart instead.