Producer Serializers

Kafka brokers are deliberately ignorant of your data: every record is stored and transmitted as a pair of opaque byte arrays, one for the key and one for the value. Serializers are the bridge between your typed application objects and those raw bytes. Choosing the right serializer — and getting it to agree with the deserializer on the consumer side — is one of the most consequential decisions in a Kafka deployment, because a mismatch surfaces only at runtime as garbled data or hard deserialization failures.

The Serializer interface

Every producer is configured with exactly two serializers, one for keys and one for values, via the mandatory key.serializer and value.serializer properties. Both must implement org.apache.kafka.common.serialization.Serializer<T>:

public interface Serializer<T> extends Closeable {
    default void configure(Map<String, ?> configs, boolean isKey) {}

    byte[] serialize(String topic, T data);

    default byte[] serialize(String topic, Headers headers, T data) {
        return serialize(topic, data);
    }

    default void close() {}
}

The contract is small. configure runs once at producer startup and receives all producer properties plus an isKey flag so a single class can behave differently for keys versus values. serialize is called for every record and must return null for null input (Kafka treats a null value as a tombstone, so never throw on null). The Headers-aware overload lets you record schema metadata in record headers — useful for schema-registry-style formats.

Built-in serializers

Kafka ships serializers for the common primitive types. You almost never need to write these yourself.

Type	Serializer class	Notes
String	StringSerializer	UTF-8 by default; override with key.serializer.encoding / value.serializer.encoding
Integer	IntegerSerializer	4-byte big-endian
Long	LongSerializer	8-byte big-endian
Short	ShortSerializer	2-byte big-endian
Float / Double	FloatSerializer / DoubleSerializer	IEEE 754
byte[]	ByteArraySerializer	Pass-through, no transformation
ByteBuffer	ByteBufferSerializer	Wraps an existing buffer
UUID	UUIDSerializer	Serializes the canonical string form
Void	VoidSerializer	Always emits null, for keyless records

A typical string-keyed, string-valued producer is configured like this:

Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
        StringSerializer.class.getName());
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
        StringSerializer.class.getName());

try (Producer<String, String> producer = new KafkaProducer<>(props)) {
    producer.send(new ProducerRecord<>("orders", "order-42", "{\"total\":99}"));
}

The generic type parameters on KafkaProducer<K, V> are a compile-time convenience only — they are not enforced against the serializer classes you name in the config, so keep them in sync manually.

Writing a custom serializer

When you need to send a domain object — say a JSON-encoded event — you write a serializer that converts your type to bytes. The example below serializes any object to JSON using Jackson. Define the event as a record:

public record OrderEvent(String orderId, String customer, long amountCents) {}

Then implement Serializer<T>:

import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.kafka.common.errors.SerializationException;
import org.apache.kafka.common.serialization.Serializer;

public class JsonSerializer<T> implements Serializer<T> {

    private final ObjectMapper mapper = new ObjectMapper();

    @Override
    public byte[] serialize(String topic, T data) {
        if (data == null) {
            return null;
        }
        try {
            return mapper.writeValueAsBytes(data);
        } catch (Exception e) {
            throw new SerializationException(
                    "Failed to serialize value for topic " + topic, e);
        }
    }
}

Two details matter for production correctness. First, return null on null input so tombstone semantics keep working. Second, wrap any failure in org.apache.kafka.common.errors.SerializationException — the producer treats it as a fatal, non-retriable error for that record and fails the send fast rather than retrying a payload that can never serialize.

Wire it in like any built-in serializer:

props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
        StringSerializer.class.getName());
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
        JsonSerializer.class.getName());

try (Producer<String, OrderEvent> producer = new KafkaProducer<>(props)) {
    var event = new OrderEvent("order-42", "acme", 9900);
    RecordMetadata md = producer.send(
            new ProducerRecord<>("orders", event.orderId(), event)).get();
    System.out.printf("sent to %s-%d @ offset %d%n",
            md.topic(), md.partition(), md.offset());
}

Output:

sent to orders-0 @ offset 17

Whatever serializer you choose on the producer, the consumer must use a compatible deserializer for the same field. A JsonSerializer on the value side requires a matching JsonDeserializer; an IntegerSerializer key needs an IntegerDeserializer. Mismatches do not fail at startup — they fail when the first record is read.

Spring for Apache Kafka

With Spring Boot you usually configure serializers declaratively and let the framework manage the producer factory. Spring’s JsonSerializer (from org.springframework.kafka.support.serializer) handles arbitrary objects out of the box:

spring:
  kafka:
    bootstrap-servers: localhost:9092
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
      properties:
        spring.json.add.type.headers: true

You then inject a typed KafkaTemplate<String, OrderEvent> and call send without touching byte arrays directly.

Beyond JSON: schemas and the registry

Hand-rolled JSON is fine for simple, low-churn payloads, but it carries no schema and no compatibility guarantees. For evolving contracts across many teams, prefer Avro, Protobuf, or JSON Schema backed by a Schema Registry, which validates compatibility on write and embeds a schema ID in each record. That topic — including KafkaAvroSerializer and registry configuration — is covered in the dedicated serialization section of these docs.

Best Practices

• Always set key.serializer and value.serializer explicitly; there are no defaults and the producer fails to start without them.
• Return null for null input in custom serializers so log-compaction tombstones keep working.
• Throw SerializationException (not a generic exception) on failure so the producer surfaces a clear, non-retriable error.
• Keep the producer’s generic type parameters aligned with the configured serializer classes — the compiler will not catch a mismatch for you.
• Make custom serializers stateless and thread-safe; the producer shares one instance across all sending threads.
• For schema evolution across teams, graduate from ad-hoc JSON to Avro/Protobuf with a Schema Registry rather than versioning fields by hand.
• Document the serializer/deserializer pairing for each topic so consumers can be configured to match.

Related Topics

• Producer Overview
• Producer Configuration
• Message Keys and Partitioning
• Producer Interceptors
• Compression