Apache Kafka® Raft protocol in Managed Service for Apache Kafka® clusters

Apache Kafka® Raft (short name: KRaft) is an approval protocol allowing you to store metadata and manage it in Apache Kafka®. KRaft makes Apache Kafka® independent of the external metadata synchronization service, ZooKeeper.

KRaft is supported by Managed Service for Apache Kafka® clusters with Apache Kafka® 3.6 or higher.

ZooKeeper is supported by clusters with Apache Kafka® 3.9 or lower.

Managed Service for Apache Kafka® cluster topologyManaged Service for Apache Kafka® cluster topology

KRaft is automatically configured and placed in a cluster based on the selected mode and the number of broker hosts:

• KRaft (combined mode): One Apache Kafka® host accommodates a broker and a KRaft metadata controller at the same time. In the cluster, only three Apache Kafka® hosts get created in one of these configurations:

  • Three hosts in the same availability zone.
  • Each host in a separate availability zone.

  You cannot set the number of broker hosts manually.

• KRaft (on separate hosts): A broker and a KRaft metadata controller are on separate hosts. When you create a multiple-host cluster, three dedicated KRaft hosts are added to it.

  The number of broker hosts is set manually.

You cannot delete KRaft hosts. The number of KRaft hosts is fixed.

For more information on creating a Managed Service for Apache Kafka® cluster with KRaft, see this guide.

Apache Kafka® Raft pros and consApache Kafka® Raft pros and cons

KRaft advantages stem from the fact that Apache Kafka® no longer depends on the ZooKeeper external system:

• There is a single mechanism in place for information security instead of multiple ones.
• A cluster with KRaft is easier to manage, as it has no ZooKeeper hosts.
• The cluster becomes more fault-tolerant and recovers almost instantly in case of a failure.
• You can create a greater number of partitions within the cluster.

For more information, see the Confluent website and documentation.

KRaft cons arise from the cluster topology specifics:

• In the combined mode, KRaft metadata controllers are not isolated from Apache Kafka® as effectively as they are in split mode, which features a dedicated subcluster for metadata controllers. Therefore, Apache Kafka® does not recommend using the combined mode in critical development environments. The combined mode is more suitable for local development and testing.
• In a cluster with KRaft, you cannot flexibly manage the number of hosts: you can have exactly three hosts.