Creating a cluster Apache Kafka®

Written by

Yandex Cloud

Updated at April 8, 2026

If you have a project, you can create an Apache Kafka® cluster in it.

CLI

Management console

If the project does not exist yet, create it: kubectl create namespace <project name>.
Create the KafkaCluster resource file, e.g., using the touch kafkacluster.yaml command.

Open the file and paste the configuration below into it:

apiVersion: kafka.stackland.yandex.cloud/v1alpha1
kind: KafkaCluster
metadata:
  name: cluster-omdb
  labels:
    environment: production
    tier: messaging
spec:
  version: "4.0.0"
  controller:
    instances: 3
    storage:
      size: 1Gi
      # storageClass: "stackland-default"
  broker:
    instances: 3
    storage:
      size: 2Gi
      # storageClass: "stackland-default"
    diskAutoScaling:
      enabled: true
      maxSize: 300Gi
      minimalIncrement: 2Gi
      standardIncreasePercent: 20
      resizeTriggerPercent: 20
  listenerType: LoadBalancer # Network listener (LoadBalancer or ClusterIP)
  config: # configuration
    offsets.topic.replication.factor: 3
    transaction.state.log.replication.factor: 3
    transaction.state.log.min.isr: 2
    default.replication.factor: 3
    min.insync.replicas: 2

Apply the manifest: kubectl apply -f kafkacluster.yaml -n <project name>. Optionally, you can specify the project name in the metadata.namespace resource property and skip it in the command.

To connect to the cluster, follow the guide on Connecting to an Apache Kafka® cluster.

If you have not opened a project yet, select one.
In the left-hand menu, select Kafka Clusters.
Click Create.
Fill out the fields as follows:
- Cluster name: Use only lowercase letters, numbers, and hyphens.
- Kafka version: Select it from the list of available versions.
- Number of replicas of the message processing service.
- Computing resources, where Limits is the upper limit and Requests is the lower limit.
- Storage, where Disk type is the storage class (stackland-nvme, stackland-ssd, stackland-hdd, stackland-other). Learn more about storage classes in Disk subsystem. You can also enable Automatic storage expansion.
- Number of replicas of the coordination service.
- Computing resources, where Limits is the upper limit and Requests is the lower limit.
- Storage, where Disk type is the storage class (stackland-nvme, stackland-ssd, stackland-hdd, stackland-other).
Click Create.

To connect to the cluster, follow the guide on Connecting to an Apache Kafka® cluster.

Creating a cluster Apache Kafka®

Was the article helpful?