Creating a Apache Kafka® cluster

1. If the project does not exist yet, create it: kubectl create namespace <project name>.

2. Create the KafkaCluster resource file, e.g., using the touch kafkacluster.yaml command.

3. Open the file and paste the configuration below into it:

   apiVersion: kafka.stackland.yandex.cloud/v1alpha1
   kind: KafkaCluster
   metadata:
     name: cluster-omdb
     labels:
       environment: production
       tier: messaging
   spec:
     version: "4.0.0"
     controller:
       instances: 3
       storage:
         size: 1Gi
         # storageClass: "stackland-default"
     broker:
       instances: 3
       storage:
         size: 2Gi
         # storageClass: "stackland-default"
       diskAutoScaling:
         enabled: true
         maxSize: 300Gi
         minimalIncrement: 2Gi
         standardIncreasePercent: 20
         resizeTriggerPercent: 20
     listenerType: LoadBalancer # Network listener (LoadBalancer or ClusterIP)
     config: # configuration
       offsets.topic.replication.factor: 3
       transaction.state.log.replication.factor: 3
       transaction.state.log.min.isr: 2
       default.replication.factor: 3
       min.insync.replicas: 2
   

4. Apply the manifest: kubectl apply -f kafkacluster.yaml -n <project name>. Optionally, you can specify the project name in the metadata.namespace resource property and skip it in the command.

To connect to the cluster, follow the guide on Connecting to an Apache Kafka® cluster.