Discuss with expertTry it for free
© 2026 Direct Cursus Technology L.L.C.

Pre-configuring an Apache Kafka® cluster connection

Written by
Updated at August 6, 2026
View in Markdown

You can connect to Managed Service for Apache Kafka® cluster hosts:

  • Over the internet if you configured public access for your cluster. This cluster allows connections only via SSL.
  • From Yandex Cloud virtual machines located in the same cloud network. If the cluster is not publicly accessible, you do not need to use SSL to connect from these VMs.

You can connect to an Apache Kafka® cluster both with encryption (SASL_SSL, port 9091) and without it (SASL_PLAINTEXT, port 9092).

To connect to an Apache Kafka® cluster:

  1. Create users for clients (producers and consumers) with access to the appropriate topics.
  2. Connect the clients to the cluster:

There are ready-made Apache Kafka® API implementations for most popular programming languages. For use cases, see Code examples.

Configuring security groups

To connect to a cluster, security groups must include rules allowing traffic from certain ports, IP addresses, or from other security groups.

Rule settings depend on the chosen connection method:

Configure all cluster security groups to allow incoming traffic on port 9091 from any IP address. To do this, create the following inbound rule:

  • Port range: 9091.
  • Protocol: TCP.
  • Source: Address range.
  • IPv4 CIDR: 0.0.0.0/0.

To allow the use of Managed Service for Apache Kafka® API, e.g., to work with Managed Schema Registry, add the following inbound rule:

  • Port range: 443.
  • Protocol: TCP.
  • Source: Address range.
  • IPv4 CIDR: 0.0.0.0/0.

  1. Configure all cluster security groups to allow incoming traffic on ports 9091 and 9092 from your VM’s security group. To do this, create the following inbound rule in these groups:

    • Port range: 9091-9092.
    • Protocol: TCP.
    • Source: Security group.
    • Security group: If your cluster and VM share the same security group, select Current (Self). Otherwise, specify the VM security group.

    To allow the use of Managed Service for Apache Kafka® API, e.g., to work with Managed Schema Registry, add the following inbound rule:

    • Port range: 443.
    • Protocol: TCP.
    • Source: Address range.
    • IPv4 CIDR: 0.0.0.0/0.

  2. Configure the VM security group to allow VM connections and traffic between the VM and cluster hosts.

    For example, you can set the following rules for your VM:

    • For incoming traffic:

      • Port range: 22.
      • Protocol: TCP.
      • Source: Address range.
      • IPv4 CIDR: 0.0.0.0/0.

      This rule allows inbound connections to the VM over SSH.

    • For outgoing traffic:

      • Protocol: Any.
      • Port range: 0-65535.
      • Destination name: Address range.
      • IPv4 CIDR: 0.0.0.0/0.

      This rule permits all outbound traffic, allowing you to install any necessary certificates and tools on your VM.

Note

You can specify more granular security group rules, such as allowing traffic only in specific subnets.

Make sure to configure the security groups correctly for all subnets where the cluster hosts will reside. With incomplete or incorrect security group settings, you may lose access to the cluster.

For more information about security groups, see here.

Obtaining an SSL certificate

To use an encrypted connection, get an SSL certificate:

mkdir -p /usr/local/share/ca-certificates/Yandex/ && \
wget "https://storage.yandexcloud.net/cloud-certs/CA.pem" \
     --output-document /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt && \
chmod 0655 /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt

The certificate will be saved to the /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt file.

mkdir $HOME\.kafka; curl.exe -o $HOME\.kafka\YandexInternalRootCA.crt https://storage.yandexcloud.net/cloud-certs/CA.pem

The certificate will be saved to the $HOME\.kafka\YandexInternalRootCA.crt file.

Your corporate security policies and antivirus software may block the certificate download. For more information, see FAQ.

This certificate is also used to access the Managed Service for Apache Kafka® REST API.

Getting FQDNs of Apache Kafka® hosts

To connect to a host, you need its fully qualified domain name (FQDN). Here is an example of a Apache Kafka® host FQDN:

rc1a-goh2a9tr********.mdb.yandexcloud.net

You can get the FQDN using one of the following methods:

  • Check the FQDN in the management console:

    1. Navigate to the cluster page.
    2. Navigate to Hosts.
    3. Copy the Host FQDN column value.

  • In the management console, copy the cluster connection command It contains the broker host FQDN. To get the command, go to the cluster page and click Connect.

  • Get the list of cluster hosts using the CLI or API.

With the Managed Service for Apache Kafka® REST API, you can send requests to any broker host: the API is available from all the cluster's broker hosts.

What's next

  • Connect to the cluster using command line tools.
  • Integrate the cluster connection into your application code.
Previous
Deleting a cluster
Next
Connecting from applications