Contact UsGet started
© 2025 Direct Cursus Technology L.L.C.

Pre-configuring an Apache Kafka® cluster connection

Written by
Updated at November 24, 2025

You can connect to Managed Service for Apache Kafka® cluster hosts:

  • Over the internet if you configured public access for the cluster. You can only connect to this type of cluster using an SSL connection.
  • From Yandex Cloud virtual machines located in the same cloud network. If the cluster is not publicly available, you do not need to use an SSL connection to connect to such VMs.

You can connect to an Apache Kafka® cluster both with encryption (SASL_SSL, port 9091) and without it (SASL_PLAINTEXT, port 9092).

To connect to an Apache Kafka® cluster:

  1. Create users for clients (producers and consumers) with access to the required topics.
  2. Connect the clients to the cluster:

There are ready-made Apache Kafka® API implementations for most popular programming languages. To view use examples, see the Code examples section.

Configuring security groups

To connect to a cluster, security groups must include rules allowing traffic from certain ports, IP addresses, or from other security groups.

Rule settings depend on the chosen connection method:

Configure all cluster security groups to allow incoming traffic on port 9091 from any IP address. To do this, create the following inbound rule:

  • Port range: 9091
  • Protocol: TCP
  • Source: CIDR
  • CIDR blocks: 0.0.0.0/0

To allow the use of Managed Service for Apache Kafka® API, e.g., to work with Managed Schema Registry, add a rule for incoming traffic:

  • Port range: 443
  • Protocol: TCP
  • Source: CIDR
  • CIDR blocks: 0.0.0.0/0

  1. Configure all the cluster security groups to allow incoming traffic from the security group where the VM is located on ports 9091 and 9092. To do this, create the following inbound rule in these groups:

    • Port range: 9091-9092
    • Protocol: TCP
    • Source: Security group
    • Security group: If your cluster and VM are in the same security group, select Current (Self). Otherwise, specify the VM security group.

    To allow the use of Managed Service for Apache Kafka® API, e.g., to work with Managed Schema Registry, add a rule for incoming traffic:

    • Port range: 443
    • Protocol: TCP
    • Source: CIDR
    • CIDR blocks: 0.0.0.0/0

  2. Configure the VM security group to allow VM connections and traffic between the VM and the cluster hosts.

    For example, you can set the following rules for a VM:

    • For incoming traffic:

      • Port range: 22
      • Protocol: TCP
      • Source: CIDR
      • CIDR blocks: 0.0.0.0/0

      This rule allows VM connections over SSH.

    • For outgoing traffic:

      • Protocol: Any (Any)
      • Port range: 0-65535
      • Destination name: CIDR
      • CIDR blocks: 0.0.0.0/0

      This rule allows all outgoing traffic, thus enabling you not only to connect to the cluster but also to install all required certificates and tools on your VM.

Note

You can specify more granular rules for your security groups, such as allowing traffic only within specific subnets.

Make sure to configure the security groups properly for all subnets where the cluster hosts will reside. With incomplete or incorrect security group settings, you may lose access to the cluster.

For more information about security groups, see Security groups.

Getting an SSL certificate

To use an encrypted connection, get an SSL certificate:

mkdir -p /usr/local/share/ca-certificates/Yandex/ && \
wget "https://storage.yandexcloud.net/cloud-certs/CA.pem" \
     --output-document /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt && \
chmod 0655 /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt

The certificate will be saved to the /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt file.

mkdir $HOME\.kafka; curl.exe -o $HOME\.kafka\YandexInternalRootCA.crt https://storage.yandexcloud.net/cloud-certs/CA.pem

The certificate will be saved to the $HOME\.kafka\YandexInternalRootCA.crt file.

Corporate policies and antivirus software can block the download of certificates. For more information, see FAQ.

This certificate is also used to access the Managed Service for Apache Kafka® REST API.

Getting FQDNs of Apache Kafka® hosts

To connect to a host, you will need its FQDN. Example of a Apache Kafka® host FQDN:

rc1a-goh2a9tr********.mdb.yandexcloud.net

You can get the FQDN by doing one of the following:

  • View the FQDN in the management console:

    1. Open to the cluster page.
    2. Navigate to Hosts.
    3. Copy the Host FQDN value.

  • In the management console, copy the cluster connection command. This command contains the broker host FQDN. To get the command, go to the cluster page and click Connect.

  • Get the list of cluster hosts using the CLI or API.

With the Managed Service for Apache Kafka® REST API, you can send requests to any broker host: the API is available from all the cluster's broker hosts.

What's next

  • Connect to the cluster using command line tools.
  • Integrate the cluster connection into your application code.
Previous
Deleting a cluster
Next
Connecting from applications