Networking in Managed Service for Apache Kafka®

When creating a cluster, you can:

• Set the network for cluster hosts.
• Specify the availability zones where the cluster hosts will reside.
• Set subnets in all availability zones.
• Turn on public access to the cluster from outside Yandex Cloud.

If there are ZooKeeper hosts in the cluster, each of the three ZooKeeper hosts will use its dedicated availability zone and the subnet selected in it. For more information, see Resource relationships in the service.

Host name and FQDNHost name and FQDN

Managed Service for Apache Kafka® generates the name of each cluster host when creating it. This name will be the host's fully qualified domain name (FQDN). The host name and, consequently, FQDN cannot be changed.

For more information on how to get a host FQDN, see this guide.

You can use the FQDN to access the host within a single cloud network. For more information, see the Yandex Virtual Private Cloud documentation.

Public access to clustersPublic access to clusters

If you request public access when creating a cluster, all broker hosts in the cluster will be accessible from outside Yandex Cloud. To connect to such a cluster, use the FQDN of one or more cluster's broker hosts.

You cannot request public access after creating a cluster.

When you delete a cluster with public access enabled, all public IP addresses assigned to this cluster are revoked.

Security groupsSecurity groups

Security groups follow the All traffic that is not allowed is prohibited principle. To connect to a cluster, configure security group rules. These rules allow traffic from certain ports, IP addresses, or other security groups. For example, a VM will not be able to connect to a cluster in the following cases:

• The VM is in subnet 10.128.0.0/16, whereas the incoming traffic rules only specify subnet 10.133.0.0/24.
• The VM is in subnet 10.133.0.0/24 but attempts to access a port not specified in the security group rules.

For information on how to configure security groups, see Pre-configuration for connection to an Apache Kafka® cluster.

Specifics of working with security groups:

• Security group settings only affect whether it will be possible to connect to the cluster. They do not affect cluster operation, such as replication of topic sections by broker hosts, connections between brokers and ZooKeeper hosts, and other features.

• Even if the cluster and the connecting VM are in the same security group, there will be no connection unless you set up rules within this group that allow traffic between the VM and the cluster.

  However, by default, such rules are contained within the security group added automatically when creating a cloud network. They are the Self rules that allow unlimited traffic within a group.