Contact UsGet started

Configuring security groups

Written by
Updated at December 11, 2024

Security groups follow the All traffic that is not allowed is prohibited principle. For a cluster to work:

  1. In its security groups, create rules to allow relevant traffic for the cluster nodes:

    Tip

    Place each of these rule sets into a separate security group.

    This will make it easy to apply these security groups to the cluster and its node groups.

  2. (Optional) If planning to use a Yandex Application Load Balancer L7 balancer together with the cluster, add rules for the balancer as well.

  3. Apply security groups with these rules to the cluster and node groups.

You can specify more detailed rules for your security groups, e.g., to allow traffic only in specific subnets.

Security groups must be correctly configured for all subnets that will host the cluster. This determines the performance and availability of the cluster and the services running there.

Prior to editing security groups or the settings of any included rules, make sure this is not going to disrupt the cluster or its node groups.

Alert

Do not delete the security groups linked to a running cluster or node group. This may cause prohibition of network traffic and disrupt the operation of the cluster and its nodes.

Creating rules for service traffic

Warning

Rules for service traffic are required for a regional cluster to work.

Allowing traffic for a cluster and node groups

For the cluster to work correctly, create rules for incoming and outgoing traffic and apply them to the cluster and node groups:

  1. Add rules for incoming traffic that allow:

    • Health checks of the network load balancer:
      • Port range: 0-65535.
      • Protocol: TCP.
      • Source: Load balancer healthchecks.
    • Transferring service traffic between the master and the nodes:
      • Port range: 0-65535.
      • Protocol: Any (Any).
      • Source: Security group.
      • Security group: Current (Self).
    • Health checks of nodes using ICMP requests from subnets within Yandex Cloud:
      • Protocol: ICMP.
      • Source: CIDR.
      • CIDR blocks: Address ranges of subnets within Yandex Cloud to health check the cluster from, e.g.:
        • 10.0.0.0/8.
        • 192.168.0.0/16.
        • 172.16.0.0/12.

  2. Add a rule for outgoing service traffic between the master and the nodes:

    • Port range: 0-65535.
    • Protocol: Any (Any).
    • Destination name: Security group.
    • Security group: Current (Self).

Allowing traffic for node groups

For node groups to run properly, create rules for incoming and outgoing traffic and apply them to node groups:

  1. Add a rule for incoming traffic that allows traffic transfer between pods and services:

    • Port range: 0-65535.
    • Protocol: Any (Any).
    • Source: CIDR.
    • CIDR blocks: CIDRs of the cluster and services, e.g., 10.96.0.0/16 and 10.112.0.0/16.

  2. Add a rule for outgoing traffic that allows node group nodes to connect to external resources, e.g., to download images from Docker Hub or work with Yandex Object Storage:

    • Port range: 0-65535.
    • Protocol: Any (Any).
    • Destination name: CIDR.
    • CIDR blocks: 0.0.0.0/0.

Allowing traffic for cluster

For the cluster to work correctly and to allow incoming connections, create rules for incoming and outgoing traffic and apply them to the cluster:

  1. Add rules for incoming traffic that allow connecting to the master via ports 6443 and 443. This will allow you to access the Kubernetes API and manage the cluster using kubectl and other utilities.

    Create two rules for incoming traffic, one per port:

    • Port range: 443, 6443.
    • Protocol: TCP.
    • Source: CIDR.
    • CIDR blocks: Specify the IP address range of the subnets you will manage the cluster from, e.g.:
      • 85.23.23.22/32: For an external network.
      • 192.168.0.0/24: For an internal network.

  2. Add a rule for outgoing traffic that allows traffic transfer between the master and metric-server pods:

    • Port range: 4443.
    • Protocol: TCP.
    • Destination name: CIDR.
    • CIDR blocks: Specify the cluster CIDR, e.g., 10.96.0.0/16.

Creating a rule for connecting to services from the internet

To be sure that the services running on nodes are accessible from the internet and subnets within Yandex Cloud, create a rule for the incoming traffic and apply it to the node group:

  • Port range: 30000-32767.
  • Protocol: TCP.
  • Source: CIDR.
  • CIDR blocks: 0.0.0.0/0.

Creating a rule for connecting to nodes via SSH

To access the nodes via SSH, create a rule for incoming traffic and apply it to the node group:

  • Port range: 22.
  • Protocol: TCP.
  • Source: CIDR.
  • CIDR blocks: IP address ranges of subnets within Yandex Cloud and public IP addresses of computers on the internet, for example:
    • 10.0.0.0/8.
    • 192.168.0.0/16.
    • 172.16.0.0/12.
    • 85.32.32.22/32.

Applying security groups with rules

Depending on the rules the security groups contain, these groups must be assigned to a cluster or node group:

Security group Object to assign the group to
Group allowing traffic for a cluster and node groups Cluster and node group
Group allowing traffic for node groups Node group
Group allowing traffic for a cluster Cluster
Group allowing connection to services from the internet Node group
Group allowing connection to nodes via SSH Node group

Examples of rules

Let’s assume you want to create a Kubernetes cluster which:

To create such a cluster:

  1. Create security groups with the required rules.

    Configuration file with security groups
    resource "yandex_vpc_security_group" "k8s-cluster-nodegroup-traffic" {
  name        = "k8s-cluster-nodegroup-traffic"
  description = "The group rules allow service traffic for the cluster and node groups. Apply it to the cluster and node groups."
  network_id  = "<cloud_network_ID>"
  ingress {
    description       = "Rule for health checks of the network load balancer."
    from_port         = 0
    to_port           = 65535
    protocol          = "TCP"
    predefined_target = "loadbalancer_healthchecks"
  }
  ingress {
    description       = "Rule for incoming service traffic between the master and the nodes."
    from_port         = 0
    to_port           = 65535
    protocol          = "ANY"
    predefined_target = "self_security_group"
  }
  ingress {
    description    = "Rule for health checks of nodes using ICMP requests from subnets within Yandex Cloud."
    protocol       = "ICMP"
    v4_cidr_blocks = ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]
  }
  egress {
    description       = "Rule for outgoing service traffic between the master and the nodes."
    from_port         = 0
    to_port           = 65535
    protocol          = "ANY"
    predefined_target = "self_security_group"
  }
}

resource "yandex_vpc_security_group" "k8s-nodegroup-traffic" {
  name        = "k8s-nodegroup-traffic"
  description = "The group rules allow service traffic for node groups. Apply it to node groups."
  network_id  = "<cloud_network_ID>"
  ingress {
    description    = "Rule for incoming traffic that allows traffic transfer between pods and services."
    from_port      = 0
    to_port        = 65535
    protocol       = "ANY"
    v4_cidr_blocks = ["10.96.0.0/16", "10.112.0.0/16"]
  }
  egress {
    description    = "Rule for outgoing traffic that allows node group nodes to connect to external resources."
    from_port      = 0
    to_port        = 65535
    protocol       = "ANY"
    v4_cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "yandex_vpc_security_group" "k8s-services-access" {
  name        = "k8s-services-access"
  description = "The group rules allow connections to services from the internet. Apply it to node groups."
  network_id  = "<cloud_network_ID>"
  ingress {
    description    = "Rule for incoming traffic that allows connection to services."
    from_port      = 30000
    to_port        = 32767
    protocol       = "TCP"
    v4_cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "yandex_vpc_security_group" "k8s-ssh-access" {
  name        = "k8s-ssh-access"
  description = "The group rules allow connection to nodes via SSH. Apply it to node groups."
  network_id  = "<cloud_network_ID>"
  ingress {
    description    = "Rule for incoming traffic that allows connection to nodes via SSH."
    port           = 22
    protocol       = "TCP"
    v4_cidr_blocks = ["85.32.32.22/32"]
  }
}

resource "yandex_vpc_security_group" "k8s-cluster-traffic" {
  name        = "k8s-cluster-traffic"
  description = "The group rules allow traffic for the cluster. Apply it to the cluster."
  network_id  = "<cloud_network_ID>"
  ingress {
    description    = "Rule for incoming traffic that allows access to the Kubernetes API (port 443)."
    port           = 443
    protocol       = "TCP"
    v4_cidr_blocks = ["203.0.113.0/24"]
  }
  ingress {
    description    = "Rule for incoming traffic that allows access to the Kubernetes API (port 6443)."
    port           = 6443
    protocol       = "TCP"
    v4_cidr_blocks = ["203.0.113.0/24"]
  }
  egress {
    description    = "Rule for outgoing traffic that allows traffic transfer between the master and metric-server pods."
    port           = 4443
    protocol       = "TCP"
    v4_cidr_blocks = ["10.96.0.0/16"]
  }
}

  2. Create a cluster and a node group.

    Apply security groups with rules as follows:

    resource "yandex_kubernetes_cluster" "k8s-cluster" {
  name = "k8s-cluster"
  service_account_id = "<service_account_ID_for_cluster>"
  node_service_account_id = "<service_account_ID_for_node_groups>"
  cluster_ipv4_range = "10.96.0.0/16"
  service_ipv4_range = "10.112.0.0/16"
  network_id = "<cloud_network_ID>"
  master {
    master_location {
      zone      = "ru-central1-a"
      subnet_id = "<cloud_subnet_ID_in_zone>"
    }
    master_location {
      zone      = "ru-central1-b"
      subnet_id = "<cloud_subnet_ID_in_zone>"
    }
    master_location {
      zone      = "ru-central1-d"
      subnet_id = "<cloud_subnet_ID_in_zone>"
    }
    security_group_ids = [
      yandex_vpc_security_group.k8s-cluster-nodegroup-traffic.id,
      yandex_vpc_security_group.k8s-cluster-traffic.id
    ]
    public_ip = true
  }
}

resource "yandex_kubernetes_node_group" "worker-nodes-a" {
  cluster_id = yandex_kubernetes_cluster.k8s-cluster.id
  name       = "worker-nodes-a"
  allocation_policy {
    location {
      zone = "ru-central1-a"
    }
  }
  scale_policy {
    fixed_scale {
      size = 1
    }
  }
  instance_template {
    network_interface {
      nat                = true
      subnet_ids         = [<cloud_subnet_ID>]
      security_group_ids = [
        yandex_vpc_security_group.k8s-cluster-nodegroup-traffic.id,
        yandex_vpc_security_group.k8s-nodegroup-traffic.id,
        yandex_vpc_security_group.k8s-services-access.id,
        yandex_vpc_security_group.k8s-ssh-access.id
      ]
    }
  }
}
Previous
Connection method overview
Next
Creating a static configuration file