Creating a Managed Service for Kubernetes cluster

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameters.

To create a Managed Service for Kubernetes cluster:

1. Specify the Managed Service for Kubernetes cluster parameters in the create command (not all parameters are given in the example):

   yc managed-kubernetes cluster create \
     --name test-k8s \
     --network-name default \
     --public-ip \
     --release-channel regular \
     --version 1.27 \
     --cluster-ipv4-range 10.1.0.0/16 \
     --service-ipv4-range 10.2.0.0/16 \
     --security-group-ids enpe5sdn7vs5********,enpj6c5ifh75******** \
     --service-account-name default-sa \
     --node-service-account-name default-sa \
     --master-location zone=ru-central1-a,subnet-id=mysubnet \
     --daily-maintenance-window start=22:00,duration=10h
     --labels <cloud_label_name=cloud_label_value>
   

   Where:

   • --name: Managed Service for Kubernetes cluster name.

   • --network-name: Network name.

     Note

     If you select a cloud network from another folder, assign the resource service account the following roles in that folder:

     • vpc.privateAdmin
     • vpc.user
     • vpc.bridgeAdmin

     To use a public IP address, also assign the vpc.publicAdmin role.

   • --public-ip: Flag indicating that the Managed Service for Kubernetes cluster needs a public IP address.

     Warning

     Do not place a cluster with a public IP address in subnets with internet access via a NAT instance. With this configuration in place, your request to the cluster’s public IP address will get a response from the NAT instance’s IP address, and the client will reject it. For more information, see Route priority in complex scenarios.

     You will not be able to edit this setting once you create a cluster.

   • --release-channel: Release channel.

     You will not be able to edit this setting once you create a cluster.

   • --version: Kubernetes version. Specify a version available for the selected release channel.

   • --cluster-ipv4-range: Range of IP addresses for allocating pod addresses.

   • --service-ipv4-range: Range of IP addresses for allocating service addresses.

   • --security-group-ids: List of Managed Service for Kubernetes cluster security group IDs.

     Warning

     The configuration of security groups determines cluster performance, availability, and services running in the cluster.

   • --service-account-id: Unique ID of the service account for the resources. This service account will be used to create the resources required for the Managed Service for Kubernetes cluster.

   • --node-service-account-id: Unique ID of the service account for the nodes. Nodes will pull the required Docker images from the registry on behalf of this account.

   • --master-location: Master configuration. Specify in the parameter the availability zone and subnet where the master will be located.

     The number of --master-location parameters depends on the type of master:

     • For the basic master, provide one --master-location parameter.
     • For a highly available master hosted across three availability zones, provide three --master-location parameters. In each one, specify different availability zones and subnets.
     • For a highly available master hosted in a single availability zone, provide three --master-location parameters. In each one, specify the same availability zone and subnet.

   • --daily-maintenance-window: Maintenance window settings.

   • --labels: Cloud labels for the cluster.

   Result:

   done (5m47s)
   id: cathn0s6qobf********
   folder_id: b1g66jflru0e********
   ...
     service_account_id: aje3932acd0c********
     node_service_account_id: aje3932acd0c********
     release_channel: REGULAR
   

2. Configure the cluster’s Container Network Interface:

   You will not be able to edit this setting once you create a cluster.

   Warning

   You cannot enable the Calico network policy controller and the Cilium tunnel mode at the same time.

   • To enable the Calico network policy controller, set the --enable-network-policy flag in the Managed Service for Kubernetes cluster create command:

     yc managed-kubernetes cluster create \
     ...
       --enable-network-policy
     

   • To enable the Cilium tunnel mode, provide the --cilium flag in the Managed Service for Kubernetes cluster create command:

     yc managed-kubernetes cluster create \
     ...
       --cilium
     

3. To use the Yandex Key Management Service encryption key for protecting sensitive data, provide the key name or ID in the Managed Service for Kubernetes cluster creation command:

   yc managed-kubernetes cluster create \
   ...
     --kms-key-name <encryption_key_name> \
     --kms-key-id <encryption_key_ID>
   

   You will not be able to edit this setting once you create a cluster.

4. To enable sending logs to Yandex Cloud Logging, provide the logging settings in the --master-logging property of the Managed Service for Kubernetes cluster create command:

   yc managed-kubernetes cluster create \
   ...
     --master-logging enabled=<send_logs>,`
       `log-group-id=<log_group_ID>,`
       `folder-id=<folder_ID>,`
       `kube-apiserver-enabled=<send_kube-apiserver_logs>,`
       `cluster-autoscaler-enabled=<send_cluster-autoscaler_logs>,`
       `events-enabled=<send_Kubernetes_events>`
       `audit-enabled=<send_audit_events>
   

   Where:

   • enabled: Flag that enables log sending, true or false.
   • log-group-id: ID of the log group to send the logs to.
   • folder-id: ID of the folder to send the logs to. The logs will be sent to the log group of the default folder.
   • kube-apiserver-enabled: Flag that enables kube-apiserver log sending, true or false.
   • cluster-autoscaler-enabled: Flag that enables cluster-autoscaler log sending, true or false.
   • events-enabled: Flag that enables Kubernetes event sending, true or false.
   • audit-enabled: Flag that enables audit event sending, true or false.

   If log sending is enabled but neither log-group-id nor folder-id is specified, the logs will be sent to the default log group of the folder with the Managed Service for Kubernetes cluster. You cannot set both log-group-id and folder-id at the same time.

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

Terraform is distributed under the Business Source License. The Yandex Cloud provider for Terraform is distributed under the MPL-2.0 license.

For more information about the provider resources, see the documentation on the Terraform website or mirror website.

If you don't have Terraform, install it and configure the Yandex Cloud provider.

To create a Managed Service for Kubernetes cluster:

1. In the configuration file, define the parameters of the resources you want to create:

   • Managed Service for Kubernetes cluster: Cluster description.

   • Network: Description of the cloud network to host the Managed Service for Kubernetes cluster. If you already have a suitable network, you do not need to describe it again.

     Note

     If you select a cloud network from another folder, assign the resource service account the following roles in that folder:

     • vpc.privateAdmin
     • vpc.user
     • vpc.bridgeAdmin

     To use a public IP address, also assign the vpc.publicAdmin role.

   • Subnets: Description of the subnets to connect the Managed Service for Kubernetes cluster hosts to. If you already have suitable subnets, you do not need to describe them again.

   • Service account for the Managed Service for Kubernetes cluster and nodes and role settings for this account. Create separate service accounts for the Managed Service for Kubernetes cluster and nodes, as required. If you already have a suitable service account, you do not need to describe it again.

   Here is the configuration file example:

   resource "yandex_kubernetes_cluster" "<Managed_Service_for_Kubernetes_cluster_name>" {
    network_id = yandex_vpc_network.<network_name>.id
    master {
      master_location {
        zone      = yandex_vpc_subnet.<subnet_name>.zone
        subnet_id = yandex_vpc_subnet.<subnet_name>.id
      }
    }
    service_account_id      = yandex_iam_service_account.<service_account_name>.id
    node_service_account_id = yandex_iam_service_account.<service_account_name>.id
      depends_on = [
        yandex_resourcemanager_folder_iam_member.k8s-clusters-agent,
        yandex_resourcemanager_folder_iam_member.vpc-public-admin,
        yandex_resourcemanager_folder_iam_member.images-puller
      ]
   }
    labels {
      "<cloud_label_name>"="<cloud_label_value>"
    }
   resource "yandex_vpc_network" "<network_name>" { name = "<network_name>" }
   
   resource "yandex_vpc_subnet" "<subnet_name>" {
    v4_cidr_blocks = ["<subnet_IP_address_range>"]
    zone           = "<availability_zone>"
    network_id     = yandex_vpc_network.<network_name>.id
   }
   
   resource "yandex_iam_service_account" "<service_account_name>" {
    name        = "<service_account_name>"
    description = "<service_account_description>"
   }
   
   resource "yandex_resourcemanager_folder_iam_member" "k8s-clusters-agent" {
    # The service account gets the "k8s.clusters.agent" role.
    folder_id = "<folder_ID>"
    role      = "k8s.clusters.agent"
    member    = "serviceAccount:${yandex_iam_service_account.<service_account_name>.id}"
   }
   
   resource "yandex_resourcemanager_folder_iam_member" "vpc-public-admin" {
    # The service account gets the "vpc.publicAdmin" role.
    folder_id = "<folder_ID>"
    role      = "vpc.publicAdmin"
    member    = "serviceAccount:${yandex_iam_service_account.<service_account_name>.id}"
   }
   
   resource "yandex_resourcemanager_folder_iam_member" "images-puller" {
    # The service account gets the "container-registry.images.puller" role.
    folder_id = "<folder_ID>"
    role      = "container-registry.images.puller"
    member    = "serviceAccount:${yandex_iam_service_account.<service_account_name>.id}"
   }
   

   Note

   Cloud labels for a Kubernetes cluster are composed according to certain rules.

   To enable sending logs to Yandex Cloud Logging, add the master_logging section to the Managed Service for Kubernetes cluster description:

   resource "yandex_kubernetes_cluster" "<cluster_name>" {
    ...
    master {
      ...
      master_logging {
        enabled                    = <log_sending>
        log_group_id               = "<log_group_ID>"
        folder_id                  = "<folder_ID>"
        kube_apiserver_enabled     = <kube-apiserver_log_sending>
        cluster_autoscaler_enabled = <cluster-autoscaler_log_sending>
        events_enabled             = <Kubernetes_event_sending>
        audit_enabled              = <audit_event_sending>
      }
    }
   }
   

   Where:

   • enabled: Flag that enables log sending, true or false.
   • log_group_id: ID of the log group to send the logs to.
   • folder_id: ID of the folder to send the logs to. The logs will be sent to the log group of the default folder.
   • kube_apiserver_enabled: Flag that enables kube-apiserver log sending, true or false.
   • cluster_autoscaler_enabled: Flag that enables cluster-autoscaler log sending, true or false.
   • events_enabled: Flag that enables Kubernetes event sending, true or false.
   • audit_enabled: Flag that enables audit event sending, true or false.

   If log sending is enabled but neither log_group_id nor folder_id is specified, the logs will be sent to the default log group of the folder with the Managed Service for Kubernetes cluster. You cannot set both log_group_id and folder_id at the same time.

   For more information, see the Terraform provider documentation.

2. Make sure the configuration files are correct.

   1. In the command line, go to the folder where you created the configuration file.

   2. Run a check using this command:

      terraform plan
      

   If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out. This is a test step; no resources will be created.

3. Create a Managed Service for Kubernetes cluster.

   1. If the configuration does not contain any errors, run this command:

      terraform apply
      

   2. Confirm that you want to create the resources.

   After this, all required resources will be created in the specified folder and the IP addresses of the VMs will be displayed in the terminal. You can check the new resources and their configuration using the management console.

To create a Managed Service for Kubernetes cluster, use the create method for the Cluster resource.

The request body depends on the master type:

• For the basic master, provide one masterSpec.locations parameter in the request.
• For a highly available master hosted across three availability zones, provide three masterSpec.locations parameters in the request. In each one, specify different availability zones and subnets.
• For a highly available master hosted in a single availability zone, provide three masterSpec.locations parameters in the request. In each one, specify the same availability zone and subnet.

When providing the masterSpec.locations parameter, you do not need to specify masterSpec.zonalMasterSpec or masterSpec.regionalMasterSpec.

To use a Yandex Key Management Service encryption key to protect secrets, provide its ID in the kmsProvider.keyId parameter.

To enable sending logs to Yandex Cloud Logging, provide the logging settings in the masterSpec.masterLogging parameter.

To add a cloud label, provide its name and value in the labels parameter.

Create a Managed Service for Kubernetes cluster and a network for it with the following test specifications:

• Name: k8s-single.

• Folder ID: b1gia87mbaomkfvsleds.

• Network: mynet.

• Subnet: mysubnet. Its network settings are as follows:

  • Availability zone: ru-central1-a.
  • Range: 10.1.0.0/16.

• Service account: myaccount.

• Service account roles: k8s.clusters.agent, vpc.publicAdmin, container-registry.images.puller, and kms.keys.encrypterDecrypter.

• Yandex Key Management Service encryption key: kms-key.

• Security group: k8s-public-services. It contains rules for connecting to services from the internet.

Install Terraform (unless you already have it), configure the provider according to this guide, and apply the configuration file:

locals {
  folder_id   = "b1gia87mbaomkfvsleds"
}

resource "yandex_kubernetes_cluster" "k8s-single" {
  name = "k8s-single"
  network_id = yandex_vpc_network.mynet.id
  master {
    master_location {
      zone      = yandex_vpc_subnet.mysubnet.zone
      subnet_id = yandex_vpc_subnet.mysubnet.id
    }
    security_group_ids = [yandex_vpc_security_group.k8s-public-services.id]
  }
  service_account_id      = yandex_iam_service_account.myaccount.id
  node_service_account_id = yandex_iam_service_account.myaccount.id
  depends_on = [
    yandex_resourcemanager_folder_iam_member.k8s-clusters-agent,
    yandex_resourcemanager_folder_iam_member.vpc-public-admin,
    yandex_resourcemanager_folder_iam_member.images-puller,
    yandex_resourcemanager_folder_iam_member.encrypterDecrypter
  ]
  kms_provider {
    key_id = yandex_kms_symmetric_key.kms-key.id
  }
}

resource "yandex_vpc_network" "mynet" {
  name = "mynet"
}

resource "yandex_vpc_subnet" "mysubnet" {
  name = "mysubnet"
  v4_cidr_blocks = ["10.1.0.0/16"]
  zone           = "ru-central1-a"
  network_id     = yandex_vpc_network.mynet.id
}

resource "yandex_iam_service_account" "myaccount" {
  name        = "myaccount"
  description = "Service account for the single Kubernetes cluster"
}

resource "yandex_resourcemanager_folder_iam_member" "k8s-clusters-agent" {
  # The service account gets the "k8s.clusters.agent" role.
  folder_id = local.folder_id
  role      = "k8s.clusters.agent"
  member    = "serviceAccount:${yandex_iam_service_account.myaccount.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "vpc-public-admin" {
  # The service account gets the "vpc.publicAdmin" role.
  folder_id = local.folder_id
  role      = "vpc.publicAdmin"
  member    = "serviceAccount:${yandex_iam_service_account.myaccount.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "images-puller" {
  # The service account gets the "container-registry.images.puller" role.
  folder_id = local.folder_id
  role      = "container-registry.images.puller"
  member    = "serviceAccount:${yandex_iam_service_account.myaccount.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "encrypterDecrypter" {
  # The service account gets the "kms.keys.encrypterDecrypter" role.
  folder_id = local.folder_id
  role      = "kms.keys.encrypterDecrypter"
  member    = "serviceAccount:${yandex_iam_service_account.myaccount.id}"
}

resource "yandex_kms_symmetric_key" "kms-key" {
  # A Yandex Key Management Service key for encrypting critical information, including passwords, OAuth tokens, and SSH keys.
  name              = "kms-key"
  default_algorithm = "AES_128"
  rotation_period   = "8760h" # 1 year.
}

resource "yandex_vpc_security_group" "k8s-public-services" {
  name        = "k8s-public-services"
  description = "Group rules allow connections to services from the internet. Apply the rules for node groups only."
  network_id  = yandex_vpc_network.mynet.id
  ingress {
    protocol          = "TCP"
    description       = "The rule allows availability checks from the load balancer's range of addresses. It is required for the operation of a fault-tolerant Managed Service for Kubernetes cluster and load balancer services."
    predefined_target = "loadbalancer_healthchecks"
    from_port         = 0
    to_port           = 65535
  }
  ingress {
    protocol          = "ANY"
    description       = "The rule allows master-to-node and node-to-node communication inside a security group."
    predefined_target = "self_security_group"
    from_port         = 0
    to_port           = 65535
  }
  ingress {
    protocol          = "ANY"
    description       = "The rule allows sub-sub and service-service interactions. Specify the subnets of your Managed Service for Kubernetes cluster and services."
    v4_cidr_blocks    = concat(yandex_vpc_subnet.mysubnet.v4_cidr_blocks)
    from_port         = 0
    to_port           = 65535
  }
  ingress {
    protocol          = "ICMP"
    description       = "The rule allows debug ICMP packets from internal subnets."
    v4_cidr_blocks    = ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]
  }
  ingress {
    protocol          = "TCP"
    description       = "The rule allows incoming traffic from the internet to a range of NodePorts. Add ports or change existing ones to the required ports."
    v4_cidr_blocks    = ["0.0.0.0/0"]
    from_port         = 30000
    to_port           = 32767
  }
  egress {
    protocol          = "ANY"
    description       = "The rule allows all outgoing traffic. Nodes can connect to Yandex Container Registry, Yandex Object Storage, Docker Hub, etc."
    v4_cidr_blocks    = ["0.0.0.0/0"]
    from_port         = 0
    to_port           = 65535
  }
}

Create a Managed Service for Kubernetes cluster and a network for it with the following test specifications:

• Name: k8s-ha-three-zones.

• Folder ID: b1gia87mbaomkfvsleds

• Network: my-ha-net.

• Subnet: mysubnet-a. Its network settings are as follows:

  • Availability zone: ru-central1-a.
  • Range: 10.5.0.0/16.

• Subnet: mysubnet-b. Its network settings are as follows:

  • Availability zone: ru-central1-b.
  • Range: 10.6.0.0/16.

• Subnet: mysubnet-d. Its network settings are as follows:

  • Availability zone: ru-central1-d.
  • Range: 10.7.0.0/16.

• Service account: ha-k8s-account.

• Service account roles: k8s.clusters.agent, vpc.publicAdmin, container-registry.images.puller, and kms.keys.encrypterDecrypter.

• Yandex Key Management Service encryption key: kms-key.

• Security group: regional-k8s-sg. It contains rules for service traffic.

Install Terraform (unless you already have it), configure the provider according to this guide, and apply the configuration file:

locals {
  folder_id   = "b1gia87mbaomkfvsleds"
}

resource "yandex_kubernetes_cluster" "k8s-ha-three-zones" {
  name = "k8s-ha-three-zones"
  network_id = yandex_vpc_network.my-ha-net.id
  master {
    master_location {
      zone      = yandex_vpc_subnet.mysubnet-a.zone
      subnet_id = yandex_vpc_subnet.mysubnet-a.id
    }
    master_location {
      zone      = yandex_vpc_subnet.mysubnet-b.zone
      subnet_id = yandex_vpc_subnet.mysubnet-b.id
    }
    master_location {
      zone      = yandex_vpc_subnet.mysubnet-d.zone
      subnet_id = yandex_vpc_subnet.mysubnet-d.id
    }
    security_group_ids = [yandex_vpc_security_group.ha-k8s-sg.id]
  }
  service_account_id      = yandex_iam_service_account.ha-k8s-account.id
  node_service_account_id = yandex_iam_service_account.ha-k8s-account.id
  depends_on = [
    yandex_resourcemanager_folder_iam_member.k8s-clusters-agent,
    yandex_resourcemanager_folder_iam_member.vpc-public-admin,
    yandex_resourcemanager_folder_iam_member.images-puller,
    yandex_resourcemanager_folder_iam_member.encrypterDecrypter
  ]
  kms_provider {
    key_id = yandex_kms_symmetric_key.kms-key.id
  }
}

resource "yandex_vpc_network" "my-ha-net" {
  name = "my-ha-net"
}

resource "yandex_vpc_subnet" "mysubnet-a" {
  name = "mysubnet-a"
  v4_cidr_blocks = ["10.5.0.0/16"]
  zone           = "ru-central1-a"
  network_id     = yandex_vpc_network.my-ha-net.id
}

resource "yandex_vpc_subnet" "mysubnet-b" {
  name = "mysubnet-b"
  v4_cidr_blocks = ["10.6.0.0/16"]
  zone           = "ru-central1-b"
  network_id     = yandex_vpc_network.my-ha-net.id
}

resource "yandex_vpc_subnet" "mysubnet-d" {
  name = "mysubnet-d"
  v4_cidr_blocks = ["10.7.0.0/16"]
  zone           = "ru-central1-d"
  network_id     = yandex_vpc_network.my-ha-net.id
}

resource "yandex_iam_service_account" "ha-k8s-account" {
  name        = "ha-k8s-account"
  description = "Service account for the highly available Kubernetes cluster"
}

resource "yandex_resourcemanager_folder_iam_member" "k8s-clusters-agent" {
  # The service account gets the "k8s.clusters.agent" role.
  folder_id = local.folder_id
  role      = "k8s.clusters.agent"
  member    = "serviceAccount:${yandex_iam_service_account.ha-k8s-account.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "vpc-public-admin" {
  # The service account gets the "vpc.publicAdmin" role.
  folder_id = local.folder_id
  role      = "vpc.publicAdmin"
  member    = "serviceAccount:${yandex_iam_service_account.ha-k8s-account.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "images-puller" {
  # The service account gets the "container-registry.images.puller" role.
  folder_id = local.folder_id
  role      = "container-registry.images.puller"
  member    = "serviceAccount:${yandex_iam_service_account.ha-k8s-account.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "encrypterDecrypter" {
  # The service account gets the "kms.keys.encrypterDecrypter" role.
  folder_id = local.folder_id
  role      = "kms.keys.encrypterDecrypter"
  member    = "serviceAccount:${yandex_iam_service_account.ha-k8s-account.id}"
}

resource "yandex_kms_symmetric_key" "kms-key" {
  # A Yandex Key Management Service key for encrypting critical information, including passwords, OAuth tokens, and SSH keys.
  name              = "kms-key"
  default_algorithm = "AES_128"
  rotation_period   = "8760h" # 1 year.
}

resource "yandex_vpc_security_group" "ha-k8s-sg" {
  name        = "ha-k8s-sg"
  description = "Group rules ensure the basic performance of the Managed Service for Kubernetes cluster. Apply it to the cluster and node groups."
  network_id  = yandex_vpc_network.my-ha-net.id
  ingress {
    protocol          = "TCP"
    description       = "The rule allows availability checks from the load balancer's range of addresses. It is required for the operation of a fault-tolerant Managed Service for Kubernetes cluster and load balancer services."
    predefined_target = "loadbalancer_healthchecks"
    from_port         = 0
    to_port           = 65535
  }
  ingress {
    protocol          = "ANY"
    description       = "The rule allows master-to-node and node-to-node communication inside a security group."
    predefined_target = "self_security_group"
    from_port         = 0
    to_port           = 65535
  }
  ingress {
    protocol          = "ANY"
    description       = "The rule allows sub-sub and service-service interactions. Specify the subnets of your Managed Service for Kubernetes cluster and services."
    v4_cidr_blocks    = concat(yandex_vpc_subnet.mysubnet-a.v4_cidr_blocks, yandex_vpc_subnet.mysubnet-b.v4_cidr_blocks, yandex_vpc_subnet.mysubnet-d.v4_cidr_blocks)
    from_port         = 0
    to_port           = 65535
  }
  ingress {
    protocol          = "ICMP"
    description       = "The rule allows debug ICMP packets from internal subnets."
    v4_cidr_blocks    = ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]
  }
  ingress {
    protocol          = "TCP"
    description       = "The rule allows incoming traffic from the internet to a range of NodePorts. Add ports or change existing ones to the required ports."
    v4_cidr_blocks    = ["0.0.0.0/0"]
    from_port         = 30000
    to_port           = 32767
  }
  egress {
    protocol          = "ANY"
    description       = "The rule allows all outgoing traffic. Nodes can connect to Yandex Container Registry, Yandex Object Storage, Docker Hub, etc."
    v4_cidr_blocks    = ["0.0.0.0/0"]
    from_port         = 0
    to_port           = 65535
  }
}

Create a Managed Service for Kubernetes cluster and a network for it with the following test specifications:

• Name: k8s-ha-one-zone.

• Folder ID: b1gia87mbaomkfvsleds.

• Network: my-ha-net.

• Subnet: my-ha-subnet. Its network settings are as follows:

  • Availability zone: ru-central1-a.
  • Range: 10.5.0.0/16.

• Service account: ha-k8s-account.

• Service account roles: k8s.clusters.agent, vpc.publicAdmin, container-registry.images.puller, and kms.keys.encrypterDecrypter.

• Yandex Key Management Service encryption key: kms-key.

• Security group: ha-k8s-sg. It contains rules for service traffic.

Install Terraform (unless you already have it), configure the provider according to this guide, and apply the configuration file:

locals {
  folder_id   = "b1gia87mbaomkfvsleds"
}

resource "yandex_kubernetes_cluster" "k8s-ha-one-zone" {
  name = "k8s-ha-one-zone"
  network_id = yandex_vpc_network.my-ha-net.id
  master {
    master_location {
      zone      = yandex_vpc_subnet.my-ha-subnet.zone
      subnet_id = yandex_vpc_subnet.my-ha-subnet.id
    }
    master_location {
      zone      = yandex_vpc_subnet.my-ha-subnet.zone
      subnet_id = yandex_vpc_subnet.my-ha-subnet.id
    }
    master_location {
      zone      = yandex_vpc_subnet.my-ha-subnet.zone
      subnet_id = yandex_vpc_subnet.my-ha-subnet.id
    }
    security_group_ids = [yandex_vpc_security_group.ha-k8s-sg.id]
  }
  service_account_id      = yandex_iam_service_account.ha-k8s-account.id
  node_service_account_id = yandex_iam_service_account.ha-k8s-account.id
  depends_on = [
    yandex_resourcemanager_folder_iam_member.k8s-clusters-agent,
    yandex_resourcemanager_folder_iam_member.vpc-public-admin,
    yandex_resourcemanager_folder_iam_member.images-puller,
    yandex_resourcemanager_folder_iam_member.encrypterDecrypter
  ]
  kms_provider {
    key_id = yandex_kms_symmetric_key.kms-key.id
  }
}

resource "yandex_vpc_network" "my-ha-net" {
  name = "my-ha-net"
}

resource "yandex_vpc_subnet" "my-ha-subnet" {
  name = "my-ha-subnet"
  v4_cidr_blocks = ["10.5.0.0/16"]
  zone           = "ru-central1-a"
  network_id     = yandex_vpc_network.my-ha-net.id
}

resource "yandex_iam_service_account" "ha-k8s-account" {
  name        = "ha-k8s-account"
  description = "Service account for the highly available Kubernetes cluster"
}

resource "yandex_resourcemanager_folder_iam_member" "k8s-clusters-agent" {
  # The service account gets the "k8s.clusters.agent" role.
  folder_id = local.folder_id
  role      = "k8s.clusters.agent"
  member    = "serviceAccount:${yandex_iam_service_account.ha-k8s-account.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "vpc-public-admin" {
  # The service account gets the "vpc.publicAdmin" role.
  folder_id = local.folder_id
  role      = "vpc.publicAdmin"
  member    = "serviceAccount:${yandex_iam_service_account.ha-k8s-account.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "images-puller" {
  # The service account gets the "container-registry.images.puller" role.
  folder_id = local.folder_id
  role      = "container-registry.images.puller"
  member    = "serviceAccount:${yandex_iam_service_account.ha-k8s-account.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "encrypterDecrypter" {
  # The service account gets the "kms.keys.encrypterDecrypter" role.
  folder_id = local.folder_id
  role      = "kms.keys.encrypterDecrypter"
  member    = "serviceAccount:${yandex_iam_service_account.ha-k8s-account.id}"
}

resource "yandex_kms_symmetric_key" "kms-key" {
  # A Yandex Key Management Service key for encrypting critical information, including passwords, OAuth tokens, and SSH keys.
  name              = "kms-key"
  default_algorithm = "AES_128"
  rotation_period   = "8760h" # 1 year.
}

resource "yandex_vpc_security_group" "ha-k8s-sg" {
  name        = "ha-k8s-sg"
  description = "Group rules ensure the basic performance of the Managed Service for Kubernetes cluster. Apply it to the cluster and node groups."
  network_id  = yandex_vpc_network.my-ha-net.id
  ingress {
    protocol          = "TCP"
    description       = "The rule allows availability checks from the load balancer's range of addresses. It is required for the operation of a fault-tolerant Managed Service for Kubernetes cluster and load balancer services."
    predefined_target = "loadbalancer_healthchecks"
    from_port         = 0
    to_port           = 65535
  }
  ingress {
    protocol          = "ANY"
    description       = "The rule allows master-to-node and node-to-node communication inside a security group."
    predefined_target = "self_security_group"
    from_port         = 0
    to_port           = 65535
  }
  ingress {
    protocol          = "ANY"
    description       = "The rule allows sub-sub and service-service interactions. Specify the subnets of your Managed Service for Kubernetes cluster and services."
    v4_cidr_blocks    = concat(yandex_vpc_subnet.my-ha-subnet.v4_cidr_blocks, yandex_vpc_subnet.my-ha-subnet.v4_cidr_blocks, yandex_vpc_subnet.my-ha-subnet.v4_cidr_blocks)
    from_port         = 0
    to_port           = 65535
  }
  ingress {
    protocol          = "ICMP"
    description       = "The rule allows debug ICMP packets from internal subnets."
    v4_cidr_blocks    = ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]
  }
  ingress {
    protocol          = "TCP"
    description       = "The rule allows incoming traffic from the internet to a range of NodePorts. Add ports or change existing ones to the required ports."
    v4_cidr_blocks    = ["0.0.0.0/0"]
    from_port         = 30000
    to_port           = 32767
  }
  egress {
    protocol          = "ANY"
    description       = "The rule allows all outgoing traffic. Nodes can connect to Yandex Container Registry, Yandex Object Storage, Docker Hub, etc."
    v4_cidr_blocks    = ["0.0.0.0/0"]
    from_port         = 0
    to_port           = 65535
  }
}