Creating a node group

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

1. Check the description of the CLI command for creating a Managed Service for Kubernetes node group:

   yc managed-kubernetes node-group create --help
   

2. Specify Managed Service for Kubernetes node group parameters in the create command (we excluded some supported parameters from the example for brevity).

   yc managed-kubernetes node-group create \
     --allowed-unsafe-sysctls <names_of_unsafe_kernel_parameters> \
     --cluster-name <cluster_name> \
     --cores <number_of_vCPUs> \
     --core-fraction <guaranteed_vCPU_share> \
     --daily-maintenance-window <update_window_settings> \
     --disk-size <storage_size_in_GB> \
     --disk-type <storage_type> \
     --fixed-size <fixed_number_of_nodes_in_group> \
     --location zone=[<availability_zone>],subnet-id=[<subnet_ID>] \
     --memory <amount_of_RAM_in_GB> \
     --name <node_group_name> \
     --network-acceleration-type <network_acceleration_type> \
     --network-interface security-group-ids=[<security_group_IDs>],ipv4-address=<IP_address_assignment_method> \
     --platform-id <platform_ID> \
     --container-runtime containerd \
     --preemptible \
     --public-ip \
     --template-labels <cloud_label_key=cloud_label_value> \
     --node-labels <k8s_label_key=k8s_label_value>
     --version <Kubernetes_version_on_group_nodes> \
     --node-name <node_name_template> \
     --node-taints <taint_policies> \
     --container-network-settings pod-mtu=<MTU_value_for_group_pods>
   

   Where:

   • --allowed-unsafe-sysctls: Permission for Managed Service for Kubernetes group nodes to use unsafe kernel parameters, comma-separated.

   • --cluster-name: Name of the Managed Service for Kubernetes cluster to create the node group in.

   • --cores: Number of vCPUs for Managed Service for Kubernetes nodes.

   • --core-fraction: Guaranteed share of vCPUs for Managed Service for Kubernetes nodes.

   • --daily-maintenance-window: Maintenance window settings.

   • --disk-size: Disk size of the Managed Service for Kubernetes node.

   • --disk-type: Disk type of the Managed Service for Kubernetes node: network-nvme or network-hdd.

   • Type of scaling:

     • --fixed-size: Fixed number of nodes in a Managed Service for Kubernetes node group.

     • --auto-scale: Configuring Managed Service for Kubernetes cluster autoscaling:

       • min: Minimum number of nodes in the group.
       • max: Maximum number of nodes in the group.
       • initial: Initial number of nodes in the group.

     You cannot change the scaling type after you create a node group.

   • --location: Availability zone and subnet to host Managed Service for Kubernetes nodes. You can specify more than one option but only a single subnet per zone. Use a separate --location parameter for each of the availability zones.

     Warning

     You can place autoscaling group nodes only in one availability zone.

     If you provide --location, --network-interface, and --public-ip in the same command, you will get an error. It is enough to specify the location of a Managed Service for Kubernetes node group either in --location or --network-interface.

     To grant internet access to Managed Service for Kubernetes cluster nodes, do one of the following:

     • Assign a public IP address to the cluster nodes, specifying --network-interface ipv4-address=nat or --network-interface ipv6-address=nat.
     • Enable access to Managed Service for Kubernetes nodes from the internet after creating a node group.

   • --memory: Amount of memory allocated for Managed Service for Kubernetes nodes.

   • --name: Managed Service for Kubernetes node group name.

   • --network-acceleration-type: Change network acceleration type:

     • standard: No acceleration.
     • software-accelerated: Software-accelerated network.

     Warning

     Before activating a software-accelerated network, make sure that you have sufficient cloud resources available to create an additional Managed Service for Kubernetes node.

   • --network-interface: Network settings:

     • security-group-ids: IDs of Security groups.
     • subnets: Names of subnets that will host the nodes.
     • ipv4-address: Method of assigning an IPv4 address.
     • ipv6-address: Method of assigning an IPv6 address.

     ipv4-address and ipv6-address determine the method of assigning an IP address:

     • auto: Only the internal IP address is assigned to the node.
     • nat: Public and internal IP addresses are assigned to the node.

   • --platform-id: Managed Service for Kubernetes node platform.

   • --container-runtime: containerd runtime environment.

   • --preemptible: Flag you set for preemptible VMs.

   • --public-ip: Flag you set if the Managed Service for Kubernetes node group needs a public IP address.

   • --template-labels: Node group cloud labels. You can specify multiple labels separated by commas.

   • --node-labels: Node group Kubernetes labels.

   • --version: Kubernetes version on the Managed Service for Kubernetes group nodes.

   • --node-name: Managed Service for Kubernetes node name template. The name is unique if the template contains at least one of the following variables:

     • {instance_group.id}: Instance group ID.
     • {instance.index}: Unique instance number in the instance group. Possible values: 1 to N, where N is the number of instances in the group.
     • {instance.index_in_zone}: Instance number in a zone. It's unique for a specific instance group within the zone.
     • {instance.short_id}: Instance ID that is unique within the group. Consists of four letters.
     • {instance.zone_id}: Zone ID.

     For example, prod-{instance.short_id}-{instance_group.id}. If not specified, the default value is used: {instance_group.id}-{instance.short_id}.

   • --node-taints: Kubernetes taint policies. You can specify multiple values.

   • --container-network-settings: MTU value for network connections to group pods. This setting is not applicable for clusters with Calico or Cilium network policy controllers.

   Result:

   done (1m17s)
   id: catpl8c44kii********
   cluster_id: catcsqidoos7********
   ...
       start_time:
         hours: 22
       duration: 36000s
   

3. To add metadata for nodes, use the --metadata or --metadata-from-file parameter.

   Use metadata to configure the method of connecting to nodes in a node group. You can configure one method only because they are mutually exclusive.

   To connect to nodes in a node group, specify metadata for the selected connection method:

   • To connect to nodes via OS Login, add metadata with the enable-oslogin key set to true.

     For more on configuring and using OS Login, see Connecting to a node via OS Login.

   • To connect to nodes using SSH keys, add metadata with the ssh-keys key and its value listing the connection details.

     For more on preparing, configuring, and using SSH keys, see Connecting to a node over SSH.

   Warning

   Metadata settings can affect the behavior and health of the group's nodes. Change these settings only if you know exactly what you want to do.

   Providing user data in the metadata with the user-data key is not supported.

   Add metadata using one of the following methods:

   • Using --metadata, specify one or more key=value pairs separated by commas.

     The key value is provided explicitly.

   • Using --metadata-from-file, specify one or more key=path_to_file_with_value pairs separated by commas.

     The key value will be read from a file. This may be of use if the value is too long to provide it explicitly or contains line breaks or other special characters.

   You can change the metadata list after you create a cluster.

4. To specify a placement group for Managed Service for Kubernetes nodes:

   1. Get a list of placement groups using the yc compute placement-group list command.

   2. Provide a placement group name or ID in the --placement group flag when creating a Managed Service for Kubernetes node group:

      yc managed-kubernetes node-group create \
      ...
        --placement-group <placement_group_name_or_ID>
      

To create a Managed Service for Kubernetes node group:

1. In the folder containing the cluster description file, create a configuration file with the new Managed Service for Kubernetes node group's parameters.

   Here is an example of the configuration file structure:

   resource "yandex_kubernetes_node_group" "<node_group_name>" {
     cluster_id = yandex_kubernetes_cluster.<cluster_name>.id
     name       = "<node_group_name>"
     ...
     instance_template {
       name       = "<node_name_template>"
       platform_id = "<platform_for_nodes>"
       network_acceleration_type = "<network_acceleration_type>"
       container_runtime {
         type = "containerd"
       }
       labels {
         "<cloud_label_name>"="<cloud_label_value>"
       }
       node_labels {
         "<Kubernetes_label_name>"="<Kubernetes_label_value>"
       }
       ...
     }
     ...
     scale_policy {
       <node_group_scaling_settings>
     }
     ...
     allocation_policy {
       location {
         zone = "<availability_zone>"
       }
     }
   }
   

   Where:

   • cluster_id: Managed Service for Kubernetes cluster ID.

   • name: Managed Service for Kubernetes node group name.

   • instance_template: Managed Service for Kubernetes node parameters:

     • name: Managed Service for Kubernetes node name template. The name is unique if the template contains at least one of the following variables:

       • {instance_group.id}: Instance group ID.
       • {instance.index}: Unique instance number in the instance group. Possible values: 1 to N, where N is the number of instances in the group.
       • {instance.index_in_zone}: Instance number in a zone. It's unique for a specific instance group within the zone.
       • {instance.short_id}: Instance ID that is unique within the group. Consists of four letters.
       • {instance.zone_id}: Zone ID.

       For example, prod-{instance.short_id}-{instance_group.id}. If not specified, the default value is used: {instance_group.id}-{instance.short_id}.

     • platform_id: Managed Service for Kubernetes node platform.

     • network_acceleration_type: Network acceleration type:

       • standard: No acceleration.
       • software-accelerated: Software-accelerated network.

       Warning

       Before activating a software-accelerated network, make sure that you have sufficient cloud resources available to create an additional Managed Service for Kubernetes node.

     • container_runtime, type: containerd runtime environment.

     • labels: Node group cloud labels. You can specify multiple labels separated by commas.

     • node_labels: Node group Kubernetes labels.

     • scale_policy: Scaling settings.

       You cannot change the scaling type after you create a node group.

     • allocation_policy: Placement settings. These contain the location section with the zone parameter that represents the availability zone you want to place the group nodes in. You can place nodes of a group with the fixed scaling type in multiple availability zones. To do this, specify each of the availability zones you need in a separate location section.

       Warning

       You can place autoscaling group nodes only in one availability zone.

   • To create a node group with a constant number of nodes, add a fixed_scale section:

     resource "yandex_kubernetes_node_group" "<node_group_name>" {
       ...
       scale_policy {
         fixed_scale {
           size = <number_of_nodes_in_group>
         }
       }
     }
     

   • To create a Managed Service for Kubernetes node group with autoscaling, add an auto_scale section:

     resource "yandex_kubernetes_node_group" "<node_group_name>" {
       ...
       scale_policy {
         auto_scale {
           min     = <minimum_number_of_nodes_in_node_group>
           max     = <maximum_number_of_nodes_in_node_group>
           initial = <initial_number_of_nodes_in_node_group>
         }
       }
     }
     

   • To add metadata for nodes, provide it in the instance_template.metadata parameter.

     Use metadata to configure the method of connecting to nodes in a node group. You can configure one method only because they are mutually exclusive.

     To connect to nodes in a node group, specify metadata for the selected connection method:

     • To connect to nodes via OS Login, add metadata with the enable-oslogin key set to true.

       For more on configuring and using OS Login, see Connecting to a node via OS Login.

     • To connect to nodes using SSH keys, add metadata with the ssh-keys key and its value listing the connection details.

       For more on preparing, configuring, and using SSH keys, see Connecting to a node over SSH.

     Warning

     Metadata settings can affect the behavior and health of the group's nodes. Change these settings only if you know exactly what you want to do.

     Providing user data in the metadata with the user-data key is not supported.

     Add metadata using one of the following methods:

     • Specify one or more key=value pairs.

       The key value is provided explicitly.

     • Specify one or more key=file(path_to_file_with_value) pairs.

       The key value will be read from a file. This may be of use if the value is too long to provide it explicitly or contains line breaks or other special characters.

     resource "yandex_kubernetes_node_group" "<node_group_name>" {
       ...
       instance_template {
         metadata = {
           "key_1" = "value"
           "key_2" = file("<path_to_file_with_value>")
           ...
         }
         ...
       }
       ...
     }
     

     You can change the metadata list after you create a cluster.

   • To add DNS records:

     1. Create an internal or public DNS zone.

     2. Add the instance_template.network_interface.ipv4_dns_records section:

        resource "yandex_kubernetes_node_group" "<node_group_name>" {
          ...
          instance_template {
            network_interface {
              ipv4_dns_records {
                fqdn        = "<DNS_record_FQDN>"
                dns_zone_id = "<DNS_zone_ID>"
                ttl         = "<DNS_record_TTL_in_seconds>"
                ptr         = "<PTR_record_creation>"
              }
            }
          }
        }
        

        Where ptr denotes a PTR record creation: true or false.

        In a DNS record's FQDN, you can use a template with variables:

        • {instance_group.id}: Instance group ID.
        • {instance.index}: Unique instance number in the instance group. Possible values: 1 to N, where N is the number of instances in the group.
        • {instance.index_in_zone}: Instance number in a zone. It is unique for a specific instance group within a zone.
        • {instance.short_id}: Instance ID that is unique within the group. It consists of four alphabetic characters.
        • {instance.zone_id}: Zone ID.

   For more information, see the Terraform provider documentation.

2. Make sure the configuration files are correct.

   1. In the command line, go to the folder where you created the configuration file.

   2. Run a check using this command:

      terraform plan
      

   If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out. This is a test step; no resources will be created.

3. Create a Managed Service for Kubernetes node group.

   1. If the configuration does not contain any errors, run this command:

      terraform apply
      

   2. Confirm that you want to create the resources.

   After this, all required resources will be created in the specified folder and the IP addresses of the VMs will be displayed in the terminal. You can check the new resources and their configuration using the management console.

Use the create API method and include the following information in the request:

• Managed Service for Kubernetes cluster ID in the clusterId parameter. You can get it with the list of Managed Service for Kubernetes clusters in the folder.

• Managed Service for Kubernetes node group configuration in the nodeTemplate parameter.

• Network acceleration type in the nodeTemplate.networkSettings.type parameter.

  Warning

  Before activating a software-accelerated network, make sure that you have sufficient cloud resources available to create an additional Managed Service for Kubernetes node.

• containerd runtime environment in the nodeTemplate.containerRuntimeSettings.type parameter.

• Node group cloud labels in the nodeTemplate.labels parameter.

• Node group Kubernetes labels in the nodeLabels parameter.

• Scaling settings in the scalePolicy parameter.

  You cannot change the scaling type after you create a node group.

• Managed Service for Kubernetes node group placement settings in the allocationPolicy parameters.

  Warning

  You can place autoscaling group nodes only in one availability zone.

• Maintenance window settings in the maintenancePolicy parameters.

• List of settings being changed in the updateMask parameter.

  Warning

  The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the updateMask parameter as a single comma-separated string.

• For nodes to use non-replicated disks, provide the network-ssd-nonreplicated value for the nodeTemplate.bootDiskSpec.diskTypeId parameter.

  You can only change the size of non-replicated disks in 93 GB increments. The maximum size of this type of disk is 4 TB.

  Alert

  Non-replicated disks have no redundancy. If a disk fails, its data will be irretrievably lost. For more information, see Non-replicated disks and ultra high-speed network storages with three replicas (SSD).

• To enable Managed Service for Kubernetes group nodes to use unsafe kernel parameters, provide their names in the allowedUnsafeSysctls parameter.

• To set taint policies, provide their values in the nodeTaints parameter.

• To set a template for Managed Service for Kubernetes node names, provide it in the nodeTemplate.name parameter. The name is unique if the template contains at least one of the following variables:

  • {instance_group.id}: Instance group ID.
  • {instance.index}: Unique instance number in the instance group. Possible values: 1 to N, where N is the number of instances in the group.
  • {instance.index_in_zone}: Instance number in a zone. It's unique for a specific instance group within the zone.
  • {instance.short_id}: Instance ID that is unique within the group. Consists of four letters.
  • {instance.zone_id}: Zone ID.

  For example, prod-{instance.short_id}-{instance_group.id}. If not specified, the default value is used: {instance_group.id}-{instance.short_id}.

• To add metadata for nodes, provide it in the nodeTemplate.metadata parameter.

  Use metadata to configure the method of connecting to nodes in a node group. You can configure one method only because they are mutually exclusive.

  To connect to nodes in a node group, specify metadata for the selected connection method:

  • To connect to nodes via OS Login, add metadata with the enable-oslogin key set to true.

    For more on configuring and using OS Login, see Connecting to a node via OS Login.

  • To connect to nodes using SSH keys, add metadata with the ssh-keys key and its value listing the connection details.

    For more on preparing, configuring, and using SSH keys, see Connecting to a node over SSH.

  Warning

  Metadata settings can affect the behavior and health of the group's nodes. Change these settings only if you know exactly what you want to do.

  Providing user data in the metadata with the user-data key is not supported.

  Add metadata by specifying one or more key=value pairs separated by commas.

  The key value is provided explicitly.

  You can change the metadata list after you create a cluster.

• To add DNS records, provide their settings in the nodeTemplate.v4AddressSpec.dnsRecordSpecs parameter. In a DNS record's FQDN, you can use the nodeTemplate.name node name template with variables.

1. Place the node group configuration file in the same folder as the cluster description file.

   resource "yandex_kubernetes_node_group" "k8s-demo-ng" {
     name        = "k8s-demo-ng"
     description = "Test node group"
     cluster_id  = "cat0adul1fj0********"
     version     = "1.29"
     instance_template {
       name = "test-{instance.short_id}-{instance_group.id}"
       platform_id = "standard-v3"
       resources {
         cores         = 2
         core_fraction = 50
         memory        = 2
       }
       boot_disk {
         size = 64
         type = "network-ssd"
       }
       network_acceleration_type = "standard"
       network_interface {
         security_group_ids = ["enp6saqnq4ie244g67sb"]
         subnet_ids         = ["e9bj3s90g9hm********"]
         nat                = true
       }
       scheduling_policy {
         preemptible = true
       }
     }
     scale_policy {
       fixed_scale {
         size = 1
       }
     }
     deploy_policy {
       max_expansion   = 3
       max_unavailable = 1
     }
     maintenance_policy {
       auto_upgrade = true
       auto_repair  = true
       maintenance_window {
         start_time = "22:00"
         duration   = "10h"
       }
     }
     node_labels = {
       node-label1 = "node-value1"
     }
     node_taints = ["taint1=taint-value1:NoSchedule"]
     labels = {
       "template-label1" = "template-value1"
     }
     allowed_unsafe_sysctls = ["kernel.msg*", "net.core.somaxconn"]
   }
   

2. Check that the configuration file is correct.

   1. In the command line, go to the folder where you created the configuration file.

   2. Run a check using this command:

      terraform plan
      

   If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out. This is a test step; no resources will be created.

3. Create a Managed Service for Kubernetes node group.

   1. If the configuration does not contain any errors, run this command:

      terraform apply
      

   2. Confirm that you want to create the resources.

   After this, all required resources will be created in the specified folder and the IP addresses of the VMs will be displayed in the terminal. You can check the new resources and their configuration using the management console.