Contact UsGet started
© 2024 Iron Hive LLC Belgrade

Creating an autoscaling instance group

Written by
Updated at November 21, 2024

You can create an autoscaling group of identical instances. This instance group will scale up or down automatically.

Alert

When creating instance groups, keep the limits in mind. Not to disrupt the component Instance Groups, do not update or delete manually created resources: target group Network Load Balancer, VMs, and disks. Instead of this, change or delete the entire group.

By default, all operations in Instance Groups are performed on behalf of a service account. If you don't have a service account, create one.

To create an autoscaling instance group:

  1. In the management console, select the folder where you want to create an instance group.
  2. In the list of services, select Compute Cloud.
  3. In the left-hand panel, select Instance groups.
  4. Click Create group of virtual machines.
  5. Under Basic parameters:

    • Enter a name and description of the instance group. The naming requirements are as follows:

      • The name must be from 3 to 63 characters long.
      • It may contain lowercase Latin letters, numbers, and hyphens.
      • The first character must be a letter and the last character cannot be a hyphen.

      Note

      The VM name is used to generate an internal FQDN only once: when creating a VM. If the internal FQDN is important to you, choose an appropriate name for the VM at the creation stage.

    • Select a service account from the list or create a new one. To be able to create, update, and delete VM instances in the instance group, assign the editor role to the service account. By default, all operations in Instance Groups are performed on behalf of a service account.

      You cannot delete a service account while it is linked to an instance group.

    • Enable Deletion protection if needed. You cannot delete a group with this option enabled.

  6. Under Allocation, select the required ones in the Availability zone field. VM groups may reside in different availability zones.
  7. Under Instance template, click Define to configure a basic instance:

    • Under General information, enter a description for the template.

    • Under Boot disk image, select a system to be deployed on the VM instance's boot disk.

    • Under Disks:

      • Select the disk type.
      • Specify the disk size.
      • To add more disks, click Add disk.

    • Under Computing resources:

      • Choose a platform.

      • Enter the required number of vCPUs, guaranteed vCPU performance, and the amount of RAM.

      • If necessary, make your VM preemptible.

        When creating a preemptible instance group, keep in mind that the virtual machines will terminate after 24 hours of continuous operation or earlier. It's possible that Instance Groups won't be able to restart them immediately due to insufficient resources. This may occur in the event of a drastic increase of the computing resource utilization in Yandex Cloud.

      • (Optional) Enable a software-accelerated network.

    • Under Network settings:

      • Enter a subnet ID or select a cloud network from the list.
        If you do not have a network, click Create a network to create one:

        • In the window that opens, enter a name for your network.
        • (Optional) To automatically create subnets, select the Create subnets option.
        • Click Create network.

        Each network must have at least one subnet. If there is no subnet, create one by selecting Create subnet.

      • In the Public address field, choose a method for assigning an IP address:

        • Auto: Assign a random IP address from the Yandex Cloud IP pool.
        • No address: Do not assign a public IP address.

      • Select the appropriate security groups.

      • (Optional) Create records for the VM in the DNS zone:

        • Expand the DNS settings for internal addresses section and click Add record.
        • Specify the zone, FQDN, and TTL for the record. When setting the FQDN, you can select Detect automatically for the zone.
          You can add multiple records to internal DNS zones. For more information, see Cloud DNS integration with Compute Cloud.

    • Under Access, specify the data for access to the VM:

      • Select a service account to link to the instance.
      • If you selected a Linux image, fill out the fields Login and SSH key. For a key, use the contents of the public key file.
      • Select Access to serial console, if needed.

    • Click Save.

  8. Under Allow when creating and updating:

    • In the Add above target value field, specify the number of instances you can exceed the group size by.

      When bringing the number of VMs in the group to the target value, VMs created in excess of the target under the max_expansion quota can remain in the group, while those that existed in the group before may be deleted.

    • In the field Reduce below target value, specify the number of instances you can decrease the group size by.

    • In the Create simultaneously field, specify how many instances can be created at the same time.

    • In Startup duration, specify the period after which the VM instance will start receiving the load.

    • In the Stop simultaneously field, specify how many instances can be stopped at the same time.

    • In the Stop VMs by strategy field, specify one of the strategies:

      • Proactive: Instance Groups itself chooses which instances to stop when updating or scaling down the group.
      • Opportunistic: Instance Groups waits for the instances to stop on their own or be stopped by the user.
  9. Under Scaling:
    • Select the Automatic scaling type.
    • Specify the following:
      • Autoscaling type: Zonal autoscaling adjusts the number of VM instances separately for each availability zone specified in the group settings. Regional does the same for the entire instance group.
      • Minimum number of instances per zone.
      • Maximum group size.
      • Utilization measurement period to average the utilization measurements for each instance in the group.
      • Instance warmup period: Startup duration of an instance in the group.
      • Stabilization period: Interval during which the required number of instances in the group cannot be reduced.
      • Initial group size.
  10. Under Metrics, specify:
    • In the Metric field:
      • CPU: Specify the average CPU utilization level that Instance Groups must support.
      • Yandex Monitoring: Specify the name of the custom metric and target value that Instance Groups must support.
  11. If needed, enable the Health checks to get information about the state of instances and their automatic recovery on failure.
    • In the Type field, select the protocol for the health checks: HTTP or TCP.
    • In the Path field (for the HTTP type), specify the URL path for the HTTP check requests sent from Instance Groups.
    • In the Port field, specify the port number from 1 to 32767 for Instance Groups to send the health check requests to.
    • In the Timeout in sec field, specify the response waiting time from 1 to 60 seconds.
    • In the Interval in sec field, specify the interval between the repeat checks from 1 to 60 seconds. The interval must be at least 1 second longer than the waiting time.
    • In the Healthy threshold field, specify the number of successful health checks required for the instance to be considered healthy.
    • In the Unhealthy threshold field, specify the number of failed health checks for the instance to be considered unhealthy.
  12. Under User-defined variables, enter the Key-Value pairs, if needed.
  13. Click Create.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

  1. View the description of the CLI command to create an instance group:

    yc compute instance-group create --help

  2. Check if there are any networks in the folder:

    yc vpc network list

    If there are none, create a network.

  3. Select one of the Yandex Cloud Marketplace public images, e.g., CentOS 7.

    To get a list of available images using the CLI, run this command:

    yc compute image list --folder-id standard-images

    Result:

    +----------------------+-------------------------------------+--------------------------+----------------------+--------+
|          ID          |                NAME                 |          FAMILY          |     PRODUCT IDS      | STATUS |
+----------------------+-------------------------------------+--------------------------+----------------------+--------+
...
| fdvk34al8k5n******** | centos-7-1549279494                 | centos-7                 | dqni65lfhvv2******** | READY  |
| fdv7ooobjfl3******** | windows-2016-gvlk-1548913814        | windows-2016-gvlk        | dqnnc72gj2is******** | READY  |
| fdv4f5kv5cvf******** | ubuntu-1604-lts-1549457823          | ubuntu-1604-lts          | dqnnb6dc7640******** | READY  |
...
+----------------------+-------------------------------------+--------------------------+----------------------+--------+

  4. Create a YAML file with any name, e.g., specification.yaml.

  5. In the created file, indicate the following:

    • General information about the instance group:

      name: first-autoscaled-group
service_account_id: <service_account_ID>
description: "This instance group was created from YAML config."

      Where:

      • name: Name of the instance group. The name must be unique within the folder. The name may contain lowercase Latin letters, numbers, and hyphens. The first character must be a letter. The last character cannot be a hyphen. The name may be up to 63 characters long.

      • service_account_id: Service account ID.

        You cannot delete a service account while it is linked to an instance group.

      • description: Description of the instance group.

    • Instance template, such as:

      instance_template:
  platform_id: standard-v3
  resources_spec:
    memory: 2g
    cores: 2
  boot_disk_spec:
    mode: READ_WRITE
    disk_spec:
      image_id: fdvk34al8k5n********
      type_id: network-hdd
      size: 32g
  network_interface_specs:
    - network_id: c64mknqgnd8a********
      primary_v4_address_spec: {}
      security_group_ids:
        - enps0ar5s3ti********
  scheduling_policy:
    preemptible: false

      By default, the disk size is specified in bytes. You can specify a different unit of measurement using the applicable suffix.

      Suffix Prefix and multiplier Example
      k kilo- (210) 640k = 640 × 210 = 655360
      m mega- (220) 48m = 48 × 220 = 50331648
      g giga- (230) 10g = 10 × 230 = 10737418240
      t tera- (240) 4t = 4 × 240 = 4398046511104
      p peta- (250) 2p = 2 × 250 = 2251799813685248

      Where:

      • platform_id: Platform ID.

      • memory: Amount of RAM.

      • cores: Number of processor cores (vCPUs).

      • mode: Disk access mode.

        • READ_ONLY: Read-only access.
        • READ_WRITE: Read/write access.

      • image_id: ID of the public image. You can view it in the management console when creating a VM or in Cloud Marketplace on the image page under Product IDs.

      • type_id: Disk type.

      • size: Disk size.

      • network_id: ID of the default-net network.

      • primary_v4_address_spec: IPv4 specification. You can allow public access to the group's instances by specifying the IP version for the public IP address. For more information, see Template description in a YAML file.

      • security_group_ids: List of security group IDs.

      • scheduling_policy: Scheduling policy configuration.

      • preemptible: Flag for creating preemptible VMs.

        • true: Create a preemptible VM.
        • false (default): Create a regular VM.

        When creating a preemptible instance group, keep in mind that the VM instances will terminate after 24 hours of continuous operation or earlier. VMs may not be able to restart immediately due to insufficient resources. This may occur in the event of a sharp increase in the use of Yandex Cloud computing resources.

    • Policies:

      deploy_policy:
  max_unavailable: 1
  max_expansion: 0
scale_policy:
  auto_scale:
    initial_size: 5
    max_size: 15
    min_zone_size: 3
    measurement_duration: 30s
    warmup_duration: 60s
    stabilization_duration: 120s
    cpu_utilization_rule: 0.75
allocation_policy:
  zones:
    - zone_id: ru-central1-a

      Where:

    Full code for the specification.yaml file:

    name: first-autoscaled-group
service_account_id: ajed6ilf11qg********
description: "This instance group was created from YAML config."
instance_template:
  platform_id: standard-v3
  resources_spec:
    memory: 2g
    cores: 2
  boot_disk_spec:
    mode: READ_WRITE
    disk_spec:
      image_id: fdvk34al8k5n********
      type_id: network-hdd
      size: 32g
  network_interface_specs:
    - network_id: c64mknqgnd8a********
      primary_v4_address_spec: {}
      security_group_ids:
        - enps0ar5s3ti********
deploy_policy:
  max_unavailable: 1
  max_expansion: 0
scale_policy:
  auto_scale:
    initial_size: 5
    max_size: 15
    min_zone_size: 3
    measurement_duration: 30s
    warmup_duration: 60s
    stabilization_duration: 120s
    cpu_utilization_rule:
      utilization_target: 75
allocation_policy:
  zones:
    - zone_id: ru-central1-a

  6. Create an instance group in the default folder:

    yc compute instance-group create --file specification.yaml

    This command creates an autoscaling instance group with the following characteristics:

    • Name: first-autoscaled-group
    • OS: CentOS 7
    • Network: default-net
    • Availability zone: ru-central1-a
    • vCPUs: 2; RAM: 2 GB
    • Network HDD: 32 GB

If you don't have Terraform, install it and configure the Yandex Cloud provider.

  1. In the configuration file, describe the parameters of the resources you want to create:

    resource "yandex_iam_service_account" "ig-sa" {
  name        = "ig-sa"
  description = "Service account for managing the instance group."
}

resource "yandex_resourcemanager_folder_iam_member" "editor" {
  folder_id = "<folder_ID>"
  role      = "editor"
  member    = "serviceAccount:${yandex_iam_service_account.ig-sa.id}"
}

resource "yandex_compute_instance_group" "ig-1" {
  name                = "autoscaled-ig"
  folder_id           = "<folder_ID>"
  service_account_id  = "${yandex_iam_service_account.ig-sa.id}"
  deletion_protection = "<deletion_protection>"
  instance_template {
    platform_id = "standard-v3"
    resources {
      memory = <RAM_amount_in_GB>
      cores  = <number_of_vCPUs>
    }

    boot_disk {
      mode = "READ_WRITE"
      initialize_params {
        image_id = "<image_ID>"
      }
    }

    network_interface {
      network_id         = "${yandex_vpc_network.network-1.id}"
      subnet_ids         = ["${yandex_vpc_subnet.subnet-1.id}"]
      security_group_ids = ["<list_of_security_group_IDs>"]
    }

    metadata = {
      ssh-keys = "<username>:<SSH_key_contents>"
    }
  }

  scale_policy {
    auto_scale {
      initial_size           = 3
      measurement_duration   = 60
      cpu_utilization_target = 75
      min_zone_size          = 3
      max_size               = 15
      warmup_duration        = 60
      stabilization_duration = 120
    }
  }

  allocation_policy {
    zones = ["ru-central1-a"]
  }

  deploy_policy {
    max_unavailable = 1
    max_expansion   = 0
  }
}

resource "yandex_vpc_network" "network-1" {
  name = "network1"
}

resource "yandex_vpc_subnet" "subnet-1" {
  name           = "subnet1"
  zone           = "ru-central1-a"
  network_id     = "${yandex_vpc_network.network-1.id}"
  v4_cidr_blocks = ["192.168.10.0/24"]
}

    Where:

    • yandex_iam_service_account: Description of the service account. All operations with instance groups are performed on behalf of the service account.

    • yandex_resourcemanager_folder_iam_member: Description of access permissions to the folder the service account belongs to. To be able to create, update, and delete instances in the instance group, assign the editor role to the service account.

    • yandex_compute_instance_group: Description of the instance group.

      • General information about the instance group:
        • name: Name of the instance group.
        • folder_id: Folder ID.
        • service_account_id: Service account ID.
        • deletion_protection: Instance group deletion protection, true or false. You cannot delete an instance group with this option enabled. The default value is false.
      • Instance template:
        • platform_id: Platform.
        • resources: Number of vCPU cores and the amount of RAM available to the VM. The values must match the selected platform.
        • boot_disk: Boot disk settings.
          • ID of the selected image. You can get the image ID from the list of public images.
          • Disk access mode: READ_ONLY (read) or READ_WRITE (read and write).
        • network_interface: Network settings. Specify the IDs of your network, subnet, and security groups.
        • metadata: In the metadata, provide the public key for VM access via SSH. For more information, see VM metadata.
      • Policies:

    • yandex_vpc_network: Description of the cloud network.

    • yandex_vpc_subnet: Description of the subnet the instance group will connect to.

      Note

      If you already have suitable resources, such as a service account, cloud network, and subnet, you do not need to describe them again. Use their names and IDs in the appropriate parameters.

    For more information about the resources you can create with Terraform, see the provider documentation.

  2. Create resources:

    1. In the terminal, change to the folder where you edited the configuration file.

    2. Make sure the configuration file is correct using the command:

      terraform validate

      If the configuration is correct, the following message is returned:

      Success! The configuration is valid.

    3. Run the command:

      terraform plan

      The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

    4. Apply the configuration changes:

      terraform apply

    5. Confirm the changes: type yes in the terminal and press Enter.

    All the resources you need will then be created in the specified folder. You can check the new resources and their configuration using the management console.

Use the create REST API method for the InstanceGroup resource or the InstanceGroupService/Create gRPC API call.

If you are using a Windows image, reset the password after creating a VM.

Previous
Creating a fixed-size instance group with an L7 load balancer
Next
Creating an instance group from a Container Optimized Image