Contact UsGet started
© 2024 Iron Hive LLC Belgrade

Updating an instance group under load

Written by
Updated at October 17, 2024

Using this step-by-step guide, you will configure an instance group and check its operation when updating the configuration. To do this:

  1. Prepare your cloud.
  2. Prepare the environment.
  3. Create an instance group from a Container Optimized Image.
  4. Create a load on an instance.
  5. Update the instance group under load.
  6. Stop the load and get the results.

If you no longer need the resources you created, delete them.

Prepare your cloud

  1. If you don't have the Yandex Cloud command line interface yet, install it.

  2. For the scripts from the tutorial to run correctly, download and install the jq utility.

  3. To check auto scaling, install the wrk benchmarking tool.

The cost of support for the Yandex Cloud instance group includes a fee for:

Prepare the environment

  1. Create a service account named for-load and assign it the editor role:

    1. In the management console, select a folder where you wish to create a service account.
    2. Go to the Service accounts tab.
    3. Click Create service account.
    4. Enter the name for-load.
    5. To assign the service account a role for the current folder, click Add role and select the role editor.
    6. Click Create.

    1. Create a service account:

      The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

      yc iam service-account create --name for-load

      Command result:

      id: ajeab0cnib1pdefe21dm
folder_id: b0g12ga82bcv0cdeferg
created_at: "2021-02-09T17:31:32.561702Z"
name: for-load

    2. Assign the role to the service account:

      yc resource-manager folder add-access-binding b0g12ga82bcv0cdeferg \
  --role editor \
  --subject serviceAccount:ajeab0cnib1pdefe21dm

    Use the methods:

    1. create for the ServiceAccount resource to create a for-load service account.
    2. setAccessBindings for the Folder resource in order to assign the editor role to the service account in the current folder.

  2. Create a network named yc-auto-network and subnets in two availability zones:

    1. In the management console, select the folder where you want to create an instance group.
    2. Select Virtual Private Cloud.
    3. Click Create network.
    4. Enter the network name yc-auto-network.
    5. Select the additional option Create subnets.
    6. Click Create network.

    1. Create a network:

      yc vpc network create --name yc-auto-network

      Command result:

      id: enpabce123hde4ft1r3t
folder_id: b0g12ga82bcv0cdeferg
created_at: "2021-02-09T17:33:32.561702Z"
name: yc-auto-network

    2. Create a subnet in the ru-central1-a zone:

      yc vpc subnet create --network-id enpabce123hde4ft1r3t --range 192.168.1.0/24 --zone ru-central1-a

      Command result:

      id: e1lnabc23r1c9d0efoje
folder_id: b0g12ga82bcv0cdeferg
created_at: "2021-02-09T17:34:32.561702Z"
network_id: enpabce123hde4ft1r3t
zone_id: ru-central1-a
v4_cidr_blocks:
- 192.168.1.0/24

    3. Create a subnet in the ru-central1-b zone:

      yc vpc subnet create --network-id enpabce123hde4ft1r3t --range 192.168.2.0/24 --zone ru-central1-b

      Command result:

      id: b1csa2b3clideftjb121
folder_id: b0g12ga82bcv0cdeferg
created_at: "2021-02-09T17:35:32.561702Z"
network_id: enpabce123hde4ft1r3t
zone_id: ru-central1-b
v4_cidr_blocks:
- 192.168.2.0/24
    1. Create a network using the method Create for the Network resource.
    2. Create subnets in the ru-central1-a and ru-central1-b availability zones using the create method for the Subnet resource.

Create an instance group from a Container Optimized Image

  1. All the instance groups are created from the image Container Optimized Image. Each instance contains a Docker container running a web server that emulates the service utilization.

    1. In the management console, select the folder where you want to create a network.
    2. Select Compute Cloud.
    3. On the Virtual machines page, go to the Instance groups tab.
    4. Click Create group.
    5. Under Basic parameters:
      • Enter the Name of the group group-for-load.
      • Select the for-load Service account.
    6. Under Allocation, select the ru-central1-a and ru-central1-b Availability zones.
    7. Under Instance template, click Define:
      • Under Image/boot disk selection, select the Container Solution tab.
      • Click Configure.
      • In the Docker container settings window:
        • Name it nginx.
        • In the Docker image field, click the Enter link button and type cr.yandex/yc/demo/autoscaling-example-app:v1.
        • Click Apply.
      • In the Disks section:
        • For the boot disk, specify the Size of 30 GB.
      • Under Network settings:
        • Select the for-load Network.
      • Under Access:
        • Select the for-load Service account.
        • In the Login field, enter the name of the user to be created on the VM.
        • In the SSH key field, paste the contents of the public key file.
      • Click Save.
    8. Under Allow when creating and updating:
      • In the Reduce below target value field, enter 4.
    9. In the Scalability section:
      • In the Type field, select Fixed.
      • Set the Size to 6.
    10. Under Integration with Load Balancer:
      • Enable the option Create target group.
      • In the Name of the target group field, enter load-generator.
    11. Click Create.

    1. Find out the ID of the latest version of the Container Optimized Image public image.

      A Container Optimized Image in a Yandex Container Registry registry can be updated and changed according to the new releases. In this case, the image on a VM isn't updated to the latest version automatically. To create an instance group with the latest Container Optimized Image version, you need to check whether it's available yourself:

      yc compute image get-latest-from-family container-optimized-image --folder-id standard-images

      Command result:

      id: fd8iv792kirahcnqnt0q
folder_id: standard-images
created_at: "2021-01-29T13:30:22Z"
...
status: READY
os:
  type: LINUX

    2. Save the specification of the instance group with network load balancer to the file specification.yaml:

      name: group-for-load # The instance group name that must be unique within the folder.
service_account_id: ajeab0cnib1p******** # Service account ID.
allocation_policy: # VM allocation policy in the group.
   zones:
     - zone_id: ru-central1-a
     - zone_id: ru-central1-b
instance_template:
  service_account_id: ajeab0cnib1p******** # ID of the service account for access to private Docker images.
  platform_id: standard-v1 # Platform ID.
  resources_spec:
    memory: 2G # Amount of memory (RAM).
    cores: 2 # Number of processor cores (vCPUs).
  boot_disk_spec:
    mode: READ_WRITE # Disk access mode (read and write).
    disk_spec:
      image_id: <ID_of_the_latest_COI_version>
    type_id: network-ssd # Disk type.
    size: 30G # Disk size.
  network_interface_specs:
   - network_id: enplhg4nncc7******** # Network ID.
     subnet_ids:
       - e1lnabc23r1c******** # Subnet ID.
       - b1csa2b3clid********
     primary_v4_address_spec: {
       one_to_one_nat_spec: {
         ip_version: IPV4 # IPv4 specification for public access to the VM.
       }
     }
  metadata: # Values to be delivered to the VM metadata.
    docker-container-declaration: |- # Key in the VM metadata that is used with the Docker Container specification.
      spec:
        containers:
          - image: cr.yandex/yc/demo/autoscaling-example-app:v1
            name: nginx
            securityContext:
              privileged: false
            restartPolicy: Always
            tty: false
            stdin: false
    ssh-keys: | # Parameter for sending an SSH key to the VM.
      yc-user:ssh-ed25519 ABC...d01 user@desktop.ru # Username for the VM connection.
deploy_policy: # Deployment policy for instances in the group.
  max_unavailable: 4
scale_policy: # Scaling policy for instances in the group.
  fixed_scale:
    size: 6
load_balancer_spec: # Network load balancer.
  target_group_spec:
    name: load-generator

      Note

      You can send an SSH key to the VM metadata using the ssh-keys parameter or in the user-data string with the user metadata. This tutorial uses the first option.

    3. Create an instance group named group-for-load using the specification specification.yaml:

      yc compute instance-group create --file=specification.yaml

      Command result:

      done (2m18s)
id: cl0kabcquk1gomdefbkk
folder_id: b0g12ga82bcv0cdeferg
...
service_account_id: ajeab0cnib1pdefe21dm
status: ACTIVE
application_load_balancer_state: {}

    Use the methods:

    1. getLatestByFamily for the Image resource to get the ID of the latest version of the container-optimized-image image in the standard-images family.

    2. createFromYaml for the InstanceGroup resource to create an instance group to meet the following specification:

      name: group-for-load # The instance group name that must be unique within the folder.
service_account_id: ajeab0cnib1p******** # Service account ID.
allocation_policy: # VM allocation policy in the group.
   zones:
     - zone_id: ru-central1-a
     - zone_id: ru-central1-b
instance_template:
  service_account_id: ajeab0cnib1p******** # ID of the service account for access to private Docker images.
  platform_id: standard-v1 # Platform ID.
  resources_spec:
    memory: 2G # Amount of memory (RAM).
    cores: 2 # Number of processor cores (vCPUs).
  boot_disk_spec:
    mode: READ_WRITE # Disk access mode (read and write).
    disk_spec:
      image_id: <ID_of_the_latest_COI_version>
    type_id: network-ssd # Disk type.
    size: 30G # Disk size.
  network_interface_specs:
   - network_id: enplhg4nncc7******** # Network ID.
     subnet_ids:
       - e1lnabc23r1c******** # Subnet ID.
       - b1csa2b3clid********
     primary_v4_address_spec: {
       one_to_one_nat_spec: {
         ip_version: IPV4 # IPv4 specification for public access to the VM.
       }
     }
  metadata: # Values to be delivered to the VM metadata.
    docker-container-declaration: |- # Key in the VM metadata that is used with the Docker Container specification.
      spec:
        containers:
          - image: cr.yandex/yc/demo/autoscaling-example-app:v1
            name: nginx
            securityContext:
              privileged: false
            restartPolicy: Always
            tty: false
            stdin: false
    ssh-keys: | # Parameter for sending an SSH key to the VM.
      yc-user:ssh-ed25519 ABC...d01 user@desktop.ru # Username for the VM connection.
deploy_policy: # Deployment policy for instances in the group.
  max_unavailable: 4
scale_policy: # Scaling policy for instances in the group.
  fixed_scale:
    size: 6
load_balancer_spec: # Network load balancer.
  target_group_spec:
    name: load-generator

  2. Make sure that the instance group was created:

    1. In the management console, select the folder where you created the instance group.
    2. Select Compute Cloud.
    3. Go to Instance groups.
    4. Click the group-for-load instance group name.
    yc compute instance-group list-instances group-for-load

    Command result:

    +----------------------+---------------------------+-----------------+-------------+----------------------+----------------+
|     INSTANCE ID      |           NAME            |   EXTERNAL IP   | INTERNAL IP |        STATUS        | STATUS MESSAGE |
+----------------------+---------------------------+-----------------+-------------+----------------------+----------------+
...
| fhmab0cfsfd125efpvn1 | cl0kabcquk1gomdefbkk-oxig | 178.154.226.108 | 10.130.0.8  | RUNNING_ACTUAL [49m] |                |
| epdabchpdef0f1e21j14 | cl0kabcquk1gomdefbkk-aqyg | 130.193.40.55   | 10.129.0.20 | RUNNING_ACTUAL [43m] |                |
+----------------------+---------------------------+-----------------+-------------+----------------------+----------------+

    See the list of the instance groups you created using the method List for the resource InstanceGroup.

Add a network load balancer to the created instance group

  1. Create a network load balancer named load-generator and connect it to the instance group you created:

    1. In the management console, select the folder where you want to create a load balancer.
    2. In the list of services, select Network Load Balancer.
    3. Click Create a network load balancer.
    4. Name it load-generator.
    5. In the Public address field, select the value Auto.
    6. Click Add listener under Listeners.
      • In the window that opens, enter http as the Listener name.
      • In the Port field, enter 80: the balancer will use this port to accept incoming traffic.
      • In the Target port field, enter 80: the balancer will redirect traffic to this port.
      • Click Add.
    7. Under Target groups, click Add target group.
    8. In the Target group drop-down list, select load-generator.
    9. In the target group block, click Configure:
      • In the window that opens, enter the Path /hello: the load balancer will use this path to send health check requests to target group VMs.
      • Click Apply.
    10. Click Create.

    1. Get the ID of the load-generator target group:

      yc load-balancer target-group get load-generator | grep "^id"

      Command result:

      id: enpsa475ej51enuam897

    2. Create a network load balancer:

      yc load-balancer network-load-balancer create \
  --name load-generator \
  --listener external-ip-version=ipv4,name=http,port=80,protocol=tcp,target-port=80 \
  --target-group healthcheck-http-port=80,healthcheck-http-path=/hello,target-group-id=<target group ID>

      Command result:

      done (14s)
id: b0ruab1ccvpd26efgii4
folder_id: b1csa2b3clideftjb121
...
    healthy_threshold: "2"
    http_options:
      port: "80"
      path: /hello
    1. Create a load balancer using the method create for the NetworkLoadBalancer resource.
    2. Add a listener to the balancer using the method addListener for the NetworkLoadBalancer resource.
    3. Attach the target group to the balancer using the method attachTargetGroup for the NetworkLoadBalancer resource.
    4. Add the balancer to the instance group using the method addTargets for the TargetGroup resource.

  2. Make sure that the network load balancer load-generator is created and attached to the instance group:

    1. In the management console, select the folder where you created the load balancer.
    2. Select Network Load Balancer.
    3. Click on the name of the network load balancer load-generator.
    yc load-balancer network-load-balancer list

    Command result:

    +----------------------+----------------+-----------------+----------+----------------+------------------------+--------+
|          ID          |      NAME      |    REGION ID    |   TYPE   | LISTENER COUNT | ATTACHED TARGET GROUPS | STATUS |
+----------------------+----------------+-----------------+----------+----------------+------------------------+--------+
| b0ruab1ccvpd26efgii4 | load-generator | ru-central1 | EXTERNAL |              1 | b0r1tabcphde28fj1dd3   | ACTIVE |
+----------------------+----------------+-----------------+----------+----------------+------------------------+--------+

    Use the list method for the NetworkLoadBalancer resource.

Create a load on an instance

  1. Get the IP address of the load balancer that you created:

    1. In the management console, select the folder where you want to create a load balancer.
    2. In the list of services, select Network Load Balancer.
    3. Copy the IP address of the load balancer.
    yc load-balancer network-load-balancer get load-generator | grep "address"

    Command result:

      address: 84.252.133.110

    Use the get method for the NetworkLoadBalancer resource.

  2. Run the command to create a load:

    wrk -t20 -c20 -d20m --timeout 20s http://<load balancer IP address>/sleep

    The command will launch the wrk utility that will send requests to 20 threads using 20 connections to the network load balancer for 20 minutes. The request timeout is 20 seconds. The load balancer will distribute the resulting load among the VMs from the group.

    After you launch wrk, a message that testing has started will appear on the screen:

    Running 20m test @ http://84.252.133.110/sleep
  20 threads and 20 connections

    Proceed to the next step without waiting for the command to complete.

Update the instance group under load

  1. In the management console, select the folder where you created the instance group.
  2. Select Compute Cloud.
  3. Go to Instance groups.
  4. Click the group-for-load instance group name.
  5. On the group page, click Edit.
  6. Under Instance template, click and select Edit.
    • Under Disks, enter the new disk size of 35 GB.
    • Click Save.
  7. Click Save changes.
  8. Under VM states, you'll see disk size changes for all VMs in the group step-by-step.

  1. In the specification.yaml specification, enter the new disk size of 35 GB and save the file:

    ...
size: 35G
...

  2. Update the instance group:

    yc compute instance-group update --name=group-for-load --file=specification.yaml

    Command result:

    done (9m24s)
id: cl10kktrjcn8polprdav
folder_id: b1g7gvsi89m34qmcm3ke
...
   name: load-generator
service_account_id: ajehbk07uus3s73pcq13
status: ACTIVE

  1. In the specification, enter the new disk size of 35 GB:

    ...
size: 35G
...

  2. Use the method updateFromYaml for the resource InstanceGroup to update the load-generator instance group according to the new specification.

Stop the load and get the results

Stop wrk by pressing Ctrl + C.

Command result:

  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.00s     1.70ms  10.01s    68.21%
    Req/Sec     0.00      0.00     0.00    100.00%
  1148 requests in 17.47m, 211.89KB read
Requests/sec:      1.09
Transfer/sec:     206.94B

If there is no error line in the result, such as Socket errors, it means that all requests were processed.

How to delete created resources

To delete the created resources:

  1. Delete the load balancer:
    1. Go to the root of the folder.
    2. In the list of services, select Network Load Balancer.
    3. To the right of the load-generator load balancer line, click and select Delete.
    4. In the window that opens, click Delete.
  2. Delete the instance group:
    1. Go to the root of the folder.
    2. In the list of services, select Compute Cloud.
    3. Go to the Instance groups tab.
    4. To the right of the load-generator instance group line, click and select Delete.
    5. In the window that opens, click Delete.
  3. Delete the service account:
    1. Go to the root of the folder.
    2. Go to the Service accounts tab.
    3. To the right of the yc-auto-sa account line, click and select Delete.
    4. In the window that opens, click Delete.
  4. Delete the network and subnets:
    1. Go to the root of the folder.
    2. In the list of services, select Virtual Private Cloud.
    3. Select the network yc-auto-network.
    4. Under Subnets:
      1. To the right of the yc-auto-subnet-1 subnet line, click and select Delete.
      2. In the window that opens, click Delete.
      3. Also delete the subnet yc-auto-subnet-2.
    5. In the upper-right corner, click Delete.

Run the following commands one-by-one:

yc load-balancer network-load-balancer delete load-generator
yc compute instance-group delete load-generator
yc iam service-account delete yc-auto-sa
yc vpc subnet delete yc-auto-subnet-1
yc vpc subnet delete yc-auto-subnet-2
yc vpc network delete yc-auto-network

Use the methods:

  1. delete for the NetworkLoadBalancer resource to delete the load-generator load balancer.
  2. delete for the InstanceGroup resource to delete the load-generator instance group.
  3. delete for the ServiceAccount resource to delete the yc-auto-sa service account.
  4. delete for the Subnet resource to delete the yc-auto-subnet-1 and yc-auto-subnet-2 subnets.
  5. delete for the resource Network to delete the network yc-auto-network.
Previous
Automatically scaling an instance group to process messages from a queue inMessage Queue
Next
Creating a trigger for budgets that invokes a function to stop VM instances