Running an instance group with autoscaling using Terraform
To create an infrastructure for an instance group with an autoscaling policy using Terraform:
If you no longer need the resources you created, delete them.
Prepare your cloud
Sign up for Yandex Cloud and create a billing account:
- Go to the management console
and log in to Yandex Cloud or create an account if you do not have one yet. - On the Yandex Cloud Billing
page, make sure you have a billing account linked and it has theACTIVE
orTRIAL_ACTIVE
status. If you do not have a billing account, create one.
If you have an active billing account, you can go to the cloud page
Learn more about clouds and folders.
-
If you don't have the Yandex Cloud command line interface yet, install it.
-
For the scripts from the tutorial to run correctly, download and install the jq
utility. -
To check auto scaling, install the wrk
benchmarking tool.
Required paid resources
The infrastructure cost includes:
- Fee for continuously running VMs (see Compute Cloud pricing).
- Fee for network load balancers and traffic balancing (see Network Load Balancer pricing).
Create an infrastructure
Terraform
For more information about the provider resources, see the documentation on the Terraform
If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.
To create an infrastructure to automatically scale an instance group using Terraform:
-
Install Terraform and specify the source for installing the Yandex Cloud provider (see Configure a provider, step 1).
-
Prepare files with the infrastructure description:
Ready-made configurationManually-
Clone the repository with configuration files:
git clone https://github.com/yandex-cloud-examples/yc-vm-group-with-autoscale.git
-
Go to the directory with the repository. Make sure it contains the following files:
vm-autoscale.tf
: New infrastructure configuration.declaration.yaml
: Description of the Docker container with a web server that will be run on the VM to simulate the load on the service.config.tpl
: Description of VM user parameters.vm-autoscale.auto.tfvars
: User data.
- Create a folder for configuration files.
- In the folder, create:
-
vm-autoscale.tf
configuration filevm-autoscale.tf
# Declaring variables for confidential parameters variable "folder_id" { type = string } variable "vm_user" { type = string } variable "ssh_key" { type = string sensitive = true } # Setting up the provider terraform { required_providers { yandex = { source = "yandex-cloud/yandex" version = ">= 0.47.0" } } } provider "yandex" { zone = "ru-central1-a" } # Creating a service account and assigning roles to it resource "yandex_iam_service_account" "for-autoscale" { name = "for-autoscale" } resource "yandex_resourcemanager_folder_iam_member" "vm-autoscale-sa-role-compute" { folder_id = var.folder_id role = "editor" member = "serviceAccount:${yandex_iam_service_account.for-autoscale.id}" } # Creating a cloud network and subnets resource "yandex_vpc_network" "vm-autoscale-network" { name = "vm-autoscale-network" } resource "yandex_vpc_subnet" "vm-autoscale-subnet-a" { name = "vm-autoscale-subnet-a" zone = "ru-central1-a" v4_cidr_blocks = ["192.168.1.0/24"] network_id = yandex_vpc_network.vm-autoscale-network.id } resource "yandex_vpc_subnet" "vm-autoscale-subnet-b" { name = "vm-autoscale-subnet-b" zone = "ru-central1-b" v4_cidr_blocks = ["192.168.2.0/24"] network_id = yandex_vpc_network.vm-autoscale-network.id } # Creating a security group resource "yandex_vpc_security_group" "sg-1" { name = "sg-autoscale" network_id = yandex_vpc_network.vm-autoscale-network.id egress { protocol = "ANY" description = "any" v4_cidr_blocks = ["0.0.0.0/0"] } ingress { protocol = "TCP" description = "ext-http" v4_cidr_blocks = ["0.0.0.0/0"] port = 80 } ingress { protocol = "TCP" description = "healthchecks" predefined_target = "loadbalancer_healthchecks" port = 80 } } # Specifying a ready-made VM image data "yandex_compute_image" "autoscale-image" { family = "container-optimized-image" } # Creating an instance group resource "yandex_compute_instance_group" "autoscale-group" { name = "autoscale-vm-ig" folder_id = var.folder_id service_account_id = yandex_iam_service_account.for-autoscale.id instance_template { platform_id = "standard-v3" resources { memory = 2 cores = 2 } boot_disk { mode = "READ_WRITE" initialize_params { image_id = data.yandex_compute_image.autoscale-image.id size = 30 } } network_interface { network_id = yandex_vpc_network.vm-autoscale-network.id subnet_ids = [ yandex_vpc_subnet.vm-autoscale-subnet-a.id, yandex_vpc_subnet.vm-autoscale-subnet-b.id ] security_group_ids = [ yandex_vpc_security_group.sg-1.id ] nat = true } metadata = { user-data = templatefile("config.tpl", { VM_USER = var.vm_user SSH_KEY = var.ssh_key }) docker-container-declaration = file("declaration.yaml") } } scale_policy { auto_scale { initial_size = 2 measurement_duration = 60 cpu_utilization_target = 40 min_zone_size = 1 max_size = 6 warmup_duration = 120 } } allocation_policy { zones = [ "ru-central1-a", "ru-central1-b" ] } deploy_policy { max_unavailable = 1 max_expansion = 0 } load_balancer { target_group_name = "auto-group-tg" target_group_description = "load balancer target group" } } # Creating a network load balancer resource "yandex_lb_network_load_balancer" "balancer" { name = "group-balancer" listener { name = "http" port = 80 target_port = 80 } attached_target_group { target_group_id = yandex_compute_instance_group.autoscale-group.load_balancer[0].target_group_id healthcheck { name = "tcp" tcp_options { port = 80 } } } }
-
declaration.yaml
file describing the Docker container with a web server that will be run on the VM to simulate the load on the service.declaration.yaml
spec: containers: - image: cr.yandex/yc/demo/web-app:v1 securityContext: privileged: false tty: false stdin: false
-
config.tpl
file with VM user parameters:config.tpl
users: - name: "${VM_USER}" groups: sudo shell: /bin/bash sudo: 'ALL=(ALL) NOPASSWD:ALL' ssh_authorized_keys: - "${SSH_KEY}"
-
vm-autoscale.auto.tfvars
user data file:vm-autoscale.auto.tfvars
folder_id = "<folder_ID>" vm_user = "<VM_user_name>" ssh_key = "<public_SSH_key_contents>"
-
For more information about the parameters of resources used in Terraform, see the provider documentation:
-
-
In the
vm-autoscale.auto.tfvars
file, set the following user-defined parameters:folder_id
: Folder ID.vm_user
: VM user name.ssh_key
: Contents of the file with a public SSH key to authenticate the user on the VM. For more information, see Creating an SSH key pair.
-
Create resources:
-
In the terminal, change to the folder where you edited the configuration file.
-
Make sure the configuration file is correct using the command:
terraform validate
If the configuration is correct, the following message is returned:
Success! The configuration is valid.
-
Run the command:
terraform plan
The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.
-
Apply the configuration changes:
terraform apply
-
Confirm the changes: type
yes
in the terminal and press Enter.
-
After creating an infrastructure, test your VM group and network load balancer.
Test your instance group and network load balancer
-
Create a load on an instance.
To do this, save the script named
request.sh
to the home directory. The script will send a request to one of the VMs through thegroup-balancer
load balancer. The request will utilize 100% CPU for 30 seconds.EXTERNAL_IP=$(yc load-balancer network-load-balancer get group-balancer --format=json | jq -r .listeners[0].address) curl "http://$EXTERNAL_IP/burn-cpu?time=30000&load=100"
-
Run the script:
CLIsh request.sh
Result:
projects/b0g12ga82bcv********/zones/ru-central1-b
-
View the instance utilization:
Management console-
In the management console
, select the folder where you created the instance group. -
Select Compute Cloud.
-
In the left-hand panel, click
Instance groups. -
Select
auto-group
. -
Go to the Monitoring tab.
The load balancer sent the request to an instance in the group. In the availability zone this instance belongs to, the average CPU utilization is higher than in other zones (see the Average CPU utilization in zone chart).
-
Test autoscaling
To test autoscaling for your instance group, increase the CPU utilization of each instance. In the specification.yaml
file, the scale_policy.auto_scale.cpu_utilization_rule.utilization_target
parameter is set to 40
: it means that the target utilization level is 40% CPU. If you exceed the target utilization, the number of VMs in the group will increase.
-
Increase the utilization for the instance group.
To do this, save the script named
load.sh
to the home directory. The script sends requests to the instance group through 12 threads for 10 minutes. Each VM instance utilizes 20% CPU on each core that processes the request. The instance group utilization is 240% CPU at any given time. To be sure that requests are evenly distributed across the instances in the group, the script executes multiple parallel requests utilizing 20% CPU rather than one request utilizing 240% CPU.EXTERNAL_IP=$(yc load-balancer network-load-balancer get group-balancer --format=json | jq -r .listeners[0].address) wrk -H "Connection: close" -t12 -c12 -d10m "http://$EXTERNAL_IP/burn-cpu?time=5000&load=20"
-
Run the script:
CLIsh load.sh
Result:
Running 10m test @ http://130.193.56.111/burn-cpu?time=5000&load=20 12 threads and 12 connections Thread Stats Avg Stdev Max +/- Stdev ... Requests/sec: 15.04 Transfer/sec: 3.20KB
-
View the utilization levels:
Management console- In the management console
, select the folder where you created theauto-group
instance group. - Select Compute Cloud.
- In the panel on the left, click
Instance groups. - Select
auto-group
. - Go to the Monitoring tab.
The chart Number of instances in zone shows how the number of instances changed in each availability zone. The chart Average CPU utilization in zone shows average CPU utilization in each availability zone. - Go to the Logs tab.
The page displays messages relating to the instance group autoscaling.
The total utilization of 240% CPU was evenly distributed between two instances in two availability zones and exceeded the target utilization of 40% CPU. Yandex Compute Cloud created one instance more in each availability zone to result in four instances in the group. When the script stopped utilizing the CPU, Compute Cloud automatically decreased the number of instances in the group to two.
- In the management console
How to delete the resources you created
To stop paying for the resources you created:
-
Open the
vm-autoscale.tf
configuration file and delete the description of the new infrastructure from it. -
Apply the changes:
-
In the terminal, change to the folder where you edited the configuration file.
-
Make sure the configuration file is correct using the command:
terraform validate
If the configuration is correct, the following message is returned:
Success! The configuration is valid.
-
Run the command:
terraform plan
The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.
-
Apply the configuration changes:
terraform apply
-
Confirm the changes: type
yes
in the terminal and press Enter.
-