Deploying a fault-tolerant architecture with preemptible VMs
In this tutorial, you will create a Yandex Cloud Functions function in Node.js that will be invoked on a schedule and restart a stopped preemptible Yandex Compute Cloud VM.
The architecture described here is suitable for VMs with non-critical loads. It allows you to use cost advantages of preemptible VMs and, in the event of a VM shutdown, ensures idle time of under one minute.
To deploy a fault-tolerant architecture with a preemptible VM:
- Prepare the environment.
- Prepare a ZIP archive with the function code.
- Create a function.
- Create a trigger.
- Test the function.
If you no longer need the resources you created, delete them.
Getting started
Sign up for Yandex Cloud and create a billing account:
- Go to the management console
and log in to Yandex Cloud or create an account if you do not have one yet. - On the Yandex Cloud Billing
page, make sure you have a billing account linked and it has theACTIVE
orTRIAL_ACTIVE
status. If you do not have a billing account, create one.
If you have an active billing account, you can go to the cloud page
Learn more about clouds and folders.
Required paid resources
The infrastructure support costs include:
- Fee for VM computing resources (see Compute Cloud pricing).
- Fee for VM disks (see Compute Cloud pricing).
- Fee for using a dynamic or static public IP address (see Yandex Virtual Private Cloud pricing).
- Secret storage and request fees (see Yandex Lockbox pricing).
- Fee for the number of function calls, computing resources allocated to a function, and outgoing traffic (see Cloud Functions pricing).
- Fee for logging operations and data storage in a log group (see Yandex Cloud Logging pricing) if using Cloud Logging.
Prepare the environment
- Create a service account that will be used to invoke the function and assign it the
functions.functionInvoker
andlockbox.payloadViewer
roles. - Create a preemptible VM.
Create a secret
Create a Yandex Lockbox secret to store an OAuth token.
Note
Use an OAuth token if you cannot request an IAM token automatically. The IAM token is updated more frequently and is therefore more secure.
- In the management console
, select the folder you want to create a secret in. - In the list of services, select Lockbox.
- Click Create secret.
- In the Name field, enter a name for the secret, e.g.,
oauth-token
. - Under Version:
- In the Key field, enter
key_token
. - In the Value field, enter the OAuth token value required for function authorization.
- In the Key field, enter
- Click Create.
To create a secret, run this command:
yc lockbox secret create --name oauth-token \
--payload "[{'key': 'key_token', 'text_value': '<OAuth_token>'}]"
Where text_value
is the OAuth token value required for function authorization.
Result:
done (1s)
id: e6qu9ik259lb********
folder_id: b1g9d2k0itu4********
...
status: ACTIVE
payload_entry_keys:
- key_token
-
In the configuration file, describe the secret parameters:
resource "yandex_lockbox_secret" "oauth-token" { name = "oauth-token" } resource "yandex_lockbox_secret_version" "my_version" { secret_id = yandex_lockbox_secret.my_secret.id entries { key = "key_token" text_value = "<OAuth token>" } }
Where:
name
: Secret namekey
: Key of the secret.text_value
: OAuth token value required for function authorization.
Note
We recommend using
yandex_lockbox_secret_version_hashed
: it stores values in Terraform state in hashed format. We continue supportingyandex_lockbox_secret_version
.For more information about
yandex_lockbox_secret_version_hashed
, see the relevant provider documentation .For more information about the parameters of resources used in Terraform, see the provider documentation:
-
Make sure the configuration files are correct.
-
In the command line, go to the folder where you created the configuration file.
-
Run a check using this command:
terraform plan
If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.
-
-
Deploy cloud resources.
-
If the configuration does not contain any errors, run this command:
terraform apply
-
Confirm the secret creation: type
yes
in the terminal and press Enter.
-
To create a secret, use the create REST API method for the Secret resource or the SecretService/Create gRPC API call.
Prepare a ZIP archive with the function code
-
Save the following code to a file named
index.js
:import { serviceClients, Session, cloudApi } from '@yandex-cloud/nodejs-sdk'; const { compute: { instance_service: { ListInstancesRequest, GetInstanceRequest, StartInstanceRequest, }, instance: { IpVersion, }, }, } = cloudApi; const FOLDER_ID = process.env.FOLDER_ID; const INSTANCE_ID = process.env.INSTANCE_ID; const OAUTHTOKEN = process.env.OAUTHTOKEN; export const handler = async function (event, context) { const session = new Session({ oauthToken: OAUTHTOKEN }); const instanceClient = session.client(serviceClients.InstanceServiceClient); const list = await instanceClient.list(ListInstancesRequest.fromPartial({ folderId: FOLDER_ID, })); const state = await instanceClient.get(GetInstanceRequest.fromPartial({ instanceId: INSTANCE_ID, })); var status = state.status if (status == 4){ const startcommand = await instanceClient.start(StartInstanceRequest.fromPartial({ instanceId: INSTANCE_ID, })); } return { statusCode: 200, body: { status } }; };
-
Save the following code to a file named
package.json
:{ "name": "my-awesome-package", "version": "1.0.0", "type": "module", "dependencies": { "@yandex-cloud/nodejs-sdk": "latest" } }
-
Add the
index.js
andpackage.json
files into thefunction-js.zip
archive.
Create a function
- In the management console
, select the folder where you want to create a function. - In the list of services, select Cloud Functions.
- Create a function:
- Click Create function.
- In the window that opens, enter
function-restart-vms
as the function name. - Click Create.
- Create a function version:
- Select the
nodejs18
runtime environment, disable the Add files with code examples option, and click Continue. - In the Method field, select
ZIP archive
. - In the File field, click Attach file and select the
function-js.zip
archive you created earlier. - Specify the entry point:
index.handler
. - Under Parameters, specify:
- Timeout, sec:
3
. - Memory:
128 MB
. - Service account: Select the previously created service account with rights to invoke the function.
- Environment variables:
FOLDER_ID
: ID of the folder you want to start the stopped VM instances in.INSTANCE_ID
: ID of the VM instance you want to start at interruption.
- Lockbox secrets:
- In the Environment variable field, specify
OAUTHTOKEN
. - In the Secret ID field, select the previously created
oauth-token
secret. - In the Version ID field, select a version for the secret.
- In the Secret key field, select
key_token
as the key name.
- In the Environment variable field, specify
- If you want to avoid logging and paying for Cloud Logging, disable logging by selecting
Not specified
in the Destination field under Logging.
- Timeout, sec:
- Click Save changes.
- Select the
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
-
Create a function named
function-restart-vms
:yc serverless function create --name function-restart-vms
Result:
id: d4ebrmenrr7l******** folder_id: b1g9d2k0itu4******** created_at: "2023-10-28T17:26:58.200799757Z" name: function-restart-vms http_invoke_url: https://functions.yandexcloud.net/d4ebrmenrr7l******** status: ACTIVE
-
Create a version of the
function-restart-vms
function:yc serverless function version create \ --function-name function-restart-vms \ --memory=128m \ --execution-timeout=3s \ --runtime=nodejs18 \ --entrypoint=index.handler \ --service-account-id=<service_account_ID> \ --environment FOLDER_ID=<folder_ID>,INSTANCE_ID=<VM_ID> \ --secret name=oauth-token,version-id=<secret_version_ID>,key=key_token,environment-variable=OAUTHTOKEN \ --source-path=./function-js.zip \ --no-logging
Where:
--function-name
: Name of the function a version of which you are creating.--memory
: Amount of RAM.--execution-timeout
: Maximum function running time before the timeout is reached.--runtime
: Runtime environment.--entrypoint
: Entry point.--service-account-id
: ID of the service account with the permissions to invoke the function.--environment
: Environment variables:FOLDER_ID
: ID of the folder you want to start the stopped VM instances in.INSTANCE_ID
: ID of the VM instance you want to start at interruption.
--secret
: Yandex Lockbox secret data:name
: Secret nameversion-id
: Secret version ID.key
: Key of the secret.environment-variable
: Environment variable the secret will be kept in.
--source-path
: Path to thefunction-js.zip
archive you created earlier.- (Optional)
--no-logging
: Set this flag to avoid logging and paying for Cloud Logging.
Result:
done (16s) id: d4etv5f4sjet******** function_id: d4ebrmenrr7l******** ... log_options: disabled: true folder_id: b1g9d2k0itu4********
If you do not have Terraform yet, install it and configure the Yandex Cloud provider.
-
In the configuration file, describe the
function-restart-vms
function parameters and its versions:resource "yandex_function" "function-restart-vms" { name = "function-restart-vms" user_hash = "first function" runtime = "nodejs18" entrypoint = "index.handler" memory = "128" execution_timeout = "3" service_account_id = "<service_account_ID>" folder_id = "<folder_ID>" environment = { FOLDER_ID = "<folder_ID>" INSTANCE_ID = "<VM_ID>" } secrets { id = "<secret_ID>" version_id = "<secret_version_ID>" key = "key_token" environment_variable = "OAUTHTOKEN" } content { zip_filename = "./function-js.zip" } }
Where:
name
: Function name.user_hash
: Any string to identify the function version.runtime
: Function runtime environment.entrypoint
: Entry point.memory
: Amount of memory allocated for the function, in MB.execution_timeout
: Function execution timeout.service_account_id
: ID of the service account with the permissions to invoke the function.folder_id
: ID of the folder you are creating the function in.environment
: Environment variables:FOLDER_ID
: ID of the folder you want to start the stopped VM instances in.INSTANCE_ID
: ID of the VM instance you want to start at interruption.
secrets
: Yandex Lockbox secret data:id
: Secret ID.version_id
: Secret version ID.key
: Key of the secret.environment_variable
: Environment variable the secret will be kept in.
zip_filename
: Path to thefunction-js.zip
archive you created earlier.
For more information about the
yandex_function
resource parameters, see the provider documentation . -
Make sure the configuration files are correct.
-
In the command line, go to the folder where you created the configuration file.
-
Run a check using this command:
terraform plan
If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.
-
-
Deploy cloud resources.
-
If the configuration does not contain any errors, run this command:
terraform apply
-
Confirm creating the function: type
yes
in the terminal and press Enter.This will create a function named
function-restart-vms
in the specified folder. You can check the new resources and their configuration using the management console or this CLI command:yc serverless function get function-restart-vms
Result:
id: d4ees84gsdsd******** folder_id: b1g9d2k0itu4******** created_at: "2023-08-09T10:11:40.740Z" name: function-restart-vms log_group_id: ckgjitlio5aj******** http_invoke_url: https://functions.yandexcloud.net/d4ees84gsdsd******** status: ACTIVE
-
To create a function, use the create REST API method for the Function resource or the FunctionService/Create gRPC API call.
To create a function version, use the createVersion REST API method for the Function resource or the FunctionService/CreateVersion gRPC API call.
Create a trigger
Note
The trigger is initiated within 5 minutes of being created.
- In the management console
, select the folder where you want to create a trigger. - In the list of services, select Cloud Functions.
- In the left-hand panel, select
Triggers. - Click Create trigger.
- Under Basic settings:
- Enter a name for the trigger:
timer
. - In the Type field, select
Timer
. - In the Launched resource field, select
Function
.
- Enter a name for the trigger:
- Under Timer settings, enter
* * ? * * *
or selectEvery minute
. - Under Function settings, select
function-restart-vms
and specify the following:- Function version tag:
$latest
. - Previously created service account with rights to invoke the function.
- Function version tag:
- Click Create trigger.
To create a trigger that invokes a function, run this command:
yc serverless trigger create timer \
--name timer \
--cron-expression '* * ? * * *' \
--invoke-function-name function-restart-vms \
--invoke-function-service-account-id <service_account_ID>
Where:
--name
: Trigger name.--cron-expression
: Function invocation schedule specified as a cron expression.--invoke-function-name
: Name of the function being invoked.--invoke-function-service-account-id
: ID of the service account with the permissions to invoke the function.
Result:
id: a1sv54ekvknb********
folder_id: b1g9d2k0itu4********
created_at: "2023-08-08T19:46:22.860681482Z"
...
function_tag: $latest
service_account_id: ajeh2dukocg3********
status: ACTIVE
To create a trigger that launches a function:
-
In the configuration file, describe the
timer
trigger parameters:resource "yandex_function_trigger" "timer" { name = "timer" timer { cron_expression = "* * ? * * *" } function { id = "<function_ID>" service_account_id = "<service_account_ID>" } }
Where:
name
: Trigger name.cron_expression
: Function invocation schedule specified as a cron expression.id
: ID of the function for the trigger to invoke.service_account_id
: ID of the service account with the permissions to invoke the function.
For more information about resource parameters in Terraform, see the provider documentation
. -
Make sure the configuration files are correct.
-
In the command line, go to the folder where you created the configuration file.
-
Run a check using this command:
terraform plan
If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.
-
-
Deploy cloud resources.
-
If the configuration does not contain any errors, run this command:
terraform apply
-
Confirm creating the resources: type
yes
in the terminal and press Enter.This will create a trigger named
timer
in the specified folder. You can check the new resources and their configuration using the management console or this CLI command:yc serverless trigger get timer
Result:
id: a1s4bvdvmod0******** folder_id: b1g9d2k0itu4******** created_at: "2023-08-09T10:19:12.356Z" ... function_id: d4ebrmenrr7l******** service_account_id: ajeh2dukocg3******** status: ACTIVE
-
To create a timer, use the create REST API method for the Trigger resource or the TriggerService/Create gRPC API call.
Test the function
- In the management console
, go to the folder your preemptible VM was created in. - In the list of services, select Compute Cloud.
- In the left-hand panel, select Virtual machines.
- Click
next to the VM name and select Stop. - In the window that opens, click Stop. The VM status will change to
Stopped
. - Check the VM status in about one minute or later. The VM status should change to
Running
.
How to delete the resources you created
To stop paying for the resources you created:
- Delete the trigger.
- Delete the function.
- Delete the secret.
- Delete the VM.
- If you logged data to a log group, delete the group.