Deploying a fault-tolerant architecture with preemptible VMs

Written by

Updated at October 18, 2024

Getting started
- Required paid resources
Prepare the environment
Create a secret
Prepare a ZIP archive with the function code
Create a function
Create a trigger
Test the function
How to delete the resources you created

In this tutorial, you will create a Yandex Cloud Functions function in Node.js that will be invoked on a schedule and restart a stopped preemptible Yandex Compute Cloud VM.

The architecture described here is suitable for VMs with non-critical loads. It allows you to use cost advantages of preemptible VMs and, in the event of a VM shutdown, ensures idle time of under one minute.

To deploy a fault-tolerant architecture with a preemptible VM:

If you no longer need the resources you created, delete them.

Getting started

Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resources

The infrastructure support costs include:

Fee for VM computing resources (see Compute Cloud pricing).
Fee for VM disks (see Compute Cloud pricing).
Fee for using a dynamic or static public IP address (see Yandex Virtual Private Cloud pricing).
Secret storage and request fees (see Yandex Lockbox pricing).
Fee for the number of function calls, computing resources allocated to a function, and outgoing traffic (see Cloud Functions pricing).
Fee for logging operations and data storage in a log group (see Yandex Cloud Logging pricing) if using Cloud Logging.

Prepare the environment

Create a service account that will be used to invoke the function and assign it the functions.functionInvoker and lockbox.payloadViewer roles.
Create a preemptible VM.

Create a secret

Create a Yandex Lockbox secret to store an OAuth token.

Note

Use an OAuth token if you cannot request an IAM token automatically. The IAM token is updated more frequently and is therefore more secure.

Management console

CLI

Terraform

API

In the management console, select the folder you want to create a secret in.
In the list of services, select Lockbox.
Click Create secret.
In the Name field, enter a name for the secret, e.g., oauth-token.
Under Version:
- In the Key field, enter key_token.
- In the Value field, enter the OAuth token value required for function authorization.
Click Create.

To create a secret, run this command:

yc lockbox secret create --name oauth-token \
  --payload "[{'key': 'key_token', 'text_value': '<OAuth_token>'}]"

Where text_value is the OAuth token value required for function authorization.

Result:

done (1s)
id: e6qu9ik259lb********
folder_id: b1g9d2k0itu4********
...
  status: ACTIVE
  payload_entry_keys:
    - key_token

In the configuration file, describe the secret parameters:
```
resource "yandex_lockbox_secret" "oauth-token" {
  name = "oauth-token"
}

resource "yandex_lockbox_secret_version" "my_version" {
  secret_id = yandex_lockbox_secret.my_secret.id
  entries {
    key        = "key_token"
    text_value = "<OAuth token>"
  }
}
```
Where:
- name: Secret name
- key: Key of the secret.
- text_value: OAuth token value required for function authorization.
Note

We recommend using yandex_lockbox_secret_version_hashed: it stores values in Terraform state in hashed format. We continue supporting yandex_lockbox_secret_version.

For more information about yandex_lockbox_secret_version_hashed, see the relevant provider documentation.

For more information about the parameters of resources used in Terraform, see the provider documentation:
- yandex_lockbox_secret.
- yandex_lockbox_secret_version.
Make sure the configuration files are correct.
1. In the command line, go to the folder where you created the configuration file.
2. Run a check using this command:
```
terraform plan
```
If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.
Deploy cloud resources.
1. If the configuration does not contain any errors, run this command:
```
terraform apply
```
2. Confirm the secret creation: type yes in the terminal and press Enter.

To create a secret, use the create REST API method for the Secret resource or the SecretService/Create gRPC API call.

Prepare a ZIP archive with the function code

Save the following code to a file named index.js:

import { serviceClients, Session, cloudApi } from '@yandex-cloud/nodejs-sdk';

const {
  compute: {
    instance_service: {
      ListInstancesRequest,
      GetInstanceRequest,
      StartInstanceRequest,
    },
    instance: {
      IpVersion,
    },
  },
} = cloudApi;

const FOLDER_ID = process.env.FOLDER_ID;
const INSTANCE_ID = process.env.INSTANCE_ID;
const OAUTHTOKEN = process.env.OAUTHTOKEN;

export const handler = async function (event, context) {
  const session = new Session({ oauthToken: OAUTHTOKEN });
  const instanceClient = session.client(serviceClients.InstanceServiceClient);
  const list = await instanceClient.list(ListInstancesRequest.fromPartial({
    folderId: FOLDER_ID,
  }));
  const state = await instanceClient.get(GetInstanceRequest.fromPartial({
    instanceId: INSTANCE_ID,
  }));

  var status = state.status

  if (status == 4){
    const startcommand = await instanceClient.start(StartInstanceRequest.fromPartial({
      instanceId: INSTANCE_ID,
    }));
  }

  return {
    statusCode: 200,
    body: {
      status
    }
  };
};

Save the following code to a file named package.json:

{
  "name": "my-awesome-package",
  "version": "1.0.0",
  "type": "module",
  "dependencies": {
    "@yandex-cloud/nodejs-sdk": "latest"
  }
}

Add the index.js and package.json files into the function-js.zip archive.

Create a function

Management console

CLI

Terraform

API

In the management console, select the folder where you want to create a function.
In the list of services, select Cloud Functions.
Create a function:
1. Click Create function.
2. In the window that opens, enter function-restart-vms as the function name.
3. Click Create.
Create a function version:
1. Select the nodejs18 runtime environment, disable the Add files with code examples option, and click Continue.
2. In the Method field, select ZIP archive.
3. In the File field, click Attach file and select the function-js.zip archive you created earlier.
4. Specify the entry point: index.handler.
5. Under Parameters, specify:
  - Timeout, sec: 3.
  - Memory: 128 MB.
  - Service account: Select the previously created service account with rights to invoke the function.
  - Environment variables:
    - FOLDER_ID: ID of the folder you want to start the stopped VM instances in.
    - INSTANCE_ID: ID of the VM instance you want to start at interruption.
  - Lockbox secrets:
    - In the Environment variable field, specify OAUTHTOKEN.
    - In the Secret ID field, select the previously created oauth-token secret.
    - In the Version ID field, select a version for the secret.
    - In the Secret key field, select key_token as the key name.
  - If you want to avoid logging and paying for Cloud Logging, disable logging by selecting Not specified in the Destination field under Logging.
6. Click Save changes.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

Create a function named function-restart-vms:

yc serverless function create --name function-restart-vms

Result:

id: d4ebrmenrr7l********
folder_id: b1g9d2k0itu4********
created_at: "2023-10-28T17:26:58.200799757Z"
name: function-restart-vms
http_invoke_url: https://functions.yandexcloud.net/d4ebrmenrr7l********
status: ACTIVE

Create a version of the function-restart-vms function:
```
yc serverless function version create \
  --function-name function-restart-vms \
  --memory=128m \
  --execution-timeout=3s \
  --runtime=nodejs18 \
  --entrypoint=index.handler \
  --service-account-id=<service_account_ID> \
  --environment FOLDER_ID=<folder_ID>,INSTANCE_ID=<VM_ID> \
  --secret name=oauth-token,version-id=<secret_version_ID>,key=key_token,environment-variable=OAUTHTOKEN \
  --source-path=./function-js.zip \
  --no-logging
```
Where:
- --function-name: Name of the function a version of which you are creating.
- --memory: Amount of RAM.
- --execution-timeout: Maximum function running time before the timeout is reached.
- --runtime: Runtime environment.
- --entrypoint: Entry point.
- --service-account-id: ID of the service account with the permissions to invoke the function.
- --environment: Environment variables:
  - FOLDER_ID: ID of the folder you want to start the stopped VM instances in.
  - INSTANCE_ID: ID of the VM instance you want to start at interruption.
- --secret: Yandex Lockbox secret data:
  - name: Secret name
  - version-id: Secret version ID.
  - key: Key of the secret.
  - environment-variable: Environment variable the secret will be kept in.
- --source-path: Path to the function-js.zip archive you created earlier.
- (Optional) --no-logging: Set this flag to avoid logging and paying for Cloud Logging.
Result:
```
done (16s)
id: d4etv5f4sjet********
function_id: d4ebrmenrr7l********
...
log_options:
  disabled: true
  folder_id: b1g9d2k0itu4********
```

If you do not have Terraform yet, install it and configure the Yandex Cloud provider.

In the configuration file, describe the function-restart-vms function parameters and its versions:
```
resource "yandex_function" "function-restart-vms" {
  name               = "function-restart-vms"
  user_hash          = "first function"
  runtime            = "nodejs18"
  entrypoint         = "index.handler"
  memory             = "128"
  execution_timeout  = "3"
  service_account_id = "<service_account_ID>"
  folder_id = "<folder_ID>"
  environment = {
    FOLDER_ID = "<folder_ID>"
    INSTANCE_ID = "<VM_ID>"
  }
  secrets {
    id = "<secret_ID>"
    version_id = "<secret_version_ID>"
    key = "key_token"
    environment_variable = "OAUTHTOKEN"
  }
  content {
    zip_filename = "./function-js.zip"
  }
}
```
Where:
- name: Function name.
- user_hash: Any string to identify the function version.
- runtime: Function runtime environment.
- entrypoint: Entry point.
- memory: Amount of memory allocated for the function, in MB.
- execution_timeout: Function execution timeout.
- service_account_id: ID of the service account with the permissions to invoke the function.
- folder_id: ID of the folder you are creating the function in.
- environment: Environment variables:
  - FOLDER_ID: ID of the folder you want to start the stopped VM instances in.
  - INSTANCE_ID: ID of the VM instance you want to start at interruption.
- secrets: Yandex Lockbox secret data:
  - id: Secret ID.
  - version_id: Secret version ID.
  - key: Key of the secret.
  - environment_variable: Environment variable the secret will be kept in.
- zip_filename: Path to the function-js.zip archive you created earlier.
For more information about the yandex_function resource parameters, see the provider documentation.
Make sure the configuration files are correct.
1. In the command line, go to the folder where you created the configuration file.
2. Run a check using this command:
```
terraform plan
```
If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.
Deploy cloud resources.
1. If the configuration does not contain any errors, run this command:
```
terraform apply
```
2. Confirm creating the function: type yes in the terminal and press Enter.
  
  This will create a function named function-restart-vms in the specified folder. You can check the new resources and their configuration using the management console or this CLI command:
```
yc serverless function get function-restart-vms
```
  Result:
```
id: d4ees84gsdsd********
folder_id: b1g9d2k0itu4********
created_at: "2023-08-09T10:11:40.740Z"
name: function-restart-vms
log_group_id: ckgjitlio5aj********
http_invoke_url: https://functions.yandexcloud.net/d4ees84gsdsd********
status: ACTIVE
```

To create a function, use the create REST API method for the Function resource or the FunctionService/Create gRPC API call.

To create a function version, use the createVersion REST API method for the Function resource or the FunctionService/CreateVersion gRPC API call.

Create a trigger

Note

The trigger is initiated within 5 minutes of being created.

Management console

CLI

Terraform

API

In the management console, select the folder where you want to create a trigger.
In the list of services, select Cloud Functions.
In the left-hand panel, select Triggers.
Click Create trigger.
Under Basic settings:
- Enter a name for the trigger: timer.
- In the Type field, select Timer.
- In the Launched resource field, select Function.
Under Timer settings, enter * * ? * * * or select Every minute.
Under Function settings, select function-restart-vms and specify the following:
- Function version tag: $latest.
- Previously created service account with rights to invoke the function.
Click Create trigger.

To create a trigger that invokes a function, run this command:

yc serverless trigger create timer \
  --name timer \
  --cron-expression '* * ? * * *' \
  --invoke-function-name function-restart-vms \
  --invoke-function-service-account-id <service_account_ID>

Where:

--name: Trigger name.
--cron-expression: Function invocation schedule specified as a cron expression.
--invoke-function-name: Name of the function being invoked.
--invoke-function-service-account-id: ID of the service account with the permissions to invoke the function.

Result:

id: a1sv54ekvknb********
folder_id: b1g9d2k0itu4********
created_at: "2023-08-08T19:46:22.860681482Z"
...
      function_tag: $latest
      service_account_id: ajeh2dukocg3********
status: ACTIVE

To create a trigger that launches a function:

In the configuration file, describe the timer trigger parameters:
```
resource "yandex_function_trigger" "timer" {
  name        = "timer"
  timer {
    cron_expression = "* * ? * * *"
  }
  function {
    id = "<function_ID>"
    service_account_id = "<service_account_ID>"
  }
}
```
Where:
- name: Trigger name.
- cron_expression: Function invocation schedule specified as a cron expression.
- id: ID of the function for the trigger to invoke.
- service_account_id: ID of the service account with the permissions to invoke the function.
For more information about resource parameters in Terraform, see the provider documentation.
Make sure the configuration files are correct.
1. In the command line, go to the folder where you created the configuration file.
2. Run a check using this command:
```
terraform plan
```
If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.
Deploy cloud resources.
1. If the configuration does not contain any errors, run this command:
```
terraform apply
```
2. Confirm creating the resources: type yes in the terminal and press Enter.
  
  This will create a trigger named timer in the specified folder. You can check the new resources and their configuration using the management console or this CLI command:
```
yc serverless trigger get timer
```
  Result:
```
id: a1s4bvdvmod0********
folder_id: b1g9d2k0itu4********
created_at: "2023-08-09T10:19:12.356Z"
...
      function_id: d4ebrmenrr7l********
      service_account_id: ajeh2dukocg3********
status: ACTIVE
```

To create a timer, use the create REST API method for the Trigger resource or the TriggerService/Create gRPC API call.

Test the function

Management console

In the management console, go to the folder your preemptible VM was created in.
In the list of services, select Compute Cloud.
In the left-hand panel, select Virtual machines.
Click next to the VM name and select Stop.
In the window that opens, click Stop. The VM status will change to Stopped.
Check the VM status in about one minute or later. The VM status should change to Running.

How to delete the resources you created

To stop paying for the resources you created:

Delete the trigger.
Delete the function.
Delete the secret.
Delete the VM.
If you logged data to a log group, delete the group.

Deploying a fault-tolerant architecture with preemptible VMs

Getting startedGetting started

Required paid resourcesRequired paid resources

Prepare the environmentPrepare the environment

Create a secretCreate a secret

Prepare a ZIP archive with the function codePrepare a ZIP archive with the function code

Create a functionCreate a function

Create a triggerCreate a trigger

Test the functionTest the function

How to delete the resources you createdHow to delete the resources you created

Was the article helpful?

Getting started

Required paid resources

Prepare the environment

Create a secret

Prepare a ZIP archive with the function code

Create a function

Create a trigger

Test the function

How to delete the resources you created