Managing backups in Managed Service for OpenSearch

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

To get a list of all backups in the folder, run this command:

yc managed-opensearch backup list

Result:

+----------------------+---------------------+-------------------+---------------------+
|          ID          |      CREATED AT     | SOURCE CLUSTER ID |      STARTED AT     |
+----------------------+---------------------+-------------------+---------------------+
| c9qlk4v13uq7******** | 2024-01-09 14:38:34 | c9qpm4i********   | 2024-01-09 14:38:28 |
| c9qpm90p3pcg******** | 2024-01-09 13:38:31 | c9qpm4i********   | 2024-01-09 13:38:28 |
+----------------------+---------------------+-------------------+---------------------+

If you want to limit the backup list displayed after running the command, include the --limit <number_of_records> parameter in the command. For example, if the output of the yc managed-opensearch backup list command takes up several screens, run the yc managed-opensearch backup list --limit 5 command. In this case, the output will contain the list of five most recent backups.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. To get a list of cluster backups:

   1. Call the Cluster.ListBackups method, e.g., via the following cURL request:

      curl \
          --request GET \
          --header "Authorization: Bearer $IAM_TOKEN" \
          --url 'https://mdb.api.cloud.yandex.net/managed-opensearch/v1/clusters/<cluster_ID>/backups'
      

      You can request the cluster ID with the list of clusters in the folder.

   2. Check the server response to make sure your request was successful.

3. To get the list of backups for all clusters in the folder:

   1. Call the Backup.List method, e.g., via the following cURL request:

      curl \
          --request GET \
          --header "Authorization: Bearer $IAM_TOKEN" \
          --url 'https://mdb.api.cloud.yandex.net/managed-opensearch/v1/backups' \
          --url-query folderId=<folder_ID>
      

      You can get the folder ID with the list of folders in the cloud.

   2. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. To get a list of cluster backups:

   1. Use the ClusterService.ListBackups call and send the following request, e.g., via gRPCurl:

      grpcurl \
          -format json \
          -import-path ~/cloudapi/ \
          -import-path ~/cloudapi/third_party/googleapis/ \
          -proto ~/cloudapi/yandex/cloud/mdb/opensearch/v1/cluster_service.proto \
          -rpc-header "Authorization: Bearer $IAM_TOKEN" \
          -d '{
                "cluster_id": "<cluster_ID>"
              }' \
          mdb.api.cloud.yandex.net:443 \
          yandex.cloud.mdb.opensearch.v1.ClusterService.ListBackups
      

      You can request the cluster ID with the list of clusters in the folder.

   2. Check the server response to make sure your request was successful.

4. To get the list of backups for all clusters in the folder:

   1. Use the BackupService.List call and send the following request, e.g., via gRPCurl:

      grpcurl \
          -format json \
          -import-path ~/cloudapi/ \
          -import-path ~/cloudapi/third_party/googleapis/ \
          -proto ~/cloudapi/yandex/cloud/mdb/opensearch/v1/backup_service.proto \
          -rpc-header "Authorization: Bearer $IAM_TOKEN" \
          -d '{
                "folder_id": "<folder_ID>"
              }' \
          mdb.api.cloud.yandex.net:443 \
          yandex.cloud.mdb.opensearch.v1.BackupService.List
      

      You can get the folder ID with the list of folders in the cloud.

   2. Check the server response to make sure your request was successful.

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

To get information about the backup of a cluster:

1. Retrieve the backup ID with a list of all backups in the folder:

   yc managed-opensearch backup list
   

   You will see the ID in the ID column of the command output.

2. Get information about the backup you need:

   yc managed-opensearch backup get <backup_ID>
   

   Command output example:

   id: c9qlk4v13uq7********
   folder_id: b1g86q4m5vej********
   source_cluster_id: c9qpm4i********
   started_at: "2024-01-09T10:38:28.683Z"
   created_at: "2024-01-09T10:38:31.685Z"
   indices:
     - .mdb-sli
     - .opendistro_security
     - .kibana_1
     - .opendistro-job-scheduler-lock
     - .opensearch-observability
     - .opendistro-ism-config
   opensearch_version: 2.8.0
   indices_total: "6"
   

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Call the Backup.Get method, e.g., via the following cURL request:

   curl \
       --request GET \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --url 'https://mdb.api.cloud.yandex.net/managed-opensearch/v1/backups/<backup_ID>'
   

   You can request the backup ID together with the list of backups.

3. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the BackupService.Get call and send the following request, e.g., via gRPCurl:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/opensearch/v1/backup_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
             "backup_id": "<backup_ID>"
           }' \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.opensearch.v1.BackupService.Get
   

   You can request the backup ID together with the list of backups.

4. Check the server response to make sure your request was successful.

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

To create a backup of cluster data, run this command:

yc managed-opensearch cluster backup <cluster_name_or_ID>

You can request the cluster name and ID with the list of clusters in the folder.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Call the Cluster.Backup method, e.g., via the following cURL request:

   curl \
       --request POST \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --header "Content-Type: application/json" \
       --url 'https://mdb.api.cloud.yandex.net/managed-opensearch/v1/clusters/<cluster_ID>:backup'
   

   You can request the cluster ID with the list of clusters in the folder.

3. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService.Backup call and send the following request, e.g., via gRPCurl:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/opensearch/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
             "cluster_id": "<cluster_ID>"
           }' \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.opensearch.v1.ClusterService.Backup
   

   You can request the cluster ID with the list of clusters in the folder.

4. View the server response to make sure your request was successful.

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

To restore a cluster from a backup:

1. Retrieve the backup ID with a list of all backups in the folder:

   yc managed-opensearch backup list
   

   Result:

   +----------------------+---------------------+-------------------+---------------------+
   |          ID          |      CREATED AT     | SOURCE CLUSTER ID |      STARTED AT     |
   +----------------------+---------------------+-------------------+---------------------+
   | c9qlk4v13uq7******** | 2024-01-09 14:38:34 | c9qpm4i********   | 2024-01-09 14:38:28 |
   | ...                                                                                  |
   +----------------------+---------------------+-------------------+---------------------+
   

   You will see the backup ID in the ID column. The CREATED AT column shows the backup completion time in yyyy-mm-dd hh:mm:ss format.

2. Send the following request to create a cluster from a backup:

   yc managed-opensearch cluster restore --backup-id <backup_ID>
   

   You can also run the command with the parameters you use when creating a cluster. For the description of such parameters, see Creating a cluster.

1. Get an IAM token for API authentication and set it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Create a file named body.json and paste the following code into it:

   {
       "backupId": "<backup_ID>",
       "folderId": "<folder_ID>",
       "name": "<cluster_name>",
       "environment": "<environment>",
       "networkId": "<network_ID>",
       "configSpec": {
           "version": "<OpenSearch_version>",
           "adminPassword": "<admin_password>",
           "opensearchSpec": {
               "nodeGroups": [
                   {
                       "name": "<OpenSearch_host_group_name>",
                       "resources": {
                           "resourcePresetId": "<host_class>",
                           "diskSize": "<storage_size_in_bytes>",
                           "diskTypeId": "<disk_type>"
                       },
                       "roles": ["<role_1>","<role_2>"],
                       "hostsCount": "<number_of_hosts>",
                       "zoneIds": [
                           "<availability_zone_1>",
                           "<availability_zone_2>",
                           "<availability_zone_3>"
                       ],
                       "subnetIds": [
                           "<subnet_1_ID>",
                           "<subnet_2_ID>",
                           "<subnet_3_ID>"
                       ]
                   }
               ]
           },
           "dashboardsSpec": {
               "nodeGroups": [
                   {
                       "name": "<Dashboards_host_group_name>",
                       "resources": {
                           "resourcePresetId": "<host_class>",
                           "diskSize": "<storage_size_in_bytes>",
                           "diskTypeId": "<disk_type>"
                       },
                       "hostsCount": "<number_of_hosts>",
                       "zoneIds": [
                           "<availability_zone_1>",
                           "<availability_zone_2>",
                           "<availability_zone_3>"
                       ],
                       "subnetIds": [
                           "<subnet_1_ID>",
                           "<subnet_2_ID>",
                           "<subnet_3_ID>"
                       ]
                   }
               ]
           }
       }
   }
   

   Where:

   • backupId: ID of the backup copy you are restoring the cluster from. You can get the ID together with a list of backups.

   • folderId: Folder ID. You can get it with the list of folders in the cloud.

   • name: Cluster name.

   • environment: Cluster environment, PRODUCTION or PRESTABLE.

   • networkId: ID of the network where the cluster will be deployed.

   • configSpec: Cluster settings:

     • version: OpenSearch version.

     • adminPassword: admin user password.

       The password must contain three groups of characters out of these four:

       • Lowercase Latin letters
       • Uppercase Latin letters
       • Numbers
       • Special characters

       The password must be from 10 to 72 characters long.

     • opensearchSpec: OpenSearch host group settings. Contain an array of nodeGroups elements. one for each host group. Each element has the following structure:

       • name: Host group name.

       • resources: Cluster resources:

         • resourcePresetId: Host class.
         • diskSize: Disk size, in bytes.
         • diskTypeId: Disk type.

       • roles: List of host roles. A cluster must include at least one group of DATA hosts and one group of MANAGER hosts. This can be a single group with two roles or several groups with different roles.

       • hostsCount: Number of hosts in the group. Minimum number of DATA hosts: one; minimum number of MANAGER hosts: three.

       • zoneIds: List of availability zones the cluster hosts are located in.

       • subnetIds: List of subnet IDs.

     • dashboardsSpec: Dashboards host group settings. Contains the nodeGroups array of elements of the same structure as opensearchSpec.nodeGroups. The roles parameter is the exception: the Dashboards hosts can only have one role, DASHBOARDS, so there is no need to specify it.

3. Call the Cluster.Restore method, e.g., via the following cURL request:

   curl \
       --request POST \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --header "Content-Type: application/json" \
       --url 'https://mdb.api.cloud.yandex.net/managed-opensearch/v1/clusters:restore' \
       --data "@body.json"
   

4. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Create a file named body.json and paste the following code into it:

   {
       "backup_id": "<backup_ID>",
       "folder_id": "<folder_ID>",
       "name": "<cluster_name>",
       "environment": "<environment>",
       "network_id": "<network_ID>",
       "config_spec": {
           "version": "<OpenSearch_version>",
           "admin_password": "<admin_password>",
           "opensearch_spec": {
               "node_groups": [
                   {
                       "name": "<OpenSearch_host_group_name>",
                       "resources": {
                           "resource_preset_id": "<host_class>",
                           "disk_size": "<storage_size_in_bytes>",
                           "disk_type_id": "<disk_type>"
                       },
                       "roles": ["<role_1>","<role_2>"],
                       "hosts_count": "<number_of_hosts>",
                       "zone_ids": [
                           "<availability_zone_1>",
                           "<availability_zone_2>",
                           "<availability_zone_3>"
                       ],
                       "subnet_ids": [
                           "<subnet_1_ID>",
                           "<subnet_2_ID>",
                           "<subnet_3_ID>"
                       ]
                   }
               ]
           },
           "dashboards_spec": {
               "node_groups": [
                   {
                       "name": "<Dashboards_host_group_name>",
                       "resources": {
                           "resource_preset_id": "<host_class>",
                           "disk_size": "<storage_size_in_bytes>",
                           "disk_type_id": "<disk_type>"
                       },
                       "hosts_count": "<number_of_hosts>",
                       "zone_ids": [
                           "<availability_zone_1>",
                           "<availability_zone_2>",
                           "<availability_zone_3>"
                       ],
                       "subnet_ids": [
                           "<subnet_1_ID>",
                           "<subnet_2_ID>",
                           "<subnet_3_ID>"
                       ]
                   }
               ]
           }
       }
   }
   

   Where:

   • backup_id: ID of the backup copy you are restoring the cluster from. You can get the ID together with a list of backups.

   • folder_id: Folder ID. You can request it with the list of folders in the cloud.

   • name: Cluster name.

   • environment: Cluster environment, PRODUCTION or PRESTABLE.

   • network_id: ID of the network where the cluster will be deployed.

   • config_spec: Cluster settings:

     • version: OpenSearch version.

     • admin_password: admin user password.

       The password must contain three groups of characters out of these four:

       • Lowercase Latin letters
       • Uppercase Latin letters
       • Numbers
       • Special characters

       The password must be from 10 to 72 characters long.

     • opensearch_spec: OpenSearch host group settings. Contain an array of nodeGroups elements. One for each host group. Each element has the following structure:

       • name: Host group name.

       • resources: Cluster resources:

         • resource_preset_id: Host class.
         • disk_size: Disk size, in bytes.
         • disk_type_id: Disk type.

       • roles: List of host roles. A cluster must include at least one group of DATA hosts and one group of MANAGER hosts. This can be a single group with two roles or several groups with different roles.

       • hosts_count: Number of hosts in the group. Minimum number of DATA hosts: one; minimum number of MANAGER hosts: three.

       • zone_ids: List of availability zones the cluster hosts are located in.

       • subnet_ids: List of subnet IDs.

     • dashboards_spec: Dashboards host group settings. Contains the node_groups array of elements of the same structure as opensearch_spec.node_groups. The roles parameter is the exception: the Dashboards hosts can only have one role, DASHBOARDS, so there is no need to specify it.

4. Use the ClusterService.Restore call and make a request, e.g., via gRPCurl:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/opensearch/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d @ \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.opensearch.v1.ClusterService.Restore \
       < body.json
   

5. Check the server response to make sure your request was successful.