Managing backups in Managed Service for Elasticsearch
Warning
Yandex Managed Service for Elasticsearch is unavailable as of April 11, 2024.
You can create an OpenSearch cluster in Yandex Cloud as an alternative to Elasticsearch.
Managed Service for Elasticsearch enables you to create index backups using both Yandex Cloud tools and the Elasticsearch snapshot mechanism. For more information about snapshots, see the Elasticsearch documentation
Creating backups with Yandex Cloud tools
You can create backups and restore clusters from existing backups.
Managed Service for Elasticsearch also creates automatic hourly backups.
Getting a list of backups
To get a list of cluster backups:
- Go to the folder page and select Managed Service for Elasticsearch.
- Click the cluster name and open the Backups tab.
To get a list of all backups in a folder:
- Go to the folder page and select Managed Service for Elasticsearch.
- Select the Backups tab.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To get a list of Elasticsearch cluster backups available in the default folder, run the command:
yc managed-elasticsearch backup list
Result:
+----------------------+----------------------+----------------------+----------------------+
| ID | CREATED AT | SOURCE CLUSTER ID | STARTED AT |
+----------------------+----------------------+----------------------+----------------------+
| c9qlk4v13uq7******** | 2020-08-10T12:00:00Z | c9qlk4v13uq7******** | 2020-08-10T11:55:17Z |
| c9qpm90p3pcg******** | 2020-08-09T22:01:04Z | c9qpm90p3pcg******** | 2020-08-09T21:30:00Z |
+----------------------+----------------------+----------------------+----------------------+
To get a list of cluster backups, use the listBackups REST API method for the Cluster resource or the ClusterService/ListBackups gRPC API call and provide the cluster ID in the clusterId
parameter of your request.
To get a list of backups for all the Managed Service for Elasticsearch clusters in the folder, use the list REST API method for the Backup resource or the BackupService/List gRPC API call and provide the folder ID in the folderId
parameter of your request.
You can get the cluster ID with a list of clusters in the folder.
Getting information about backups
To get information about the backup of an existing cluster:
- Go to the folder page and select Managed Service for Elasticsearch.
- Click the cluster name and open the Backups tab.
To get information about the backup of a previously deleted cluster:
- Go to the folder page and select Managed Service for Elasticsearch.
- Select the Backups tab.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To get information about a Elasticsearch cluster backup, run the command:
yc managed-elasticsearch backup get <backup_ID>
You can retrieve the backup ID with a list of backups.
To get information about a backup, use the get REST API method for the Backup resource or the BackupService/Get gRPC API call and provide the backup ID in the backupId
parameter of your request.
To find out the ID, retrieve a list of backups.
Creating a backup
- Go to the folder page and select Managed Service for Elasticsearch.
- Click the cluster name and open the Backups tab.
- Click
Create backup.
The service will start creating a backup without an additional confirmation.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To create a cluster backup:
-
View a description of the CLI create Elasticsearch backup command:
yc managed-elasticsearch cluster backup --help
-
Request a backup to be created by specifying the cluster name or ID:
yc managed-elasticsearch cluster backup <cluster_name_or_ID>
You can get the cluster ID and name with a list of clusters.
To create a backup, use the backup REST API method for the Cluster resource or the ClusterService/Backup gRPC API call and provide the cluster ID in the clusterId
parameter of your request.
You can get the cluster ID with a list of clusters in the folder.
Warning
While you are creating your backup, the cluster performance might degrade.
Restoring clusters from backups
When you restore a cluster from a backup, you create a new cluster with the backup data. If the folder has insufficient resources to create such a cluster, you will not be able to restore from the backup. You can manage the restore speed using Elasticsearch tools
When creating a new cluster, set all required parameters.
To restore an existing cluster from a backup:
- Go to the folder page and select Managed Service for Elasticsearch.
- Click the cluster name and open the Backups tab.
- Click
for the backup you need, then click Restore cluster. - Set up the new cluster. You can select a folder for the new cluster from the Folder list.
- Click Restore cluster.
To restore a previously deleted cluster from a backup:
- Go to the folder page and select Managed Service for Elasticsearch.
- Select the Backups tab.
- Find the backup you need using the backup creation time and cluster ID. The Name column contains IDs formatted as
<cluster_ID>:<backup_ID>
. - Click
for the backup you need, then click Restore cluster. - Set up the new cluster. You can select a folder for the new cluster from the Folder list.
- Click Restore cluster.
Managed Service for Elasticsearch will launch the operation to create a cluster from the backup.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To restore a cluster from a backup:
-
View a description of the CLI restore Elasticsearch cluster command:
yc managed-elasticsearch cluster restore --help
-
Getting a list of available Elasticsearch cluster backups:
yc managed-elasticsearch backup list
Result:
+--------------------------+----------------------+----------------------+----------------------+ | ID | CREATED AT | SOURCE CLUSTER ID | STARTED AT | +--------------------------+----------------------+----------------------+----------------------+ | c9qlk4v13uq79r9******** | 2020-08-10T12:00:00Z | c9qlk4v13uq7******** | 2020-08-10T11:55:17Z | | ... | +--------------------------+----------------------+----------------------+----------------------+
The backup completion time is shown in the
CREATED AT
column of the list of available backups, inyyyy-mm-ddThh:mm:ssZ
format (2020-08-10T12:00:00Z
in the example above). -
Request the creation of a cluster from a backup:
yc managed-elasticsearch cluster restore \ --backup-id=<backup_ID> \ --name=<cluster_name> \ --environment=<environment> \ --network-name=<network_name> \ --host zone-id=<availability_zone>,` `subnet-id=<subnet_ID>,` `assign-public-ip=<public_access>,` `type=<host_role> \ --datanode-resource-preset=<Data_node_host_class> \ --datanode-disk-size=<Data_node_storage_size_in_GB> \ --datanode-disk-type=<Data_node_disk_type> \ --masternode-resource-preset=<Master_node_host_class> \ --masternode-disk-size=<Master_node_storage_size_in_GB> \ --masternode-disk-type=<Master_node_disk_type> \ --admin-password=<admin_user_password>
Where:
-
--backup-id
: Backup ID. -
--name
: Cluster name. -
--environment
: Environment:PRESTABLE
: For testing purposes. The prestable environment is similar to the production environment and likewise covered by the SLA, but it is the first to get new functionalities, improvements, and bug fixes. In the prestable environment, you can test compatibility of new versions with your application.PRODUCTION
: For stable versions of your apps.
-
--network-name
: Network name. -
--host
: Host parameters:-
zone-id
: Availability zone. -
subnet-name
: Subnet name. Specify if two or more subnets are created in the selected availability zone. -
assign-public-ip
: Flag indicating that the host requires public access,true
orfalse
. -
type
: Host role:datanode
ormasternode
.
-
-
--datanode-resource-preset
: Class of hosts with the Data node role. -
--datanode-disk-size
: Storage size in GB for hosts with the Data node role. -
--datanode-disk-type
: Disk type for hosts with the Data node role:network-hdd
network-ssd
local-ssd
network-ssd-nonreplicated
-
--masternode-resource-preset
: Class of hosts with the Master node role. -
--masternode-disk-size
: Storage size in GB for hosts with the Master node role. -
--masternode-disk-type
: Disk type for hosts with the Master node role. Onlynetwork-ssd
is available. -
--admin-password
:admin
user password.This is a special user that is required for managing clusters and cannot be deleted. It is assigned the
superuser
role and can perform any operations on clusters.Tip
To perform routine tasks, we recommend creating individual users. For more information, see Managing users.
-
To restore a cluster from a backup, use the restore REST API method for the Cluster resource or the ClusterService/Restore gRPC API call and provide the following in the request:
- ID of the backup you need, in the
backupId
parameter. To find out the ID, get a list of cluster backups. - Name of the new cluster that will contain the data recovered from the backup, in the
name
parameter. It must be unique within the folder. - Cluster environment in the
environment
parameter. - Cluster configuration in the
configSpec
parameter. - Configuration of the cluster hosts in one or more
hostSpecs
parameters. - Network ID in the
networkId
parameter.
Backups using snapshots
To work with snapshots, use the Elasticsearch public API
Retrieving a snapshot list
-
Find the repository containing snapshot backups in the Elasticsearch repository list:
GET https://admin:<password>@<host_FQDN>:9200/_snapshot/_all
If the repository you need is not on the list, connect it.
-
Get a list of snapshots in the repository:
GET https://admin:<password>@<host_FQDN>:9200/_snapshot/<repository>/_all
Each snapshot is a single backup.
Creating a snapshot
-
In the Elasticsearch repository list, find the repository where you want to create the snapshot:
GET https://admin:<password>@<host_FQDN>:9200/_snapshot/_all
If the repository you need is not on the list, connect it.
-
Create a snapshot
of the required data or cluster in the selected repository:PUT https://admin:<password>@<host_FQDN>:9200/_snapshot/<repository>/<snapshot>
Restoring a cluster from a snapshot
Warning
When restoring from snapshots, the following restrictions apply:
- The Elasticsearch version in the cluster must be equal to or higher than the Elasticsearch version where the snapshot was taken.
- To restore the indexes, be sure that the major Elasticsearch version in the cluster is no more than one unit higher than the major Elasticsearch version where the snapshot was taken. For example, you can use version 6.0 (but not 7.0) to recover indexes created in version 5.0.
-
Create a new Elasticsearch cluster in the required configuration, but do not populate it with data.
When creating a cluster, select:
-
The number and class of hosts as well as the storage size and disk type based on snapshot size and performance requirements. If necessary, upgrade your host class or increase cluster storage size.
-
The Elasticsearch version used to make the snapshot or higher.
-
-
Close the open indexes using the Elasticsearch API
:POST: https://admin:<password>@<host_FQDN>:9200/<index>/_close
To restore an entire cluster, close all open indexes. To restore individual indexes, close only those indexes.
-
Retrieve a list of backups and find the required snapshot.
-
Start restoring
an entire cluster or individual data indexes and streams from the required snapshot.
For more information about restoring from snapshots, see the Elasticsearch documentation