Managing backups in Yandex MPP Analytics for PostgreSQL
You can view your existing backups and restore clusters from them.
Getting a list of backups
To get the list of cluster backups:
- Navigate to the folder dashboard
and select Yandex MPP Analytics for PostgreSQL. - Click the name of your cluster and open the
Backups tab.
To get the list of all backups in the folder:
- Navigate to the folder dashboard
and select Yandex MPP Analytics for PostgreSQL. - In the left-hand panel, select
Backups.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To get a list of Greenplum® cluster backups, run this command:
yc managed-greenplum cluster list-backups <cluster_name_or_ID>
You can get the cluster ID and name with the list of clusters in the folder.
Result:
+--------------------------+---------------------+----------------------+---------------------+
| ID | CREATED AT | SOURCE CLUSTER ID | STARTED AT |
+--------------------------+---------------------+----------------------+---------------------+
| c9qgo11pud7k********:... | 2020-08-10 12:00:00 | c9qgo11pud7k******** | 2020-08-10 11:55:17 |
| ... |
+--------------------------+---------------------+----------------------+---------------------+
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
To get a list of cluster backups:
-
Call the Cluster.ListBackups method, e.g., via the following cURL
request:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters/<cluster_ID>/backups'You can request the cluster ID with the list of clusters in the folder.
-
Check the server response to make sure your request was successful.
-
-
To get a list of backups for all clusters in a folder:
-
Call the Backup.List method, e.g., via the following cURL
request:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/backups' \ --url-query folderId=<folder_ID>You can request the folder ID with the list of folders in the cloud.
-
Check the server response to make sure your request was successful.
-
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
To get a list of cluster backups:
-
Use the ClusterService.ListBackups call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.ClusterService.ListBackupsYou can request the cluster ID with the list of clusters in the folder.
-
Check the server response to make sure your request was successful.
-
-
To get a list of backups for all clusters in a folder:
-
Use the BackupService.List call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/backup_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "folder_id": "<folder_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.BackupService.ListYou can get the folder ID with the list of folders in the cloud.
-
Check the server response to make sure your request was successful.
-
Getting backup info
To get information about a backup of an existing cluster:
- Navigate to the folder dashboard
and select Yandex MPP Analytics for PostgreSQL. - Click the name of your cluster and open the
Backups tab.
To get information about a backup of a previously deleted cluster:
- Navigate to the folder dashboard
and select Yandex MPP Analytics for PostgreSQL. - In the left-hand panel, select
Backups.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Backup.Get method, e.g., via the following cURL
request:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/backups/<backup_ID>'You can request the backup ID with the list of backups.
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the BackupService.Get call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/backup_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "backup_id": "<backup_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.BackupService.GetYou can request the backup ID with the list of backups.
-
Check the server response to make sure your request was successful.
Creating a backup
- Navigate to the folder dashboard and select Yandex MPP Analytics for PostgreSQL.
- Click the name of your cluster and open the
Backups tab. - Click
Create backup.
The service will start creating a backup without an additional confirmation.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.Backup method, e.g., via the following cURL
request:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters/<cluster_ID>:backup'You can request the cluster ID with the list of clusters in the folder.
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService.Backup call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.ClusterService.BackupYou can request the cluster ID with the list of clusters in the folder.
-
View the server response to make sure your request was successful.
Warning
While you are creating your backup, the cluster performance might degrade.
Restoring a cluster from a backup
The point-in-time recovery (PITR) technology enables you to revert a cluster's state to any restore point created after saving a backup. For more information, see Backups.
When you restore a cluster from a backup, you create a new cluster with the backup data. If the folder lacks resources to create such a cluster, data will not be restored from the backup.
For a new cluster, you need to configure all settings that are required when creating it.
To migrate the hosts of a Yandex MPP Analytics for PostgreSQL cluster to a different availability zone, restore your cluster from a backup. When restoring a cluster from a backup, specify a new availability zone. If your cluster operates as a Yandex Data Transfer endpoint, create the endpoint and transfer again after restoring the cluster from a backup.
Warning
When restoring a cluster from a backup, there will be restrictions on the new cluster's configuration:
- The total number of segments must be the same as in the source cluster.
- The disk size per segment in the new cluster must be at least as large as in the source cluster.
Example
The source cluster has four segment hosts, each containing four segments. The total number of segments is 16. When restoring the cluster, you can choose two segment hosts with eight segments per host, so that the total number of segments remains 16.
To ensure that the disk size per segment does not decrease, the disk size in each segment host must at least double.
If you set the current time as the restore time, the new cluster will match the state of the latest available restore point.
Before you begin, assign to your Yandex Cloud account the managed-greenplum.restorer role or higher for the backup folder and the new cluster folder.
To restore an existing cluster from a backup:
-
Navigate to the folder dashboard
and select Yandex MPP Analytics for PostgreSQL. -
Click the name of your cluster and open the
Backups tab. -
Click
for the backup you need and click Restore cluster. -
Configure the new cluster. You can select a folder for the new cluster from the Folder list.
-
In the Date and time of recovery (UTC) setting, specify the time point to which you want to revert the cluster's state. You can enter a value manually or select one from the drop-down calendar. The system will select the restore point closest to that time.
If you do not edit this setting, the cluster's state will be restored from the backup. No restore points will be used.
-
If you want to restore only certain databases or tables, list them in the Databases and tables for recovery field. If you leave the field blank, the whole cluster will be restored.
-
In the Host count setting, specify the number of segment hosts.
-
In the Segments per host setting, specify the number of segments per host.
The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
Optionally, select groups of dedicated hosts to place master hosts or segment hosts on dedicated hosts. You can assign groups to one of the two Greenplum® host types or to both of them at once.
You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Yandex MPP Analytics for PostgreSQL.
-
Click Create.
To restore a previously deleted cluster from a backup:
-
Navigate to the folder dashboard
and select Yandex MPP Analytics for PostgreSQL. -
In the left-hand panel, select
Backups. -
Find the backup you need using the backup creation time and cluster ID. The ID column contains IDs in
<cluster_ID>:<backup_ID>format. -
Click
for the backup at hand and click Restore cluster. -
Configure the new cluster. You can select a folder for the new cluster from the Folder list.
-
In the Date and time of recovery (UTC) setting, specify the time point to which you want to revert the cluster's state. You can enter a value manually or select one from the drop-down calendar. The system will select the restore point closest to that time.
If you do not edit this setting, the cluster's state will be restored from the backup. No restore points will be used.
-
If you want to restore only certain databases or tables, list them in the Databases and tables for recovery field. If you leave the field blank, the whole cluster will be restored.
-
In the Host count setting, specify the number of segment hosts.
-
In the Segments per host setting, specify the number of segments per host.
The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
Optionally, select groups of dedicated hosts to place master hosts or segment hosts on dedicated hosts. You can assign groups to one of the two Greenplum® host types or to both of them at once.
You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Yandex MPP Analytics for PostgreSQL.
-
Click Create.
Yandex MPP Analytics for PostgreSQL will launch the operation to create a cluster from the backup.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To restore a cluster from a backup:
-
View the description of the CLI command for restoring a Greenplum® cluster:
yc managed-greenplum cluster restore --help -
Request the creation of a cluster from a backup:
yc managed-greenplum cluster restore \ --backup-id=<backup_ID> \ --time=<time_point> \ --name=<cluster_name> \ --environment=<environment> \ --network-name=default \ --master-resource-preset=<host_class> \ --master-disk-size=<storage_size_in_GB> \ --master-disk-type=<disk_type> \ --segment-resource-preset=<host_class> \ --segment-disk-size=<storage_size_in_GB> \ --segment-disk-type=<disk_type> \ --segment-host-count <number_of_segment_hosts> \ --segment-in-host <number_of_segments_per_host> \ --restore-only=<list_of_DBs_and_tables_to_restore> \ --zone-id=<availability_zone> \ --subnet-id=<subnet_ID> \ --assign-public-ip=<enable_public_access_to_cluster> \ --master-host-group-ids=<IDs_of_dedicated_host_groups_for_master_hosts> \ --segment-host-group-ids=<IDs_of_dedicated_host_groups_for_segment_hosts> \ --service-account <service_account_ID>Where:
-
--backup-id: Backup ID. -
--time: Time point to restore the Greenplum® cluster to, inyyyy-mm-ddThh:mm:ssZformat. By default, the cluster will be restored to the backup state. -
--name: Cluster name. -
--environment: Environment:PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by an SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test the new versions for compatibility with your application.PRODUCTION: For stable versions of your apps.
-
--network-name: Network name. -
--master-resource-preset: Master host class. -
--master-disk-size: Size of master host storage in GB. -
--master-disk-type: Disk type for master hosts. -
--segment-resource-preset: Segment host class.The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
--segment-disk-size: Size of segment host storage in GB. -
--segment-disk-type: Disk type for segment hosts. -
--segment-host-count: Number of segment hosts. -
--segment-in-host: Number of segments per host. -
--restore-only: Comma-separated list of DBs and tables to restore from the backup. This is an optional setting. Supported formats:<DB>/<schema>/<table>,<DB>/<table>, and<DB>. You may use the*wildcard symbol as well. If you omit this setting, the whole cluster will be restored. -
--zone-id: Availability zone. -
--master-host-group-idsand--segment-host-group-ids: IDs of dedicated host groups for master and segment hosts. This is an optional setting.You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Yandex MPP Analytics for PostgreSQL.
-
--subnet-id: Subnet ID. Specify it if the selected availability zone has two or more subnets. -
--assign-public-ip: Flag you set if the cluster needs access from the internet. -
--service-account: Service account ID.
-
-
Get an IAM token for API authentication and set it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Create a file named
body.jsonand paste the following code into it:{ "backupId": "<backup_ID>", "time": "<time>", "folderId": "<folder_ID>", "name": "<cluster_name>", "environment": "<environment>", "networkId": "<network_ID>", "config": { "zoneId": "<availability_zone>", "subnetId": "<subnet_ID>", "assignPublicIp": "<enable_public_access_to_cluster_hosts>" }, "masterResources": { "resourcePresetId": "<host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" }, "segmentResources": { "resourcePresetId": "<host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" }, "segmentHostCount": "<number_of_segment_hosts>", "segmentInHost": "<number_of_segments_per_host>", "restoreOnly": [ "<DB_and_table_1>", "<DB_and_table_2>", ... "<DB_and_table_N>" ], "masterHostGroupIds": [ "string" ], "segmentHostGroupIds": [ "string" ], "serviceAccountId": "<service_account_ID>", "logging": { "enabled": "<enable_transferring_logs>", "commandCenterEnabled": "<transfer_Yandex_Command_Center_logs>", "greenplumEnabled": "<transfer_Greenplum®_logs>", "poolerEnabled": "<transfer_connection_pooler_logs>", "folderId": "<folder_ID>" } }Where:
-
backupId: Backup ID. You can request it with the list of backups. -
time: Time point to restore the Greenplum® cluster to, inyyyy-mm-ddThh:mm:ssZformat. By default, the cluster will be restored to the backup state. -
folderId: ID of the folder you want to restore the cluster to. You can request the ID with the list of folders in the cloud. By default, the cluster is restored to the same folder the backup is in. -
name: Name of the new cluster. -
environment: Environment:PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by an SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test the new versions for compatibility with your application.PRODUCTION: For stable versions of your apps.
-
networkId: Network ID. -
config: Cluster settings:zoneId: Availability zone.subnetId: Subnet ID.assignPublicIp: Public access to cluster hosts,trueorfalse.
-
masterResources,segmentResources: Master and segment host configuration in the cluster:resourcePresetId: Host class.diskSize: Disk size, in bytes.diskTypeId: Disk type.
-
segmentHostCount: Number of segment hosts, from2to32. -
segmentInHost: Number of segments per host. The maximum value of this setting depends on the host class.The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
restoreOnly: List of DBs and tables to restore from the backup. This is an optional setting. Supported formats:<DB>/<schema>/<table>,<DB>/<table>, and<DB>. You may use the*wildcard symbol as well. If you omit this setting, the whole cluster will be restored. -
masterHostGroupIdsandsegmentHostGroupIds: IDs of dedicated host groups for master and segment hosts. This is an optional setting.You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Yandex MPP Analytics for PostgreSQL.
-
serviceAccountId: Service account ID. -
logging: Settings for transferring logs to Yandex Cloud Logging:-
enabled: Manages log transfer,trueorfalse. To enable parameters responsible for transferring specific logs, provide thetruevalue. -
commandCenterEnabled: Transferring Command Center logs,trueorfalse. -
greenplumEnabled: Transferring Greenplum® logs,trueorfalse. -
poolerEnabled: Transferring connection pooler logs,trueorfalse. -
folderId: Specify the ID of the folder whose log group you want to use. -
logGroupId: ID of the log group to write logs to.Specify either
folderIdorlogGroupId.
-
-
-
Call the Cluster.Restore method, e.g., via the following cURL
request:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters:restore' \ --data "@body.json" -
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Create a file named
body.jsonand paste the following code into it:{ "backup_id": "<backup_ID>", "time": "<time>", "folder_id": "<folder_ID>", "name": "<cluster_name>", "environment": "<environment>", "network_id": "<network_ID>", "config": { "zone_id": "<availability_zone>", "subnet_id": "<subnet_ID>", "assign_public_ip": "<enable_public_access_to_cluster_hosts>" }, "master_resources": { "resource_preset_id": "<host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" }, "segment_resources": { "resource_preset_id": "<host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" }, "segment_host_count": "<number_of_segment_hosts>", "segment_in_host": "<number_of_segments_per_host>", "restore_only": [ "<DB_and_table_1>", "<DB_and_table_2>", ... "<DB_and_table_N>" ], "master_host_group_ids": [ "string" ], "segment_host_group_ids": [ "string" ], "service_account_id": "<service_account_ID>" }Where:
-
backup_id: Backup ID. You can request it with the list of backups. -
time: Time point to restore the Greenplum® cluster to, inyyyy-mm-ddThh:mm:ssZformat. By default, the cluster will be restored to the backup state. -
folder_id: ID of the folder you want to restore the cluster to. You can request the ID with the list of folders in the cloud. By default, the cluster is restored to the same folder the backup is in. -
name: Name of the new cluster. -
environment: Environment:PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by an SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test the new versions for compatibility with your application.PRODUCTION: For stable versions of your apps.
-
network_id: Network ID. -
config: Cluster settings:zone_id: Availability zone.subnet_id: Subnet ID.assign_public_ip: Public access to cluster hosts,trueorfalse.
-
master_resources,segment_resources: Master and segment host configuration in the cluster:resource_preset_id: Host class.disk_size: Disk size, in bytes.disk_type_id: Disk type.
-
segment_host_count: Number of segment hosts, from2to32. -
segment_in_host: Number of segments per host. The maximum value of this setting depends on the host class.The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
restore_only: List of DBs and tables to restore from the backup. This is an optional setting. Supported formats:<DB>/<schema>/<table>,<DB>/<table>, and<DB>. You may use the*wildcard symbol as well. If you omit this setting, the whole cluster will be restored. -
master_host_group_idsandsegment_host_group_ids: IDs of dedicated host groups for master and segment hosts. This is an optional setting.You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Yandex MPP Analytics for PostgreSQL.
-
service_account_id: Service account ID. -
logging: Settings for transferring logs to Yandex Cloud Logging:-
enabled: Manages log transfer,trueorfalse. To enable parameters responsible for transferring specific logs, provide thetruevalue. -
command_center_enabled: Transferring Command Center logs,trueorfalse. -
greenplum_enabled: Transferring Greenplum® logs,trueorfalse. -
pooler_enabled: Transferring connection pooler logs,trueorfalse. -
folder_id: Specify the ID of the folder whose log group you want to use. -
log_group_id: ID of the log group to write logs to.Specify either
folder_idorlog_group_id.
-
-
-
Use the ClusterService.Restore call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d @ \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.ClusterService.Restore \ < body.json -
View the server response to make sure your request was successful.
Greenplum® and Greenplum Database® are registered trademarks or trademarks of Broadcom Inc. in the United States and/or other countries.