Managing backups in Managed Service for Greenplum®
You can view your existing backups and restore clusters from them.
Getting a list of backups
To get a list of cluster backups:
- Navigate to the folder dashboard
and select Managed Service for Greenplum. - Click the name of the cluster you need and select the
Backups tab.
To get a list of all backups in a folder:
- Navigate to the folder dashboard
and select Managed Service for Greenplum. - In the left-hand panel, select
Backups.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To get a list of Greenplum® cluster backups, run the command:
yc managed-greenplum cluster list-backups <cluster_name_or_ID>
You can get the cluster ID and name with the list of clusters in the folder.
Result:
+--------------------------+---------------------+----------------------+---------------------+
| ID | CREATED AT | SOURCE CLUSTER ID | STARTED AT |
+--------------------------+---------------------+----------------------+---------------------+
| c9qgo11pud7k********:... | 2020-08-10 12:00:00 | c9qgo11pud7k******** | 2020-08-10 11:55:17 |
| ... |
+--------------------------+---------------------+----------------------+---------------------+
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
To get a list of cluster backups:
-
Use the Cluster.ListBackups method and send the following request, e.g., via cURL
:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters/<cluster_ID>/backups'You can request the cluster ID with the list of clusters in the folder.
-
View the server response to make sure your request was successful.
-
-
To get a list of backups for all the clusters in a folder:
-
Use the Backup.List method and send the following request, e.g., via cURL
:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/backups' \ --url-query folderId=<folder_ID>You can get the folder ID with the list of folders in the cloud.
-
View the server response to make sure your request was successful.
-
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
To get a list of cluster backups:
-
Use the ClusterService.ListBackups call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.ClusterService.ListBackupsYou can request the cluster ID with the list of clusters in the folder.
-
View the server response to make sure your request was successful.
-
-
To get a list of backups for all the clusters in a folder:
-
Use the BackupService.List call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/backup_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "folder_id": "<folder_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.BackupService.ListYou can get the folder ID with the list of folders in the cloud.
-
View the server response to make sure your request was successful.
-
Getting backup info
To get information about a backup of an existing cluster:
- Navigate to the folder dashboard
and select Managed Service for Greenplum. - Click the name of the cluster you need and select the
Backups tab.
To get information about a backup of a previously deleted cluster:
- Navigate to the folder dashboard
and select Managed Service for Greenplum. - In the left-hand panel, select
Backups.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Backup.Get method and send the following request, e.g., via cURL
:curl \ --request GET \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/backups/<backup_ID>'You can request the backup ID with the list of backups.
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the BackupService.Get call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/backup_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "backup_id": "<backup_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.BackupService.GetYou can request the backup ID with the list of backups.
-
View the server response to make sure your request was successful.
Creating a backup
- Navigate to the folder dashboard and select Managed Service for Greenplum.
- Click the name of the cluster you need and select the
Backups tab. - Click
Create backup.
The service will start creating a backup without an additional confirmation.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Use the Cluster.Backup method and send the following request, e.g., via cURL
:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters/<cluster_ID>:backup'You can request the cluster ID with the list of clusters in the folder.
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Use the ClusterService.Backup call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>" }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.ClusterService.BackupYou can request the cluster ID with the list of clusters in the folder.
-
View the server response to make sure your request was successful.
Warning
While you are creating your backup, the cluster performance might degrade.
Restoring a cluster from a backup
The Point-in-Time Recovery (PITR) technology enables you to restore cluster state to any recovery point created after saving a backup. For more information, see Backups.
When you restore a cluster from a backup, you create a new cluster with the backup data. If the folder lacks resources to create such a cluster, data will not be restored from the backup.
For a new cluster, you should set all the parameters that are required at creation.
To migrate Managed Service for Greenplum® cluster hosts to a different availability zone, restore your cluster from a backup. During cluster recovery from a backup, specify a new availability zone. If your cluster operates as a Yandex Data Transfer endpoint, after recovering it from a backup, create the endpoint and the transfer again.
Warning
Recovering from a backup imposes restrictions on parameters of the new cluster.
- The total number of segments must be the same as in the source cluster.
- The disk size per segment in the new cluster must be at least as large as in the source cluster.
Example
The source cluster has four segment hosts, each containing four segments. The total number of segments is 16. When restoring the cluster, you can choose two segment hosts with eight segments per host, so that the total number of segments remains 16.
To ensure that the disk size per segment does not decrease, the disk size in each segment host must at least double.
If you set the current time as the recovery time, the new cluster will match the state of the latest available recovery point.
Before you begin, assign the managed-greenplum.editor role or higher to your Yandex Cloud account for the backup folder and the new cluster folder.
To restore an existing cluster from a backup:
-
Navigate to the folder dashboard
and select Managed Service for Greenplum. -
Click the name of the cluster you need and select the
Backups tab. -
Click
for the backup you need and click Restore cluster. -
Set up the new cluster. You can select a folder for the new cluster from the Folder list.
-
In the Date and time of recovery (UTC) setting, specify the time point to which you want to restore the cluster. You can enter a value manually or select it from the drop-down calendar. The service will select the recovery point closest to that time.
If you do not change the setting, the cluster state will be copied from a backup. Recovery points will not be used.
-
If you want to restore only certain databases or tables, list them in the Databases and tables for recovery field. If you leave the field blank, the whole cluster will be restored.
-
In the Host count setting, specify the number of segment hosts.
-
In the Segments per host setting, specify the number of segments per host.
The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
Optionally, select groups of dedicated hosts to place master hosts or segment hosts on the dedicated hosts. You can assign groups to one or both of the two types of Greenplum® hosts.
You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Managed Service for Greenplum®.
-
Click Create.
To restore a previously deleted cluster from a backup:
-
Navigate to the folder dashboard
and select Managed Service for Greenplum. -
In the left-hand panel, select
Backups. -
Find the backup you need using the backup creation time and cluster ID. The ID column contains IDs in
<cluster_ID>:<backup_ID>format. -
Click
for the backup you need and click Restore cluster. -
Set up the new cluster. You can select a folder for the new cluster from the Folder list.
-
In the Date and time of recovery (UTC) setting, specify the time point to which you want to restore the cluster. You can enter a value manually or select it from the drop-down calendar. The service will select the recovery point closest to that time.
If you do not change the setting, the cluster state will be copied from a backup. Recovery points will not be used.
-
If you want to restore only certain databases or tables, list them in the Databases and tables for recovery field. If you leave the field blank, the whole cluster will be restored.
-
In the Host count setting, specify the number of segment hosts.
-
In the Segments per host setting, specify the number of segments per host.
The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
Optionally, select groups of dedicated hosts to place master hosts or segment hosts on the dedicated hosts. You can assign groups to one or both of the two types of Greenplum® hosts.
You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Managed Service for Greenplum®.
-
Click Create.
Managed Service for Greenplum® will launch the operation to create a cluster from the backup.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To restore a cluster from a backup:
-
View the description of the CLI command to restore a Greenplum® cluster:
yc managed-greenplum cluster restore --help -
Make a request to create a cluster from a backup:
yc managed-greenplum cluster restore \ --backup-id=<backup_ID> \ --time=<time_point> \ --name=<cluster_name> \ --environment=<environment> \ --network-name=default \ --master-resource-preset=<host_class> \ --master-disk-size=<storage_size_in_GB> \ --master-disk-type=<disk_type> \ --segment-resource-preset=<host_class> \ --segment-disk-size=<storage_size_in_GB> \ --segment-disk-type=<disk_type> \ --segment-host-count <number_of_segment_hosts> \ --segment-in-host <number_of_segments_per_host> \ --restore-only=<list_of_DBs_and_tables_to_restore> \ --zone-id=<availability_zone> \ --subnet-id=<subnet_ID> \ --assign-public-ip=<enable_public_access_to_cluster> \ --master-host-group-ids=<IDs_of_dedicated_host_groups_for_master_hosts> \ --segment-host-group-ids=<IDs_of_dedicated_host_groups_for_segment_hosts> \ --service-account <service_account_ID>Where:
-
--backup-id: Backup ID. -
--time: Time point to restore the Greenplum® cluster to, inyyyy-mm-ddThh:mm:ssZtime format. By default, the cluster will be restored from a backup. -
--name: Cluster name. -
--environment: Environment:PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by the SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test new versions for compatibility with your application.PRODUCTION: For stable versions of your apps.
-
--network-name: Network name. -
--master-resource-preset: Master host class. -
--master-disk-size: Size of master host storage in GB. -
--master-disk-type: Master host disk type. -
--segment-resource-preset: Segment host class.The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
--segment-disk-size: Size of segment host storage in GB. -
--segment-disk-type: Segment host disk type. -
--segment-host-count: Number of segment hosts. -
--segment-in-host: Number of segments per host. -
--restore-only: (Optional) Comma-separated list of DBs and tables to restore from the backup. Supported formats:<DB>/<schema>/<table>,<DB>/<table>, and<DB>. You may use the*wildcard symbol as well. If you omit this parameter, the whole cluster will be restored. -
--zone-id: Availability zone. -
--master-host-group-idsand--segment-host-group-ids: IDs of dedicated host groups for master and segment hosts. This is an optional parameter.You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Managed Service for Greenplum®.
-
--subnet-id: Subnet ID. Specify it if the selected availability zone has two or more subnets. -
--assign-public-ip: Flag you set if the cluster needs access from the internet. -
--service-account: Service account ID.
-
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Create a file named
body.jsonand paste the following code into it:{ "backupId": "<backup_ID>", "time": "<time>", "folderId": "<folder_ID>", "name": "<cluster_name>", "environment": "<environment>", "networkId": "<network_ID>", "config": { "zoneId": "<availability_zone>", "subnetId": "<subnet_ID>", "assignPublicIp": "<enable_public_access_to_cluster_hosts>" }, "masterResources": { "resourcePresetId": "<host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" }, "segmentResources": { "resourcePresetId": "<host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" }, "segmentHostCount": "<number_of_segment_hosts>", "segmentInHost": "<number_of_segments_per_host>", "restoreOnly": [ "<DB_and_table_1>", "<DB_and_table_2>", ... "<DB_and_table_N>" ], "masterHostGroupIds": [ "string" ], "segmentHostGroupIds": [ "string" ], "serviceAccountId": "<service_account_ID>", "logging": { "enabled": "<enable_transferring_logs>", "commandCenterEnabled": "<transfer_Yandex_Command_Center_logs>", "greenplumEnabled": "<transfer_Greenplum®_logs>", "poolerEnabled": "<transfer_connection_pooler_logs>", "folderId": "<folder_ID>" } }Where:
-
backupId: Backup ID. You can request it with the list of backups. -
time: Time point to restore the Greenplum® cluster to, inyyyy-mm-ddThh:mm:ssZtime format. By default, the cluster will be restored from a backup. -
folderId: ID of the folder you want to restore the cluster to. You can request the ID with the list of folders in the cloud. By default, the cluster is restored to the same folder the backup is in. -
name: Name of the new cluster. -
environment: Environment:PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by the SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test new versions for compatibility with your application.PRODUCTION: For stable versions of your apps.
-
networkId: Network ID. -
config: Cluster settings:zoneId: Availability zone.subnetId: Subnet ID.assignPublicIp: Public access to cluster hosts,trueorfalse.
-
masterResources,segmentResources: Master and segment host configuration in the cluster:resourcePresetId: Host class.diskSize: Disk size in bytes.diskTypeId: Disk type.
-
segmentHostCount: Number of segment hosts, from2to32. -
segmentInHost: Number of segments per host. The maximum value of this parameter depends on the host class.The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
restoreOnly: (Optional) List of DBs and tables to restore from the backup. Supported formats:<DB>/<schema>/<table>,<DB>/<table>, and<DB>. You may use the*wildcard symbol as well. If you omit this parameter, the whole cluster will be restored. -
masterHostGroupIdsandsegmentHostGroupIds: IDs of dedicated host groups for master and segment hosts. This is an optional parameter.You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Managed Service for Greenplum®.
-
serviceAccountId: Service account ID. -
logging: Settings for transferring logs to Yandex Cloud Logging:-
enabled: Manages log transfer,trueorfalse. To enable parameters responsible for transferring specific logs, provide thetruevalue. -
commandCenterEnabled: Transferring Command Center logs,trueorfalse. -
greenplumEnabled: Transferring Greenplum® logs,trueorfalse. -
poolerEnabled: Transferring connection pooler logs,trueorfalse. -
folderId: Specify the ID of the folder whose log group you want to use. -
logGroupId: ID of the log group to write logs to.Specify either
folderIdorlogGroupId.
-
-
-
Use the Cluster.Restore method and send the following request, e.g., via cURL
:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters:restore' \ --data "@body.json" -
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and save it as an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Create a file named
body.jsonand paste the following code into it:{ "backup_id": "<backup_ID>", "time": "<time>", "folder_id": "<folder_ID>", "name": "<cluster_name>", "environment": "<environment>", "network_id": "<network_ID>", "config": { "zone_id": "<availability_zone>", "subnet_id": "<subnet_ID>", "assign_public_ip": "<enable_public_access_to_cluster_hosts>" }, "master_resources": { "resource_preset_id": "<host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" }, "segment_resources": { "resource_preset_id": "<host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" }, "segment_host_count": "<number_of_segment_hosts>", "segment_in_host": "<number_of_segments_per_host>", "restore_only": [ "<DB_and_table_1>", "<DB_and_table_2>", ... "<DB_and_table_N>" ], "master_host_group_ids": [ "string" ], "segment_host_group_ids": [ "string" ], "service_account_id": "<service_account_ID>" }Where:
-
backup_id: Backup ID. You can request it with the list of backups. -
time: Time point to restore the Greenplum® cluster to, inyyyy-mm-ddThh:mm:ssZtime format. By default, the cluster will be restored from a backup. -
folder_id: ID of the folder you want to restore the cluster to. You can request the ID with the list of folders in the cloud. By default, the cluster is restored to the same folder the backup is in. -
name: Name of the new cluster. -
environment: Environment:PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by the SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test new versions for compatibility with your application.PRODUCTION: For stable versions of your apps.
-
network_id: Network ID. -
config: Cluster settings:zone_id: Availability zone.subnet_id: Subnet ID.assign_public_ip: Public access to cluster hosts,trueorfalse.
-
master_resources,segment_resources: Master and segment host configuration in the cluster:resource_preset_id: Host class.disk_size: Disk size in bytes.disk_type_id: Disk type.
-
segment_host_count: Number of segment hosts, from2to32. -
segment_in_host: Number of segments per host. The maximum value of this parameter depends on the host class.The segment host class and the number of segments per host affect the maximum amount of memory allocated to each Greenplum® server process. If you select a host class with small RAM and specify a large number of segments, an error may occur.
-
restore_only: (Optional) List of DBs and tables to restore from the backup. Supported formats:<DB>/<schema>/<table>,<DB>/<table>, and<DB>. You may use the*wildcard symbol as well. If you omit this parameter, the whole cluster will be restored. -
master_host_group_idsandsegment_host_group_ids: IDs of dedicated host groups for master and segment hosts. This is an optional parameter.You must first create a group of dedicated hosts in Yandex Compute Cloud.
You cannot edit this setting after you create a cluster.
If using dedicated hosts, the cluster cost is a sum of the charge for computing resources Yandex Compute Cloud and the markup Managed Service for Greenplum®.
-
service_account_id: Service account ID. -
logging: Settings for transferring logs to Yandex Cloud Logging:-
enabled: Manages log transfer,trueorfalse. To enable parameters responsible for transferring specific logs, provide thetruevalue. -
command_center_enabled: Transferring Command Center logs,trueorfalse. -
greenplum_enabled: Transferring Greenplum® logs,trueorfalse. -
pooler_enabled: Transferring connection pooler logs,trueorfalse. -
folder_id: Specify the ID of the folder whose log group you want to use. -
log_group_id: ID of the log group to write logs to.Specify either
folder_idorlog_group_id.
-
-
-
Use the ClusterService.Restore call and send the following request, e.g., via gRPCurl
:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d @ \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.greenplum.v1.ClusterService.Restore \ < body.json -
View the server response to make sure your request was successful.
Greenplum® and Greenplum Database® are registered trademarks or trademarks of Broadcom Inc. in the United States and/or other countries.