Creating a Sharded PostgreSQL cluster
Note
This feature is at the Preview stage.
Creating a cluster
-
In the management console
, select the folder where you want to create a Sharded PostgreSQL cluster. -
Select Yandex Managed Service for Sharded PostgreSQL.
-
Click Create cluster.
-
Under Basic parameters:
-
Give the cluster a name. The name must be unique within the folder.
-
Optionally, enter a description for the cluster.
-
Optionally, create labels:
- Click Add label.
- Enter a label in
key: valueformat. - Press Enter.
-
Select your cluster environment. Note that you cannot change the environment once the cluster is created:
PRODUCTION: For stable versions of your applications.PRESTABLE: For testing purposes. The prestable environment is similar to the production environment and likewise covered by an SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test new versions for compatibility with your application.
-
Select the sharding type:
Standard: Cluster will consist only of infrastructure hosts.Advanced: Cluster will consist only of router hosts and, optionally, coordinator hosts.
-
-
Under Network settings, select a network and security groups for the cluster.
-
Specify the computing resource configuration:
-
For standard sharding, specify the infrastructure host configuration under Infrastructure.
-
For advanced sharding, specify the router host configuration under Router.
Optionally, under Coordinator, enable Coordinator and specify the coordinator host configuration.
To specify your computing resource configuration:
-
Select a platform in the Platform field.
-
Specify the Type of virtual machine where you want to deploy the hosts:
-
Select Host class.
-
Under Storage, select the disk type and specify the storage size.
-
Under Hosts:
-
Click Add host to add the required number of hosts to create together with the Sharded PostgreSQL cluster.
-
Click
and specify the following for each host:- Availability zone.
- Subnet: By default, each host is created in a separate subnet.
- Public access to make the host accessible from outside Yandex Cloud, if required.
After creating a Sharded PostgreSQL cluster, you can add more hosts to it if your folder resource quotas allow.
-
-
-
Under Database, specify the database parameters:
-
Database name. Acceptable length is from 1 to 63 characters. Can contain lowercase and uppercase Latin letters, numbers, underscores, and hyphens.
-
Username. Acceptable length is from 1 to 63 characters. Can contain lowercase and uppercase Latin letters, numbers, underscores, and hyphens, but can't start with a hyphen.
-
Password: It may be from 8 to 128 characters long.
-
-
If needed, configure advanced cluster settings:
-
Password from Sharded PostgreSQL console. It may be from 8 to 128 characters long.
-
Min. logging level: Execution log will register logs of this or higher level. The available levels are
DEBUG,INFO,WARN,ERROR,FATAL, andPANIC. The default level isINFO. -
Backup start time (UTC): Time interval during which the cluster backup starts. Time is specified in 24-hour UTC format. The default time is
22:00 - 23:00UTC. -
Retention period for automatic backups, days: Automatic backups will be stored for this many days. The default value is seven days.
-
Maintenance window: Maintenance window settings:
- To enable maintenance at any time, select arbitrary (default).
- To specify the preferred maintenance start time, select by schedule and specify the desired day of the week and UTC hour. For example, you can choose a time when the cluster is least loaded.
Maintenance operations are carried out both on enabled and disabled clusters. They may include updating the DBMS, applying patches, and so on.
-
DataLens access: Enables you to analyze cluster data in Yandex DataLens.
-
WebSQL access: Enables SQL queries against cluster databases from the Yandex Cloud management console using Yandex WebSQL.
-
Access from Data Transfer: Enable this option to allow cluster access from Yandex Data Transfer.
-
Deletion protection: Manages cluster protection against accidental deletion.
Even with cluster deletion protection enabled, one can still delete a user or database or connect manually and delete the database contents.
-
-
To configure cluster-level DBMS settings, under DBMS settings, click Settings.
-
Click Create cluster.
-
Get an IAM token for API authentication and put it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Create a file named
body.jsonand paste the following code into it:{ "folderId": "<folder_ID>", "name": "<cluster_name>", "description": "<description>", "environment": "<environment>", "securityGroupIds": [ "<security_group_1_ID>", "<security_group_2_ID>", ... "<security_group_N_ID>" ], "networkId": "<network_ID>", "deletionProtection": <protect_cluster_from_deletion>, "configSpec": { "spqrSpec": { "router": { "config": { "showNoticeMessages": <show_information_notifications>, "timeQuantiles": [ <list_of_time_quantiles_for_displaying_statistics> ], "defaultRouteBehavior": "<allow_multishard_requests>", "preferSameAvailabilityZone": <routing_priority_to_router_availability_zone> }, "resources": { "resourcePresetId": "<router_host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" } }, "coordinator": { "resources": { "resourcePresetId": "<coordinator_host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" } }, "infra": { "router": { "showNoticeMessages": <show_information_notifications>, "time_quantiles": [ <list_of_time_quantiles_for_displaying_statistics> ], "defaultRouteBehavior": "<allow_multishard_requests>", "preferSameAvailabilityZone": <routing_priority_to_router_availability_zone> }, "resources": { "resourcePresetId": "<INFRA_host_class>", "diskSize": "<storage_size_in_bytes>", "diskTypeId": "<disk_type>" } }, "consolePassword": "<Sharded_PostgreSQL_console_password>", "logLevel": "<logging_level>" }, "backupWindowStart": { "hours": "<hours>", "minutes": "<minutes>", "seconds": "<seconds>", "nanos": "<nanoseconds>" }, "backupRetainPeriodDays": "<number_of_days>", }, "databaseSpecs": [ { "name": "<DB_name>" }, { <similar_configuration_for_DB_2> }, { ... }, { <similar_configuration_for_DB_N> } ], "userSpecs": [ { "name": "<username>", "password": "<user_password>", "permissions": [ { "databaseName": "<DB_name>" }, { <database_2_name> }, { ... }, { <DB_N_name> } ], "settings": { "connectionLimit": "<number_of_user_connections>", "connectionRetries": "<number_of_retries_when_connecting>" }, "grants": [ "privilege_1", ..., "privilege_N" ] }, { <similar_settings_for_user_2> }, { ... }, { <similar_settings_for_user_N> } ], "hostSpecs": [ { "zoneId": "<availability_zone>", "subnetId": "<subnet_ID>", "assignPublicIp": <allow_public_access_to_host>, "type": "<host_type>" }, { <similar_settings_for_host_2> }, { ... }, { <similar_settings_for_host_N> } ], "shardSpecs": [ { "shardName": "<shard_name>", "mdbPostgresql": { "clusterId": "<cluster_ID>" } }, { <similar_set_of_settings_for_shard_2> }, { ... }, { <similar_set_of_settings_for_shard_N> } ], "maintenanceWindow": { "weeklyMaintenanceWindow": { "day": "<day_of_week>", "hour": "<hour>" } } }Where:
-
folderId: Folder ID. You can get it from the list of your cloud folders. -
name: Cluster name. -
environment: Cluster environment,PRODUCTIONorPRESTABLE. -
networkId: ID of the network the cluster will be deployed in.Warning
The cloud network selected for cluster deployment cannot be changed later. If you need to migrate your cluster to another cloud network in the future, use the restore from a backup feature and specify the required network for your new cluster copy.
-
securityGroupIds: Security group IDs. -
deletionProtection: Cluster deletion protection,trueorfalse.Even with deletion protection enabled, one can still connect to the cluster manually and delete the data.
-
configSpec: Cluster settings:-
spqrSpec: Sharded PostgreSQL settings.-
router: For advanced sharding, configure the following router settings:-
config: Router configuration:showNoticeMessages: Show information notifications,trueorfalse.timeQuantiles: Array of time quantile strings for displaying statistics. The default values are"0.5","0.75","0.9","0.95","0.99","0.999","0.9999".defaultRouteBehavior: Router's multishard request execution policy. Possible values:BLOCKorALLOW.preferSameAvailabilityZone: Enable priority routing of read requests to the router's availability zone,trueorfalse.
-
resources:ROUTERhost resource parameters:resourcePresetId: Host class.diskSize: Disk size, in bytes.diskTypeId: Disk type.
-
-
coordinator: For advanced sharding, set the following coordinator resource parameters:resourcePresetId: Host class.diskSize: Disk size, in bytes.diskTypeId: Disk type.
-
infra: For standard sharding, set the followingINFRAhost settings:-
resources: Resource parameters:resourcePresetId: Host class.diskSize: Disk size, in bytes.diskTypeId: Disk type.
-
router: Router configuration:showNoticeMessages: Show information notifications,trueorfalse.timeQuantiles: Array of time quantiles for displaying statistics. The default values are0.5,0.75,0.9,0.95,0.99,0.999,0.9999.defaultRouteBehavior: Router's multishard request execution policy. Possible values:BLOCKorALLOW.preferSameAvailabilityZone: Enable priority routing of read requests to the router's availability zone,trueorfalse.
-
consolePassword: Sharded PostgreSQL console password. -
logLevel: Query logging level:DEBUG,INFO,WARNING,ERROR,FATAL,PANIC.
-
-
-
backupWindowStart: Backup window settings.Here, specify the backup start time. Allowed values:
hours: From0to23hours.minutes: From0to59minutes.seconds: From0to59seconds.nanos: From0to999999999nanoseconds.
-
backupRetainPeriodDays: Number of days to retain the cluster backup. Possible values: between7and60days.
-
-
databaseSpecs: Database settings as an array of elements, one per database. Each element has the following structure:name: Database name.
-
userSpecs: User settings as an array of elements, one per user. Each element has the following structure:-
name: Username. -
password: User password. The password must be from 8 to 128 characters long. -
permissions.databaseName: Name of the database the user can access. -
settings: Database user connection parameters:connectionLimit: Connection limit.connectionRetries: Number of reconnect attempts.
-
grants: User privileges as an array of strings. The possible values arereader,writer,admin, ortransfer.
-
-
hostSpecs: Cluster host settings as an array of elements, one per host. Each element has the following structure:-
zoneId: Availability zone. -
subnetId: Subnet ID. -
assignPublicIp: Permission to connect to the host from the internet,trueorfalse. -
type: Host type. The possible values are:ROUTER: Router in a cluster with advanced sharding.COORDINATOR: Coordinator in a cluster with advanced sharding.INFRA:INFRAhost in a cluster with standard sharding.
-
-
shardSpecs: Shard settings as an array of elements, one per shard. Each element has the following structure:shardName: Shard name.mdbPostgresql.clusterId: Managed Service for PostgreSQL cluster ID within the shard.
-
maintenanceWindow: Maintenance window settings:day: Day of the week, inDDDformat, for scheduled maintenance.hour: Hour of day, inHHformat, for scheduled maintenance. Allowed values range from1to24.
-
-
Call the Cluster.Create method, e.g., via the following cURL
request:curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-spqr/v1/clusters' \ --data "@body.json" -
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it in an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume that the repository contents reside in the
~/cloudapi/directory. -
Create a file named
body.jsonand paste the following code into it:{ "folder_id": "<folder_ID>", "name": "<cluster_name>", "description": "<description>", "environment": "<environment>", "security_group_ids": [ "<security_group_1_ID>", "<security_group_2_ID>", ... "<security_group_N_ID>" ], "network_id": "<network_ID>", "deletion_protection": <protect_cluster_from_deletion>, "config_spec": { "spqr_spec": { "router": { "config": { "show_notice_messages": { "value": <show_information_notifications> }, "time_quantiles": [ <list_of_time_quantiles_for_displaying_statistics> ], "default_route_behavior": "<allow_multishard_requests>", "prefer_same_availability_zone": { "value": <routing_priority_to_router_availability_zone> } }, "resources": { "resource_preset_id": "<router_host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" } }, "coordinator": { "resources": { "resource_preset_id": "<coordinator_host_class>", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" } }, "infra": { "resources": { "resource_preset_id": "INFRA_host_class", "disk_size": "<storage_size_in_bytes>", "disk_type_id": "<disk_type>" }, "router": { "show_notice_messages": { "value": <show_information_notifications> }, "time_quantiles": [ <list_of_time_quantiles_for_displaying_statistics> ], "default_route_behavior": "<allow_multishard_requests>", "prefer_same_availability_zone": { "value": <routing_priority_to_router_availability_zone> } }, }, "console_password": "<Sharded_PostgreSQL_console_password>", "log_level": "<logging_level>" }, "backup_window_start": { "hours": "<hours>", "minutes": "<minutes>", "seconds": "<seconds>", "nanos": "<nanoseconds>" }, "backup_retain_period_days": "<number_of_days>" }, "database_specs": [ { "name": "<DB_name>" }, { <similar_configuration_for_DB_2> }, { ... }, { <similar_configuration_for_DB_N> } ], "user_specs": [ { "name": "<username>", "password": "<user_password>", "permissions": [ { "database_name": "<DB_name>" }, { <database_2_name> }, { ... }, { <DB_N_name> } ], "settings": { "connection_limit": { "value": <number_of_user_connections> }, "connection_retries": { "value": <number_of_retries_when_connecting> } }, "grants": [ "privilege_1", ..., "privilege_N" ] }, { <similar_settings_for_user_2> }, { ... }, { <similar_settings_for_user_N> } ], "host_specs": [ { "zone_id": "<availability_zone>", "subnet_id": "<subnet_ID>", "assign_public_ip": <allow_public_access_to_host>, "type": "<host_type>" }, { <similar_settings_for_host_2> }, { ... }, { <similar_settings_for_host_N> } ], "shard_specs": [ { "shard_name": "<shard_name>", "mdb_postgresql": { "cluster_id": "<cluster_ID>" } }, { <similar_set_of_settings_for_shard_2> }, { ... }, { <similar_set_of_settings_for_shard_N> } ], "maintenance_window": { "weekly_maintenance_window": { "day": "<day_of_week>", "hour": "<hour>" } } }Where:
-
folder_id: Folder ID. You can get it with the list of folders in the cloud. -
name: Cluster name. -
environment: Cluster environment,PRODUCTIONorPRESTABLE. -
network_id: ID of the network the cluster will be deployed in.Warning
The cloud network selected for cluster deployment cannot be changed later. If you need to migrate your cluster to another cloud network in the future, use the restore from a backup feature and specify the required network for your new cluster copy.
-
security_group_ids: Security group IDs. -
deletion_protection: Cluster deletion protection,trueorfalse.Even with deletion protection enabled, one can still connect to the cluster manually and delete the data.
-
config_spec: Cluster settings:-
spqr_spec: Sharded PostgreSQL settings:-
router: For advanced sharding, configure the following router settings:-
config: Router configuration:show_notice_messages: Show information notifications,trueorfalse.time_quantiles: Array of time quantiles for displaying statistics. The default values are0.5,0.75,0.9,0.95,0.99,0.999,0.9999.default_route_behavior: Router's multishard request execution policy. Possible values:BLOCKorALLOW.prefer_same_availability_zone: Enable priority routing of read requests to the router's availability zone,trueorfalse.
-
resources:ROUTERhost resource parameters:resource_preset_id: Host class.disk_size: Disk size, in bytes.disk_type_id: Disk type.
-
-
coordinator: For advanced sharding, set the following coordinator resource parameters:resource_preset_id: Host class.disk_size: Disk size, in bytes.disk_type_id: Disk type.
-
infra: For standard sharding, set the followingINFRAhost settings:-
resources: Resource parameters:resource_preset_id: Host class.disk_size: Disk size, in bytes.disk_type_id: Disk type.
-
router: Router configuration:default_route_behavior: Default router behavior. Possible values:BLOCKorALLOWthe request.prefer_same_availability_zone: Enable priority routing to the router's availability zone,trueorfalse.
-
console_password: Sharded PostgreSQL console password. -
log_level: Query logging level:DEBUG,INFO,WARNING,ERROR,FATAL,PANIC.
-
-
-
backup_window_start: Backup window settings.Here, specify the backup start time. Allowed values:
hours: From0to23hours.minutes: From0to59minutes.seconds: From0to59seconds.nanos: From0to999999999nanoseconds.
-
backup_retain_period_days: Number of days to retain the cluster backup. Possible values: between7and60days.
-
-
database_specs: Database settings as an array of elements, one for each DB. Each element has the following structure:name: Database name.
-
user_specs: User settings as an array of elements, one per user. Each element has the following structure:-
name: Username. -
password: User password. The password must be from 8 to 128 characters long. -
permissions.database_name: Name of the database the user can access. -
settings: Database user connection parameters:connection_limit: Connection limit.connection_retries: Number of reconnect attempts.
-
grants: User privileges as an array of strings. The possible values arereader,writer,admin, ortransfer.
-
-
host_specs: Cluster host settings as an array of elements, one per host. Each element has the following structure:-
zone_id: Availability zone -
subnet_id: Subnet ID. -
assign_public_ip: Permission to connect to the host from the internet,trueorfalse. -
type: Host type. The possible values are:ROUTER: Router in a cluster with advanced sharding.COORDINATOR: Coordinator in a cluster with advanced sharding.INFRA:INFRAhost in a cluster with standard sharding.
-
-
shard_specs: Shard settings as an array of elements, one per shard. Each element has the following structure:shard_name: Shard name.mdb_postgresql.cluster_id: Managed Service for PostgreSQL cluster ID within the shard.
-
maintenance_window: Maintenance window settings:day: Day of the week, inDDDformat, for scheduled maintenance.hour: Hour of day, inHHformat, for scheduled maintenance. Allowed values range from1to24.
-
-
Call the ClusterService.Create method, e.g., via the following gRPCurl
request:grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/spqr/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d @ \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.spqr.v1.ClusterService.Create \ < body.json -
Check the server response to make sure your request was successful.