Managing disk space
When the storage usage exceeds 97%, the host automatically switches to read-only mode.
To prevent issues with database write operations, use one of the following methods:
-
Set up alerts in Yandex Monitoring to monitor storage utilization.
-
Manually disable read-only mode for the cluster and free up the storage space by deleting non-essential data.
-
Increase the storage size to automatically disable read-only mode. You can also change the disk type.
Set up alerts in Yandex Monitoring
-
Go to Monitoring.
-
Select the Managed Service for PostgreSQL service dashboard.
-
Create an alert with the following settings:
-
Metrics: Configure the following metric settings:
-
Cloud
-
Folder
-
Managed Service for PostgreSQL service
-
Managed Service for PostgreSQL cluster ID
You can get the cluster ID from the folder’s cluster list.
-
disk.free_byteslabel
-
-
Alert condition: Define the
Less than or equalscondition for disk usage percentage that will trigger the alert:- Aggregation function:
Minimum(metric’s minimum value over the period). - Warning:
90(90% of the storage size). - Alarm:
95(95% of the storage size). - Evaluation window: Preferred metric update period.
- Aggregation function:
-
Add the notification channel you created earlier.
-
Manually disable read-only mode for the cluster
Alert
Take measures to prevent free disk space from being fully depleted during the following operations. Otherwise, with the fail-safe mechanism disabled, PostgreSQL will crash, rendering the cluster inoperable.
To disable read-only mode:
-
Connect to the database using your preferred method.
-
Start a transaction and run the following statement within it:
SET LOCAL transaction_read_only TO off; -
In the same transaction, clean up the data you no longer need using the
DROPorTRUNCATEstatements. Avoid theDELETEstatement because it marks rows as deleted without physically purging them from the database. -
Commit the transaction and restart all database connections.
For example, to remove the
ExcessDataTable1table you no longer need, use the following transaction:BEGIN; SET LOCAL transaction_read_only TO off; DROP TABLE ExcessDataTable1; COMMIT;
Changing the disk type and increasing the storage size
Note
Some PostgreSQL settings depend on the storage size.
Make sure the cloud has enough quota to increase the storage size. Open the cloud's Quotas
Warning
- You cannot decrease the storage size.
- While resizing the storage, cluster hosts will be unavailable.
To change the disk type and increase the storage size for a cluster:
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To change the disk type and increase the storage size for a cluster:
-
See the description of the CLI command for updating a cluster:
yc managed-postgresql cluster update --help -
Specify the disk type and the storage size in the cluster update command. The storage size must not be less than the
disk_sizevalue in the cluster settings:yc managed-postgresql cluster update <cluster_name_or_ID> \ --disk-type <disk_type> \ --disk-size <storage_size_in_GB>
To change the disk type and increase the storage size for a cluster:
-
Open the current Terraform configuration file describing your infrastructure.
To learn how to create this file, see Creating a cluster.
For a complete list of configurable Managed Service for PostgreSQL cluster fields, refer to the Terraform provider guides.
-
In the Managed Service for PostgreSQL cluster description, update the
disk_type_idanddisk_sizeattributes in theconfig.resourcesblock:resource "yandex_mdb_postgresql_cluster" "<cluster_name>" { ... config { resources { disk_type_id = "<disk_type>" disk_size = <storage_size_in_GB> ... } } } -
Check if the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
Time limits
A Terraform provider sets the timeout for Managed Service for PostgreSQL cluster operations:
- Creating a cluster, including restoring from a backup: 30 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the
timeoutsblock to the cluster description, for example:resource "yandex_mdb_postgresql_cluster" "<cluster_name>" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } } -
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.Update method, e.g., via the following cURL
request:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-postgresql/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.resources.diskTypeId,configSpec.resources.diskSize", "configSpec": { "resources": { "diskTypeId": "<disk_type>", "diskSize": "<storage_size_in_bytes>" } } }'Where:
-
updateMask: Comma-separated list of settings you want to update. -
configSpec.resources: Storage settings:diskTypeId: Disk type.diskSize: New storage size in bytes.
You can request the cluster ID with the list of clusters in the folder.
-
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Call the ClusterService.Update method, e.g., via the following gRPCurl
request:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/postgresql/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.resources.disk_type_id", "config_spec.resources.disk_size" ] }, "config_spec": { "resources": { "disk_type_id": "<disk_type>", "disk_size": "<storage_size_in_bytes>" } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.postgresql.v1.ClusterService.UpdateWhere:
-
update_mask: List of settings you want to update as an array of strings (paths[]). -
config_spec.resources: Storage settings:disk_type_id: Disk type.disk_size: New storage size in bytes.
You can request the cluster ID with the list of clusters in the folder.
-
-
Check the server response to make sure your request was successful.
Setting up storage autoscaling
Note
Some PostgreSQL settings depend on the storage size.
Make sure the cloud has enough quota to increase the storage size. Open the cloud's Quotas
Warning
- You cannot decrease the storage size.
- While resizing the storage, cluster hosts will be unavailable.
-
Go to Managed Service for PostgreSQL.
-
Select your cluster and click Edit in the top panel.
-
Under Automatic increase of storage size:
-
In the Increase size field, specify the storage usage percentage that will trigger storage increase. You can configure scaling rules to trigger:
- At the next maintenance window.
- Immediately.
You can set both thresholds, provided that the threshold for immediate scaling is higher than that for scaling during the maintenance window.
To learn more about storage scaling rules, see this section.
-
In the Maximum storage size field, specify the maximum storage size that can be set during automatic scaling.
-
-
Click Save changes.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To set up storage autoscaling:
-
See the description of the CLI command for updating a cluster:
yc managed-postgresql cluster update --help -
In the cluster update command, specify the maximum storage size and the conditions for autoscaling.
Make sure the maximum storage size is greater than the
disk_sizecluster setting. The storage usage percentage triggering an immediate increase should be higher than the percentage scheduling an increase for the next maintenance window.yc managed-postgresql cluster update <cluster_ID_or_name> \ --disk-size-autoscaling disk-size-limit=<maximum_storage_size_in_bytes>,` `planned-usage-threshold=<scheduled_increase_percentage>,` `emergency-usage-threshold=<immediate_increase_percentage>If you have configured storage scaling during a maintenance window, set the maintenance schedule.
To learn more about storage scaling rules, see this section.
-
Open the current Terraform configuration file describing your infrastructure.
To learn how to create this file, see Creating a cluster.
For a complete list of configurable Managed Service for PostgreSQL cluster fields, refer to the Terraform provider guides.
-
Add the
disk_size_autoscalingsection to theconfigblock:resource "yandex_mdb_postgresql_cluster" "<cluster_name>" { ... config { ... disk_size_autoscaling { disk_size_limit = <maximum_storage_size_GiB> emergency_usage_threshold = <threshold_for_immediate_increase_in_percent> planned_usage_threshold = <threshold_for_scheduled_increase_in_percent> } ... } ... }Where:
-
disk_size_limit: Maximum object size after increase, in gibibytes. -
emergency_usage_threshold: Storage utilization threshold to trigger a storage increase right away, in percent. This is an optional setting. The default value is0(automatic increase is disabled).The possible values range from
0to100. -
planned_usage_threshold: Storage utilization threshold to trigger a storage increase during the next maintenance window, in percent. This is an optional setting. The default value is0(automatic increase is disabled).The possible values range from
0to100.
For more information about storage increase conditions, see this section.
Warning
If you specify both thresholds,
emergency_usage_thresholdmust not be less thanplanned_usage_threshold. -
-
If you specified
planned_usage_threshold, configure the maintenance schedule. -
Check if the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
Time limits
A Terraform provider sets the timeout for Managed Service for PostgreSQL cluster operations:
- Creating a cluster, including restoring from a backup: 30 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the
timeoutsblock to the cluster description, for example:resource "yandex_mdb_postgresql_cluster" "<cluster_name>" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } } -
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.Update method, e.g., via the following cURL
request:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-postgresql/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.diskSizeAutoscaling,maintenanceWindow", "configSpec": { "diskSizeAutoscaling": { "plannedUsageThreshold": "<threshold_for_scheduled_increase_in_percent>", "emergencyUsageThreshold": "<threshold_for_immediate_increase_in_percent>", "diskSizeLimit": "<maximum_storage_size_in_bytes>" } }, "maintenanceWindow": { "weeklyMaintenanceWindow": { "day": "<day_of_week>", "hour": "<hour>" } } }'Where:
-
updateMask: Comma-separated list of settings you want to update.Here we provide only the
configSpec.diskSizeAutoscalingandmaintenanceWindowsettings. -
configSpec: Cluster settings:-
diskSizeAutoscaling: Automatic storage size increase settings:-
plannedUsageThreshold: Storage utilization threshold to trigger a storage increase during the next maintenance window, in percent. This is an optional setting. The default value is0(automatic increase is disabled).The possible values range from
0to100. -
emergencyUsageThreshold: Storage utilization threshold to trigger a storage increase right away, in percent. This is an optional setting. The default value is0(automatic increase is disabled).The possible values range from
0to100. -
diskSizeLimit: Maximum object size after increase, in bytes.
Warning
-
When using
plannedUsageThreshold, make sure to specify themaintenanceWindowsetting. -
If you specify both thresholds,
emergencyUsageThresholdmust not be less thanplannedUsageThreshold.
For more information about storage increase conditions, see this section.
-
Use a value between
0and100. The default value is0(autoscale disabled). The value of this setting must be greater than or equal toplannedUsageThreshold.diskSizeLimit: Maximum storage size, in bytes, that can be set when one of the specified storage usage thresholds is met.
To learn more about storage scaling rules, see this section.
-
-
maintenanceWindow: Maintenance window schedule. This setting is required only ifplannedUsageThresholdis set. It contains the following subsettings:day: Day of the week, inDDDformat, for scheduled maintenance.hour: Hour of day, inHHformat, for scheduled maintenance. The possible values range from1to24.
You can request the cluster ID with the list of clusters in the folder.
-
-
Check the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Call the ClusterService.Update method, e.g., via the following gRPCurl
request:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/postgresql/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.disk_size_autoscaling", "maintenance_window" ] }, "config_spec": { "disk_size_autoscaling": { "planned_usage_threshold": "<threshold_for_scheduled_increase_in_percent>", "emergency_usage_threshold": "<threshold_for_immediate_increase_in_percent>", "disk_size_limit": "<maximum_storage_size_in_bytes>" } }, "maintenance_window": { "weekly_maintenance_window": { "day": "<day_of_week>", "hour": "<hour>" } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.postgresql.v1.ClusterService.UpdateWhere:
-
update_mask: List of settings you want to update as an array of strings (paths[]).Here we provide only the
config_spec.disk_size_autoscalingandmaintenance_windowsettings. -
config_spec: Cluster settings:-
disk_size_autoscaling: Automatic storage size increase settings:-
planned_usage_threshold: Storage utilization threshold to trigger a storage increase during the next maintenance window, in percent. This is an optional setting. The default value is0(automatic increase is disabled).The possible values range from
0to100. -
emergency_usage_threshold: Storage utilization threshold to trigger a storage increase right away, in percent. This is an optional setting. The default value is0(automatic increase is disabled).The possible values range from
0to100. -
disk_size_limit: Maximum object size after an increase, in bytes.
Warning
-
When using
planned_usage_threshold, make sure to specify themaintenance_windowsetting. -
If you specify both thresholds,
emergency_usage_thresholdmust not be less thanplanned_usage_threshold.
For more information about storage increase conditions, see this section.
-
To learn more about storage scaling rules, see this section.
-
-
maintenance_window: Maintenance window schedule. This setting is required only ifplanned_usage_thresholdis set. Contains the following parameters:day: Day of the week, inDDDformat, for scheduled maintenance.hour: Hour of day, inHHformat, for scheduled maintenance. The possible values range from1to24.
You can request the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
If the specified threshold is reached, the storage size increases differently depending on disk type:
-
For network HDDs and SSDs, by the higher of the two values: 20 GB or 20% of the current disk size.
-
For non-replicated SSDs and ultra high-speed network SSDs with three replicas, by 93 GB.
-
For local SSDs:
- In an Intel Broadwell or Intel Cascade Lake cluster, by 100 GB.
- Intel Ice Lake cluster, by 368 GB.
If the threshold is reached again, the storage size will be automatically increased until it reaches the specified maximum. After that, you can specify a new maximum storage size manually.