Managing disk space
When the storage is more than 97% full, the host automatically switches to read-only mode.
To avoid issues with writing to the database, use one of the following methods:
-
Set up alerts in Yandex Monitoring to monitor storage utilization.
-
Manually get the cluster out of the read-only mode and free up the storage space by deleting some data.
-
Increase the storage size to automatically disable the read-only mode.
Set up alerts in Yandex Monitoring
-
Go to the folder page and select Monitoring.
-
Select Managed Service for PostgreSQL.
-
Create an alert with the following properties:
-
Metrics: Set the following metric parameters:
-
Cloud
-
Folder
-
Managed Service for PostgreSQL service
-
Managed Service for PostgreSQL cluster ID
You can get the cluster ID with a list of clusters in the folder.
-
disk.free_bytes
label
-
-
Alert condition: Set the
Less than or equals
condition for free disk space utilization percentage to trigger the alert:- Aggregation function:
Minimum
(minimum metric value for the period). - Warning:
90
(90% of the storage size). - Alarm:
95
(95% of the storage size). - Evaluation window: Preferred metric update period.
- Aggregation function:
-
Add the previously created notification channel.
-
Manually get the cluster out of the read-only mode
Alert
Do not allow free disk space to drop to zero during the following actions. Otherwise, since the fail-safe mechanism is disabled, PostgreSQL will crash and the cluster will stop operating.
To disable the read-only mode:
-
Connect to the database in any appropriate way.
-
Open a transaction and run the following command inside it:
SET LOCAL transaction_read_only TO off;
-
As part of the same transaction, delete the data you do not need using the
DROP
orTRUNCATE
operators. Do not use theDELETE
operator, as it marks rows as deleted but does not physically delete them from the database. -
Commit the transaction and restart all connections to the database.
For example, if your database contains a table called
ExcessDataTable1
that you no longer need, delete it using the following transaction:
BEGIN; SET LOCAL transaction_read_only TO off; DROP TABLE ExcessDataTable1; COMMIT;
Increasing storage size
Note
Some PostgreSQL settings depend on the storage size.
Make sure the cloud has enough quota to increase the storage size. Open the cloud's Quotas
Warning
- You cannot decrease the storage size.
- While resizing the storage, cluster hosts will be unavailable.
To increase the cluster storage size:
- Go to the folder page and select Managed Service for PostgreSQL.
- Select a cluster and click
Edit in the top panel. - Under Size of storage, specify the required value.
- Click Save changes.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To increase the cluster storage size:
-
View the description of the update cluster CLI command:
yc managed-postgresql cluster update --help
-
Specify the required storage in the cluster update command (it must be at least as large as
disk_size
in the cluster properties):yc managed-postgresql cluster update <cluster_name_or_ID> \ --disk-size <storage_size_in_GB>
To increase the cluster storage size:
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about creating this file, see Creating clusters.
For a complete list of available Managed Service for PostgreSQL cluster configuration fields, see the Terraform provider documentation
. -
In the Managed Service for PostgreSQL cluster description, change the
disk_size
attribute value underconfig.resources
:resource "yandex_mdb_postgresql_cluster" "<cluster_name>" { ... config { resources { disk_size = <storage_size_in_GB> ... } } }
-
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
Time limits
A Terraform provider sets the timeout for Managed Service for PostgreSQL cluster operations:
- Creating a cluster, including restoring from a backup: 30 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?
Add the
timeouts
block to the cluster description, for example:resource "yandex_mdb_postgresql_cluster" "<cluster_name>" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } }
-
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>"
-
Use the Cluster.update method and make a request, e.g., via cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMask
parameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-postgresql/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.resources.diskSize", "configSpec": { "resources": { "diskSize": "<storage_size_in_bytes>" } } }'
Where:
-
updateMask
: List of parameters to update as a single string, separated by commas.Only one parameter is provided in this case.
-
configSpec.resources.diskSize
: New storage size in bytes.
You can get the cluster ID with a list of clusters in the folder.
-
-
View the server response to make sure the request was successful.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>"
-
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
Below, we assume the repository contents are stored in the
~/cloudapi/
directory. -
Use the ClusterService/Update call and make a request, e.g., via gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_mask
parameter as an array ofpaths[]
strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }
grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/postgresql/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.resources.disk_size" ] }, "config_spec": { "resources": { "disk_size": "<storage_size_in_bytes>" } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.postgresql.v1.ClusterService.Update
Where:
-
update_mask
: List of parameters to update as an array ofpaths[]
strings.In this case, only one parameter is provided.
-
config_spec.resources.disk_size
: New storage size in bytes.
You can get the cluster ID with a list of clusters in the folder.
-
-
View the server response to make sure the request was successful.
Setting up automatic increase of storage size
Note
Some PostgreSQL settings depend on the storage size.
Make sure the cloud has enough quota to increase the storage size. Open the cloud's Quotas
Warning
- You cannot decrease the storage size.
- While resizing the storage, cluster hosts will be unavailable.
-
Go to the folder page and select Managed Service for PostgreSQL.
-
Select the cluster and click Edit in the top panel.
-
Under Automatic increase of storage size:
-
In the Increase size field, set the storage utilization percentage to trigger storage increase. You can configure the increase to take place:
- During the next maintenance window.
- Right away.
You can enable both rules, but the threshold for immediate increase should be higher than that for increase during the maintenance window.
-
In the Maximum storage size field, specify the maximum storage size that can be set when increasing the storage size automatically.
-
-
Click Save changes.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To set up automatic increase of storage size:
-
View the description of the update cluster CLI command:
yc managed-postgresql cluster update --help
-
Set the maximum storage size and conditions for its increase in the update cluster command.
Make sure the maximum storage size is greater than the
disk_size
value in the cluster properties. The percentage for immediate increase should be higher than that for increase during the next maintenance window.yc managed-postgresql cluster update <cluster_ID_or_name> \ --disk-size-autoscaling disk-size-limit=<maximum_storage_size_in_bytes>,` `planned-usage-threshold=<scheduled_increase_percentage>,` `emergency-usage-threshold=<immediate_increase_percentage>
If you have set up the storage size to increase within the maintenance window, set up a schedule for the maintenance window.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>"
-
Use the Cluster.update method and make a request, e.g., via cURL
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMask
parameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-postgresql/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.diskSizeAutoscaling,maintenanceWindow", "configSpec": { "diskSizeAutoscaling": { "plannedUsageThreshold": "<scheduled_increase_percentage>", "emergencyUsageThreshold": "<immediate_increase_percentage>", "diskSizeLimit": "<maximum_storage_size_in_bytes>" } }, "maintenanceWindow": { "weeklyMaintenanceWindow": { "day": "<day_of_week>", "hour": "<hour>" } } }'
Where:
-
updateMask
: List of parameters to update as a single string, separated by commas.In this case, provide only
configSpec.diskSizeAutoscaling
andmaintenanceWindow
. -
configSpec.diskSizeAutoscaling
: Automatic storage size increase settings:-
plannedUsageThreshold
: Storage utilization percentage to trigger a storage increase during the next maintenance window.Use a percentage value between
0
and100
. The default value is0
(automatic increase is disabled).If you set this parameter, configure the maintenance window schedule in the
maintenanceWindow
parameter. -
emergencyUsageThreshold
: Storage utilization percentage to trigger an immediate storage increase.Use a percentage value between
0
and100
. The default value is0
(automatic increase is disabled). This parameter value must be greater than or equal toplannedUsageThreshold
. -
diskSizeLimit
: Maximum storage size, in bytes, that can be set when utilization reaches one of the specified percentages.
-
-
maintenanceWindow
: Maintenance window schedule. It is required only if theplannedUsageThreshold
parameter is set. Contains the following parameters:day
: Day of week, inDDD
format, for scheduled maintenance.hour
: Hour, inHH
format, for scheduled maintenance. The values range from1
to24
.
You can get the cluster ID with a list of clusters in the folder.
-
-
View the server response to make sure the request was successful.
-
Get an IAM token for API authentication and put it into the environment variable:
export IAM_TOKEN="<IAM_token>"
-
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
Below, we assume the repository contents are stored in the
~/cloudapi/
directory. -
Use the ClusterService/Update call and make a request, e.g., via gRPCurl
:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_mask
parameter as an array ofpaths[]
strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }
grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/postgresql/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.disk_size_autoscaling", "maintenance_window" ] }, "config_spec": { "disk_size_autoscaling": { "planned_usage_threshold": "<scheduled_increase_percentage>", "emergency_usage_threshold": "<immediate_increase_percentage>", "disk_size_limit": "<maximum_storage_size_in_bytes>" } }, "maintenance_window": { "weekly_maintenance_window": { "day": "<day_of_week>", "hour": "<hour>" } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.postgresql.v1.ClusterService.Update
Where:
-
update_mask
: List of parameters to update as an array ofpaths[]
strings.In this case, provide only
config_spec.disk_size_autoscaling
andmaintenance_window
. -
config_spec.disk_size_autoscaling
: Automatic storage size increase settings:-
planned_usage_threshold
: Storage utilization percentage to trigger a storage increase during the next maintenance window.Use a percentage value between
0
and100
. The default value is0
(automatic increase is disabled).If you set this parameter, configure the maintenance window schedule in the
maintenance_window
parameter. -
emergency_usage_threshold
: Storage utilization percentage to trigger an immediate storage increase.Use a percentage value between
0
and100
. The default value is0
(automatic increase is disabled). This parameter value must be greater than or equal toplanned_usage_threshold
. -
disk_size_limit
: Maximum storage size, in bytes, that can be set when utilization reaches one of the specified percentages.
-
-
maintenance_window
: Maintenance window schedule. It is required only if theplanned_usage_threshold
parameter is set. Contains the following parameters:day
: Day of week, inDDD
format, for scheduled maintenance.hour
: Hour, inHH
format, for scheduled maintenance. The values range from1
to24
.
You can get the cluster ID with a list of clusters in the folder.
-
-
View the server response to make sure the request was successful.
If the specified threshold is reached, the storage size increases differently depending on disk type:
-
For network HDDs and SSDs, by the higher of the two values: 20 GB or 20% of the current disk size.
-
For non-replicated SSDs, by 93 GB.
-
For local SSDs:
- In an Intel Broadwell or Intel Cascade Lake cluster, by 100 GB.
- Intel Ice Lake cluster, by 368 GB.
If the threshold is reached again, the storage size will be automatically increased until it reaches the specified maximum. After that, you can specify a new maximum storage size manually.