Contact UsGet started
© 2024 Iron Hive LLC Belgrade

Managing disk space

Written by
Updated at October 28, 2024

When the storage is more than 97% full, the host automatically switches to read-only mode.
To avoid issues with writing to the database, use one of the following methods:

Set up alerts in Yandex Monitoring

  1. Go to the folder page and select Monitoring.

  2. Select Managed Service for PostgreSQL.

  3. Create a notification channel.

  4. Create an alert with the following properties:

    1. Metrics: Set the following metric parameters:

      • Cloud

      • Folder

      • Managed Service for PostgreSQL service

      • Managed Service for PostgreSQL cluster ID

        You can get the cluster ID with a list of clusters in the folder.

      • disk.free_bytes label

    2. Alert condition: Set the Less than or equals condition for free disk space utilization percentage to trigger the alert:

      • Aggregation function: Minimum (minimum metric value for the period).
      • Warning: 90 (90% of the storage size).
      • Alarm: 95 (95% of the storage size).
      • Evaluation window: Preferred metric update period.

    3. Add the previously created notification channel.

Manually get the cluster out of the read-only mode

Alert

Do not allow free disk space to drop to zero during the following actions. Otherwise, since the fail-safe mechanism is disabled, PostgreSQL will crash and the cluster will stop operating.

To disable the read-only mode:

  1. Connect to the database in any appropriate way.

  2. Open a transaction and run the following command inside it:

    SET LOCAL transaction_read_only TO off;

  3. As part of the same transaction, delete the data you do not need using the DROP or TRUNCATE operators. Do not use the DELETE operator, as it marks rows as deleted but does not physically delete them from the database.

  4. Commit the transaction and restart all connections to the database.

For example, if your database contains a table called ExcessDataTable1 that you no longer need, delete it using the following transaction:

BEGIN;
SET LOCAL transaction_read_only TO off;
DROP TABLE ExcessDataTable1;
COMMIT;

Increasing storage size

Note

Some PostgreSQL settings depend on the storage size.

Make sure the cloud has enough quota to increase the storage size. Open the cloud's Quotas page and make sure there is space available under Managed Databases in the HDD storage capacity or SSD storage capacity line.

Warning

  • You cannot decrease the storage size.
  • While resizing the storage, cluster hosts will be unavailable.

To increase the cluster storage size:

  1. Go to the folder page and select Managed Service for PostgreSQL.
  2. Select a cluster and click Edit in the top panel.
  3. Under Size of storage, specify the required value.
  4. Click Save changes.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To increase the cluster storage size:

  1. View a description of the update cluster CLI command:

    yc managed-postgresql cluster update --help

  2. Specify the required storage in the cluster update command (it must be at least as large as disk_size in the cluster properties):

    yc managed-postgresql cluster update <cluster_name_or_ID> \
    --disk-size <storage_size_in_GB>

To increase the cluster storage size:

  1. Open the current Terraform configuration file with an infrastructure plan.

    For more information about creating this file, see Creating clusters.

    For a complete list of available Managed Service for PostgreSQL cluster configuration fields, see the Terraform provider documentation.

  2. In the Managed Service for PostgreSQL cluster description, change the disk_size attribute value under config.resources:

    resource "yandex_mdb_postgresql_cluster" "<cluster_name>" {
  ...
  config {
    resources {
      disk_size = <storage_size_in_GB>
      ...
    }
  }
}

  3. Make sure the settings are correct.

    1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.

    2. Run the command:

      terraform validate

      If there are errors in the configuration files, Terraform will point to them.

  4. Confirm updating the resources.

    1. Run the command to view planned changes:

      terraform plan

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

    2. If you are happy with the planned changes, apply them:

      1. Run the command:

        terraform apply

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

    Time limits

    A Terraform provider sets the timeout for Managed Service for PostgreSQL cluster operations:

    • Creating a cluster, including restoring from a backup: 30 minutes.
    • Editing a cluster: 60 minutes.
    • Deleting a cluster: 15 minutes.

    Operations exceeding the set timeout are interrupted.

    How do I change these limits?

    Add the timeouts block to the cluster description, for example:

    resource "yandex_mdb_postgresql_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

  1. Get an IAM token for API authentication and put it into the environment variable:

    export IAM_TOKEN="<IAM_token>"

  2. Use the Cluster.update method and make a request, e.g., via cURL:

    Warning

    This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask parameter (one line separated by commas).

    curl \
  --request PATCH \
  --header "Authorization: Bearer $IAM_TOKEN" \
  --header "Content-Type: application/json" \
  --url 'https://mdb.api.cloud.yandex.net/managed-postgresql/v1/clusters/<cluster_ID>' \
  --data '{
            "updateMask": "configSpec.resources.diskSize",
            "configSpec": {
              "resources": {
                "diskSize": "<storage_size_in_bytes>"
              }
            }
          }'

    Where:

    • updateMask: List of parameters to update as a single string, separated by commas.

      Only one parameter is provided in this case.

    • configSpec.resources.diskSize: New storage size in bytes.

    You can get the cluster ID with a list of clusters in the folder.

  3. View the server response to make sure the request was successful.

  1. Get an IAM token for API authentication and put it into the environment variable:

    export IAM_TOKEN="<IAM_token>"

  2. Clone the cloudapi repository:

    cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi

    Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

  3. Use the ClusterService/Update call and make a request, e.g., via gRPCurl:

    Warning

    This API method will assign default values to all object parameters not explicitly set in the request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

    Format for listing settings
    "update_mask": {
    "paths": [
        "<setting_1>",
        "<setting_2>",
        ...
        "<setting_N>"
    ]
}

    grpcurl \
  -format json \
  -import-path ~/cloudapi/ \
  -import-path ~/cloudapi/third_party/googleapis/ \
  -proto ~/cloudapi/yandex/cloud/mdb/postgresql/v1/cluster_service.proto \
  -rpc-header "Authorization: Bearer $IAM_TOKEN" \
  -d '{
        "cluster_id": "<cluster_ID>",
        "update_mask": {
          "paths": [
            "config_spec.resources.disk_size"
          ]
        },
        "config_spec": {
          "resources": {
            "disk_size": "<storage_size_in_bytes>"
          }
        }
      }' \
  mdb.api.cloud.yandex.net:443 \
  yandex.cloud.mdb.postgresql.v1.ClusterService.Update

    Where:

    • update_mask: List of parameters to update as an array of paths[] strings.

      Only one parameter is provided in this case.

    • config_spec.resources.disk_size: New storage size in bytes.

    You can get the cluster ID with a list of clusters in the folder.

  4. View the server response to make sure the request was successful.

Setting up automatic increase of storage size

Note

Some PostgreSQL settings depend on the storage size.

Make sure the cloud has enough quota to increase the storage size. Open the cloud's Quotas page and make sure there is space available under Managed Databases in the HDD storage capacity or SSD storage capacity line.

Warning

  • You cannot decrease the storage size.
  • While resizing the storage, cluster hosts will be unavailable.

  1. Go to the folder page and select Managed Service for PostgreSQL.

  2. Select the cluster and click Edit in the top panel.

  3. Under Automatic increase of storage size:

    1. In the Increase size field, set the storage utilization percentage to trigger storage increase. You can configure the increase to take place:

      You can enable both rules, but the threshold for immediate increase should be higher than that for increase during the maintenance window.

    2. In the Maximum storage size field, specify the maximum storage size that can be set when increasing the storage size automatically.

  4. Click Save changes.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To set up automatic increase of storage size:

  1. View a description of the update cluster CLI command:

    yc managed-postgresql cluster update --help

  2. Set the maximum storage size and conditions for its increase in the update cluster command.

    Make sure the maximum storage size is greater than the disk_size value in the cluster properties. The percentage for immediate increase should be higher than that for increase during the next maintenance window.

    yc managed-postgresql cluster update <cluster_ID_or_name> \
    --disk-size-autoscaling disk-size-limit=<maximum_storage_size_in_bytes>,`
                           `planned-usage-threshold=<scheduled_increase_percentage>,`
                           `emergency-usage-threshold=<immediate_increase_percentage>

    If you have set up the storage size to increase within the maintenance window, set up a schedule for the maintenance window.

  1. Get an IAM token for API authentication and put it into the environment variable:

    export IAM_TOKEN="<IAM_token>"

  2. Use the Cluster.update method and make a request, e.g., via cURL:

    Warning

    This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask parameter (one line separated by commas).

    curl \
  --request PATCH \
  --header "Authorization: Bearer $IAM_TOKEN" \
  --header "Content-Type: application/json" \
  --url 'https://mdb.api.cloud.yandex.net/managed-postgresql/v1/clusters/<cluster_ID>' \
  --data '{
            "updateMask": "configSpec.diskSizeAutoscaling,maintenanceWindow",
            "configSpec": {
              "diskSizeAutoscaling": {
                "plannedUsageThreshold": "<scheduled_increase_percentage>",
                "emergencyUsageThreshold": "<immediate_increase_percentage>",
                "diskSizeLimit": "<maximum_storage_size_in_bytes>"
              }
            },
            "maintenanceWindow": {
              "weeklyMaintenanceWindow": {
                "day": "<day_of_week>",
                "hour": "<hour>"
              }
            }
          }'

    Where:

    • updateMask: List of parameters to update as a single string, separated by commas.

      In this case, provide only configSpec.diskSizeAutoscaling and maintenanceWindow.

    • configSpec.diskSizeAutoscaling: Automatic storage size increase settings:

      • plannedUsageThreshold: Storage utilization percentage to trigger a storage increase during the next maintenance window.

        Use a percentage value between 0 and 100. The default value is 0 (automatic increase is disabled).

        If you set this parameter, configure the maintenance window schedule in the maintenanceWindow parameter.

      • emergencyUsageThreshold: Storage utilization percentage to trigger an immediate storage increase.

        Use a percentage value between 0 and 100. The default value is 0 (automatic increase is disabled). This parameter value must be greater than or equal to plannedUsageThreshold.

      • diskSizeLimit: Maximum storage size, in bytes, that can be set when utilization reaches one of the specified percentages.

    • maintenanceWindow: Maintenance window schedule. It is required only if the plannedUsageThreshold parameter is set. Contains the following parameters:

      • day: Day of week, in DDD format, for scheduled maintenance.
      • hour: Hour, in HH format, for scheduled maintenance. The values range from 1 to 24.

    You can get the cluster ID with a list of clusters in the folder.

  3. View the server response to make sure the request was successful.

  1. Get an IAM token for API authentication and put it into the environment variable:

    export IAM_TOKEN="<IAM_token>"

  2. Clone the cloudapi repository:

    cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi

    Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

  3. Use the ClusterService/Update call and make a request, e.g., via gRPCurl:

    Warning

    This API method will assign default values to all object parameters not explicitly set in the request. To avoid this, list the settings you want to change in the update_mask parameter as an array of paths[] strings.

    Format for listing settings
    "update_mask": {
    "paths": [
        "<setting_1>",
        "<setting_2>",
        ...
        "<setting_N>"
    ]
}

    grpcurl \
  -format json \
  -import-path ~/cloudapi/ \
  -import-path ~/cloudapi/third_party/googleapis/ \
  -proto ~/cloudapi/yandex/cloud/mdb/postgresql/v1/cluster_service.proto \
  -rpc-header "Authorization: Bearer $IAM_TOKEN" \
  -d '{
        "cluster_id": "<cluster_ID>",
        "update_mask": {
          "paths": [
            "config_spec.disk_size_autoscaling",
            "maintenance_window"
          ]
        },
        "config_spec": {
          "disk_size_autoscaling": {
            "planned_usage_threshold": "<scheduled_increase_percentage>",
            "emergency_usage_threshold": "<immediate_increase_percentage>",
            "disk_size_limit": "<maximum_storage_size_in_bytes>"
          }
        },
        "maintenance_window": {
          "weekly_maintenance_window": {
            "day": "<day_of_week>",
            "hour": "<hour>"
          }
        }
      }' \
  mdb.api.cloud.yandex.net:443 \
  yandex.cloud.mdb.postgresql.v1.ClusterService.Update

    Where:

    • update_mask: List of parameters to update as an array of paths[] strings.

      In this case, provide only config_spec.disk_size_autoscaling and maintenance_window.

    • config_spec.disk_size_autoscaling: Automatic storage size increase settings:

      • planned_usage_threshold: Storage utilization percentage to trigger a storage increase during the next maintenance window.

        Use a percentage value between 0 and 100. The default value is 0 (automatic increase is disabled).

        If you set this parameter, configure the maintenance window schedule in the maintenance_window parameter.

      • emergency_usage_threshold: Storage utilization percentage to trigger an immediate storage increase.

        Use a percentage value between 0 and 100. The default value is 0 (automatic increase is disabled). This parameter value must be greater than or equal to planned_usage_threshold.

      • disk_size_limit: Maximum storage size, in bytes, that can be set when utilization reaches one of the specified percentages.

    • maintenance_window: Maintenance window schedule. It is required only if the planned_usage_threshold parameter is set. Contains the following parameters:

      • day: Day of week, in DDD format, for scheduled maintenance.
      • hour: Hour, in HH format, for scheduled maintenance. The values range from 1 to 24.

    You can get the cluster ID with a list of clusters in the folder.

  4. View the server response to make sure the request was successful.

If the specified threshold is reached, the storage size increases differently depending on disk type:

  • For network HDDs and SSDs, by the higher of the two values: 20 GB or 20% of the current disk size.

  • For non-replicated SSDs, by 93 GB.

  • For local SSDs:

    • In an Intel Broadwell or Intel Cascade Lake cluster, by 100 GB.
    • Intel Ice Lake cluster, by 368 GB.

If the threshold is reached again, the storage size will be automatically increased until it reaches the specified maximum. After that, you can specify a new maximum storage size manually.

Previous
Managing backups
Next
Updating the PostgreSQL version