Managed Service for PostgreSQL storage

Managed Service for PostgreSQL allows you to use network and local storage drives for database clusters. Network drives are based on network blocks, which are virtual disks in the Yandex Cloud infrastructure. Local disks are physically located on the database host servers.

When creating a cluster, you can select the following disk types for data storage:

• Network HDDs (network-hdd): Most cost-effective option for clusters with low requirements for read and write performance.

• Network SSDs (network-ssd): Compromise solution: slower than local SSDs, network SSDs ensure data integrity in the event of Yandex Cloud hardware failure.

• Non-replicated SSDs (network-ssd-nonreplicated): Network disks with higher performance achieved by eliminating redundancy.

  You can only increase the size of these disks in 93 GB increments.

• Ultra high-speed network SSDs with three replicas (network-ssd-io-m3): Network disks that deliver performance equivalent to non-replicated SSDs while ensuring redundancy.

  You can only increase the size of these disks in 93 GB increments.

• Local SSDs (local-ssd): The highest-performing disks.

  You can expand such a storage as follows:

  • For Intel Broadwell and Intel Cascade Lake: Only in 100 GB increments.
  • For Intel Ice Lake and AMD Zen 4: Only in 368 GB increments.

  For clusters with hosts residing in the ru-central1-d availability zone, local SSD storage is not available if using Intel Cascade Lake.

Selecting disk type when creating a clusterSelecting disk type when creating a cluster

The number of hosts you can create together with a PostgreSQL cluster depends on the selected disk type:

• You can create a cluster only with three or more hosts when using the following disk types:

  • Local SSDs (local-ssd)
  • Non-replicated SSDs (network-ssd-nonreplicated)

  This cluster will be fault-tolerant.

  Local SSD storage has an effect on how much a cluster will cost: you pay for it even if it is stopped. For more information, see Pricing policy.

• You can add any number of hosts within the current quota when using the following disk types:

  • Network HDDs (network-hdd)
  • Network SSDs (network-ssd)
  • Ultra high-speed network SSDs with three replicas (network-ssd-io-m3)

For more information about limits on the number of hosts per cluster, see Quotas and limits.

Disk encryptionDisk encryption

When creating or restoring a cluster from a backup, you can encrypt the storage disk with a custom KMS key. To encrypt a disk of an already created cluster, disable encryption, or encrypt a disk with a different key, create a backup of the cluster and restore it with the new settings.

To create an encrypted disk, you need the kms.keys.user role or higher.

If you deactivate the key used to encrypt a disk, access to the data will be suspended until you reactivate the key.

Disk space managementDisk space management

When the storage is more than 97% full, the host automatically switches to read-only mode. All DBs get the DEFAULT_TRANSACTION_READ_ONLY = TRUE setting through the ALTER DATABASE query.

In this mode, the INSERT, DELETE, or UPDATE queries result in an error.

You can monitor storage utilization on cluster hosts by setting up alerts in Yandex Monitoring:

Recovering a cluster from read-only modeRecovering a cluster from read-only mode

Use one of these methods:

• Increase the storage capacity to exceed the threshold value. Managed Service for PostgreSQL will then disable read-only mode automatically.

• Disable read-only mode manually and free up storage space by deleting some data.

  Alert

  When doing so, make sure the amount of free disk space never reaches zero. Otherwise, since the fail-safe mechanism is disabled, PostgreSQL will crash and the cluster will stop operating.

Automatic increase of storage sizeAutomatic increase of storage size

Automatic storage size increase prevents situations where the disk runs out of free space and hosts switch to read-only mode. The storage size increases upon reaching the specified threshold percentage of the total capacity. There are two thresholds:

• Scheduled increase threshold: To plan this increase, an algorithm analyzes data from the last few hours and estimates how quickly the storage is filling up. If the calculations show that the specified threshold will be exceeded by the start of the nearest maintenance window, the system schedules a storage increase. If a check at the maintenance start shows that the threshold was indeed exceeded, the storage size is increased.

• Immediate increase threshold: When reached, the storage size increases immediately.

You can use either one or both thresholds. If you set both, make sure the immediate increase threshold is higher than the scheduled one.

Upon reaching the specified threshold, the storage expands with the increase size depending on the disk type:

• For network HDDs and SSDs, the increase is the larger of these two values: 20 GB or 20% of the current disk size.

• For non-replicated SSDs and ultra-fast network SSDs with triple replication, the increase is 93 GB.

• For local SSDs:

  • In an Intel Broadwell or Intel Cascade Lake cluster, the increase is 100 GB.
  • In Intel Ice Lake cluster, the increase is 368 GB.

If the threshold is triggered again, the storage size will automatically increase once more. This process will repeat until the storage size reaches the specified maximum. After that, you can specify a new maximum storage size manually.

You can configure automatic increase of storage size when creating or updating a cluster. If you set the scheduled increase threshold, you also need to configure the maintenance window schedule.

Use casesUse cases

• Writing data from a device into a database
• Writing load balancer logs to PostgreSQL
• Creating an MLFlow server for logging experiments and artifacts
• Monitoring the status of geographically distributed devices