Managed Service for PostgreSQL storage

Managed Service for PostgreSQL allows you to use network and local storage drives for database clusters. Network drives are based on network blocks, which are virtual disks in the Yandex Cloud infrastructure. Local disks are physically located on the database host servers.

When creating a cluster, you can select the following disk types for data storage:

• Network HDDs (network-hdd): Most cost-effective option for clusters that do not require high read/write performance.

• Network SSDs (network-ssd): Balanced solution. Such disks are slower than local SSD storage, but, unlike local disks, they ensure data integrity if Yandex Cloud hardware fails.

• Non-replicated SSD (network-ssd-nonreplicated): Network disks with enhanced performance achieved by eliminating redundancy.

  You can increase the size of such disks only in increments of 93 GB.

• Ultra high-speed network SSDs with three replicas (network-ssd-io-m3): Network disks with the same performance characteristics as non-replicated ones. This disk type provides redundancy.

  You can increase the size of such disks only in increments of 93 GB.

• Local SSDs (local-ssd): Disks with the best performance.

  The size of such a storage can be increased:

  • For Intel Broadwell and Intel Cascade Lake: Only in 100 GB increments.
  • For Intel Ice Lake: In 368 GB increments only.

  For clusters with hosts residing in the ru-central1-d availability zone, local SSD storage is not available if using Intel Cascade Lake.

Selecting disk type when creating a clusterSelecting disk type when creating a cluster

The number of hosts you can create together with a PostgreSQL cluster depends on the selected disk type:

• You can create a cluster only with three or more hosts when using the following disk types:

  • Local SSDs (local-ssd)
  • Non-replicated SSDs (network-ssd-nonreplicated)

  This cluster will be fault-tolerant.

  Local SSD storage has an effect on how much a cluster will cost: you pay for it even if it is stopped. For more information, see Pricing policy.

• You can add any number of hosts within the current quota when using the following disk types:

  • Network HDDs (network-hdd)
  • Network SSDs (network-ssd)
  • Ultra high-speed network SSDs with three replicas (network-ssd-io-m3)

For more information about limits on the number of hosts per cluster, see Quotas and limits.

Disk encryptionDisk encryption

When creating or restoring a cluster from a backup, you can encrypt the storage disk with a custom KMS key. To encrypt a disk of an already created cluster, disable encryption, or encrypt a disk with a different key, create a backup of the cluster and restore it with the new settings.

To create an encrypted disk, you need the kms.keys.user role or higher.

If you deactivate the key used to encrypt a disk, access to the data will be suspended until you reactivate the key.

Disk space managementDisk space management

When the storage is more than 97% full, the host automatically switches to read-only mode. All DBs get the DEFAULT_TRANSACTION_READ_ONLY = TRUE setting through the ALTER DATABASE query.

In this mode, the INSERT, DELETE, or UPDATE queries result in an error.

You can monitor storage utilization on cluster hosts by setting up alerts in Yandex Monitoring:

Recovering a cluster from read-only modeRecovering a cluster from read-only mode

Use one of these methods:

• Increase the storage capacity to exceed the threshold value. Managed Service for PostgreSQL will then disable read-only mode automatically.

• Disable read-only mode manually and free up storage space by deleting some data.

  Alert

  When doing so, make sure the amount of free disk space never reaches zero. Otherwise, since the fail-safe mechanism is disabled, PostgreSQL will crash and the cluster will stop operating.

Automatic increase of storage sizeAutomatic increase of storage size

Automatic storage size increase prevents situations where the disk runs out of free space and hosts switch to read-only mode. The storage size increases upon reaching the specified threshold percentage of the total capacity. There are two thresholds:

• Scheduled increase threshold: To plan this increase, an algorithm analyzes data from the last few hours and estimates how quickly the storage is filling up. If the calculations show that the specified threshold will be exceeded by the start of the nearest maintenance window, the system schedules a storage increase. If a check at the maintenance start shows that the threshold was indeed exceeded, the storage size is increased.

• Immediate increase threshold: When reached, the storage size increases immediately.

You can use either one or both thresholds. If you set both, make sure the immediate increase threshold is higher than the scheduled one.

If the specified threshold is reached, the storage size increases differently depending on disk type:

• For network HDDs and SSDs, by the higher of the two values: 20 GB or 20% of the current disk size.

• For non-replicated SSDs and ultra high-speed network SSDs with three replicas, by 93 GB.

• For local SSDs:

  • In an Intel Broadwell or Intel Cascade Lake cluster, by 100 GB.
  • Intel Ice Lake cluster, by 368 GB.

If the threshold is reached again, the storage size will be automatically increased until it reaches the specified maximum. After that, you can specify a new maximum storage size manually.

You can configure automatic increase of storage size when creating or updating a cluster. If you set the scheduled increase threshold, you also need to configure the maintenance window schedule.

Use casesUse cases

• Writing data from a device into a database
• Writing load balancer logs to PostgreSQL
• Creating an MLFlow server for logging experiments and artifacts
• Monitoring the status of geographically distributed devices