Migrating PostgreSQL clusters

To migrate your database to Yandex Managed Service for PostgreSQL, you need to directly transfer the data, acquire a write lock for the old database, and switch over the load to the database cluster in Yandex Cloud.

Performing migration with Data Transfer lets you:

• Go without creating an intermediate VM or granting online access to your Managed Service for PostgreSQL target cluster.
• Minimize the DB downtime when migrating data.
• Migrate from older PostgreSQL versions to newer versions.

For more information, see What tasks Yandex Data Transfer is used for.

To use this migration method, allow connecting to the source cluster from the internet.

Transferring dataTransferring data

1. Prepare the source cluster.

2. Prepare the infrastructure:

   Manually

   Terraform

   1. Create a Managed Service for PostgreSQL target cluster in any suitable configuration. In which case:

      • The PostgreSQL version must be the same or higher than in the source cluster. You cannot perform migration while downgrading PostgreSQL version.
      • When creating a cluster, specify the same database name as in the source cluster.
      • Enable the same PostgreSQL extensions as in the source cluster.

   2. Prepare the target cluster.

   3. Create a source endpoint with the following parameters:

      • Database type: PostgreSQL.
      • Endpoint parameters → Connection settings: Custom installation.

      Specify the parameters for connecting to the source cluster.

   4. Create a target endpoint with the following parameters:

      • Database type: PostgreSQL.
      • Endpoint parameters → Connection settings: Managed Service for PostgreSQL cluster.

      Specify the ID of the target cluster.

   5. Create a transfer of the Snapshot and increment type that will use the created endpoints.

   6. Activate the transfer.

      Warning

      Abstain from making any changes to the data schema in the source and target clusters when the data transfer is running. For more information, see Working with databases during transfer.

   1. If you do not have Terraform yet, install it.

   2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

   3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

   4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

   5. Download the data-transfer-pgsql-mpg.tf configuration file to the same working directory.

      This file describes:

      • Network.
      • Subnet.
      • Security group and the rule required to connect to a cluster.
      • Managed Service for PostgreSQL target cluster.
      • Source endpoint.
      • Target endpoint.
      • Transfer.

   6. Specify the following in the data-transfer-pgsql-mpg.tf file:

      • Source endpoint parameters.

      • pg-extensions: List of PostgreSQL extensions in the source cluster.

      • Target cluster parameters also used as target endpoint parameters:

        • target_pgsql_version: PostgreSQL version. Must be the same or higher than in the source cluster.
        • target_user and target_password: Name and user password of the database owner.

   7. Check that the Terraform configuration files are correct using this command:

      terraform validate
      

      If there are any errors in the configuration files, Terraform will point them out.

   8. Create the required infrastructure:

      1. Run the command to view planned changes:

         terraform plan
         

         If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

      2. If you are happy with the planned changes, apply them:

         1. Run the command:

            terraform apply
            

         2. Confirm the update of resources.

         3. Wait for the operation to complete.

      All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

      Once created, your transfer will be activated automatically.

3. Wait for the transfer status to change to Replicating.

4. Switch the source cluster to read-only.

5. On the transfer monitoring page, wait for the Maximum data transfer delay metric to decrease to zero. This means that all changes that occurred in the source cluster after data copying was completed are transferred to the target cluster.

6. Deactivate the transfer and wait for its status to change to Stopped.

   For more information about transfer statuses, see Transfer lifecycle.

7. Transfer the load to the target cluster.

8. Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need:

   Manually created resources

   Resources created with Terraform

   • Delete the Managed Service for PostgreSQL cluster.
   • Delete the stopped transfer.
   • Delete endpoints for both the source and target.

   1. In the terminal window, go to the directory containing the infrastructure plan.

      Warning

      Make sure the directory has no Terraform manifests with the resources you want to keep. Terraform deletes all resources that were created using the manifests in the current directory.

   2. Delete resources:

      1. Run this command:

         terraform destroy
         

      2. Confirm deleting the resources and wait for the operation to complete.

      All the resources described in the Terraform manifests will be deleted.

Transferring tables with data types from PostgreSQL extensionsTransferring tables with data types from PostgreSQL extensions

Data Transfer allows you to copy tables, the columns of which contain data types defined in PostgreSQL and tables with derived types, i.e. arrays of these types and composite types with fields of these types. Currently, there is a limitation: the data type must implement binary I/O functions. This means that, for the type of data in the pg_type system view, the typsend and typreceive column values must not be zero.

For example, for the PostGIS extension, columns of the GEOMETRY, GEOMETRY_DUMP, and GEOGRAPHY types can be transferred, while those of the BOX2D and BOX3D types cannot.

See alsoSee also

For other migration methods, see the Yandex Managed Service for PostgreSQL documentation.