Transferring data from a Greenplum®/Apache Cloudberry™ source endpoint

Connection type: Select a database connection option:

• Self-managed: Allows you to specify connection settings manually.

  Select Managed Service for Greenplum cluster as the installation type and configure these settings:

  • Cluster: Select the cluster to connect to.
  • Database: Specify the name of the database in the selected cluster.
  • User: Specify the username Data Transfer will use to connect to the database.
  • Password: Enter the user password for access to the database.

• Connection Manager: Allows connecting to the cluster via Yandex Connection Manager:

  • Select the folder with the Yandex MPP Analytics for PostgreSQL cluster.

  • Select Managed DB cluster as the installation type and configure these settings:

    • Cluster for Managed DB: Select the cluster to connect to.
    • Connection: Select or create a connection in Connection Manager.
    • Database: Specify the name of the database in the selected cluster.

  Warning

  To use a connection from Connection Manager, the user must have access permissions for this connection of connection-manager.user or higher.

Security groups: Select the cloud network to host the endpoint and security groups for network traffic.

Thus, you will be able to apply the specified security group rules to the VMs and clusters in the selected network without changing the settings of these VMs and clusters. For more information, see Networking in Yandex Data Transfer.

Connection type: Select a database connection option:

• Self-managed: Allows you to specify connection settings manually:

  • Coordinator host: Specify the IP or FQDN of the primary master host to connect to.

  • Coordinator port: Specify the port for Data Transfer to use to connect to the primary master host.

  • Coordinator mirror host: Specify the IP address or FQDN of the standby master host to connect to (leave the field empty if your cluster only has one master host).

  • Coordinator mirror port: Specify the port for Data Transfer to use to connect to the standby master host (leave the field empty if there is only one master host in your cluster).

  • Cluster Segments: Specify segment host connection information. If you omit these, segment host addresses will be retrieved automatically from the master host housekeeping table.

  • CA certificate: Upload the certificate file or add its contents as text if data encryption is required, e.g., to comply with the PCI DSS requirements.

    Warning

    If no certificate is added, the transfer may fail with an error.

  • Subnet ID: Select or create a subnet in the required availability zone. The transfer will use this subnet to access the database.

    If this field has a value specified for both endpoints, both subnets must be hosted in the same availability zone.

  • Database: Enter the database name.

  • User: Specify the username Data Transfer will use to connect to the database.

  • Password: Enter the user password for access to the database.

• Connection Manager: Allows connecting to a database using Yandex Connection Manager:

  • Select the folder where the Connection Manager connection was created.

  • Select Custom installation as the installation type and configure these settings:

    • Connection: Select or create a connection in Connection Manager.

    • Database: Specify the database name in the custom installation.

    • Subnet ID: Select or create a subnet in the required availability zone. The transfer will use this subnet to access the database.

      If this field has a value specified for both endpoints, both subnets must be hosted in the same availability zone.

    Warning

    To use a connection from Connection Manager, the user must have access permissions for this connection of connection-manager.user or higher.

Security groups: Select the cloud network to host the endpoint and security groups for network traffic.

This will allow you to apply the specified security group rules to VMs and DBs in the selected network without reconfiguring these VMs and DBs. For more information, see Networking in Yandex Data Transfer.

Included tables: Only data from the tables listed here will be transferred.

If a table is partitioned, you can use this field to specify both the entire table and individual partitions.

Make sure that, for tables to be included in the list, all the necessary privileges are granted to the user to perform the data transfer.

When you add new tables when editing an endpoint used in Snapshot and increment or Replication transfers with the Replicating status, the data history for these tables will not get uploaded. To add a table with its historical data, use the List of objects for transfer field in the transfer settings.

Excluded tables: Data from the listed tables is not transferred.

If a table is partitioned, to exclude it from the list, make sure to list all of its partitions.

The lists include the name of the schema that describes the DB contents, structure, and integrity constraints, as well as the table name. Both lists support expressions in the following format:

• <schema_name>.<table_name>: Full table name.
• <schema_name>.*: All tables in the specified schema.
• <table_name>: Table in the default schema.

The names of the tables you include or exclude must follow the Greenplum® and Apache Cloudberry™ identifier naming rules. Double quotes within a table name are not supported. Outer quotes are only used as delimiters and will be deleted when processing paths.

Snapshot consistency: When enabled, Data Transfer will apply additional steps to the source to assure snapshot consistency.

Service object schema: Schema for placing auxiliary objects of the transfer.

The schema name must meet the ID naming rules in Greenplum®. Double quotes are not supported in schema names.

Use direct reading from segments: Disables gpfdist for transfers from Greenplum® to Greenplum®.