Contact UsGet started

Managing endpoints

Written by
Updated at June 28, 2024

The source endpoint describes the settings of the database from which the information will be transferred using Data Transfer, while the target endpoint, the database you are going to transfer the data to. You can create, edit, clone, or delete such endpoints.

Getting a list of endpoints

  1. Go to the folder page and select Yandex Data Transfer.
  2. In the left-hand panel, select Endpoints.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To get a list of transfers in a folder, run the following command:

yc datatransfer endpoint list

Use the list API method.

Getting detailed information about an endpoint

  1. Go to the folder page and select Yandex Data Transfer.
  2. In the left-hand panel, select Endpoints.
  3. Click the required endpoint name.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To get information about an endpoint, run this command:

yc datatransfer endpoint get <endpoint_ID>

You can get the endpoint ID with a list of endpoints in the folder.

Use the get API method and provide the endpoint ID value in the endpointId request parameter.

To find out the endpoint ID, get a list of endpoints in the folder.

Creating an endpoint

Warning

To create or edit an endpoint of a managed database, you need the service or primitive viewer role assigned for the folder hosting a cluster of this managed database.

  1. Go to the folder page and select Yandex Data Transfer.

  2. In the left-hand panel, select Endpoints.

  3. Click Create endpoint.

  4. In the Direction field, select Source or Target.

  5. Enter a name for the endpoint. Use lowercase Latin letters and numbers.

  6. (Optional) Enter a description of the endpoint.

  7. Select the appropriate value in the Database type field. This will create an endpoint of the relevant type.

  8. Set the endpoint parameters:

  9. Click Create endpoint.

Note

You can create endpoints using the CLI commands only for sources and targets of the ClickHouse®, MongoDB, MySQL®, and PostgreSQL types.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

  1. View a description of the CLI create endpoint command:

    yc datatransfer endpoint create --help

  2. Specify the endpoint parameters in the create command (only some of the supported parameters are given in the example):

    yc datatransfer endpoint create <endpoint_type> \
  --name <endpoint_name> \
  --description <endpoint_description> \
  <endpoint_parameters>

    You can view the endpoint type and parameters in the settings section for the appropriate data source or target.

Note

You can create endpoints with Terraform only for ClickHouse®, MongoDB, MySQL®, and PostgreSQL sources and targets.

Terraform allows you to quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. Configuration files store the infrastructure description in the HashiCorp Configuration Language (HCL). Terraform and its providers are distributed under the Business Source License.

For more information about the provider resources, see the documentation on the Terraform website or the mirror.

If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

If you don't have Terraform, install it and configure the Yandex Cloud provider.

  1. Create a configuration file with a description of your endpoint.

    Here is an example of the configuration file structure:

    resource "ycp_datatransfer_endpoint" "<endpoint_name_in_Terraform>" {
  name = "<endpoint_name>"
  settings {
    <endpoint_type> {
      <endpoint_parameters>
    }
  }
}

You can retrieve a complete resource schema by running terraform providers schema.

  1. You can view the endpoint type and parameters in the settings section for the appropriate data source or target.

  2. Make sure the settings are correct.

    1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.

    2. Run the command:

      terraform validate

      If there are errors in the configuration files, Terraform will point to them.

  3. Confirm updating the resources.

    1. Run the command to view planned changes:

      terraform plan

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

    2. If you are happy with the planned changes, apply them:

      1. Run the command:

        terraform apply

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

  4. For more information, see the Terraform provider documentation.

Note

You can create endpoints with the API only for ClickHouse®, Apache Kafka®, MongoDB, MySQL®, and PostgreSQL sources and targets.

Use the create API method and include the following information in the request:

  • ID of the folder to contain the endpoint in the folderId parameter.
  • Endpoint name in the name parameter.
  • Endpoint description in the description parameter.
  • Endpoint parameters in the settings parameter.

You can view the endpoint parameters in the settings section for the appropriate data source or target.

Updating the endpoint

Warning

To create or edit an endpoint of a managed database, you need the service or primitive viewer role assigned for the folder hosting a cluster of this managed database.

Note

You cannot change the endpoint type and the database type. For some endpoints, you cannot change the connection type, either.

  1. Go to the folder page and select Yandex Data Transfer.

  2. In the left-hand panel, select Endpoints.

  3. Select an endpoint and click Edit in the top panel.

  4. Edit the endpoint parameters:

  5. Click Apply.

Note

You can change endpoints using the CLI commands only for ClickHouse®, MongoDB, MySQL®, and PostgreSQL sources and targets.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To update the endpoint settings:

  1. View a description of the update transfer CLI command:

    yc datatransfer endpoint update --help

  2. Run the following command by providing a list of the settings to change (the example below does not contain a full list of parameters):

    yc datatransfer transfer update <endpoint_type> <endpoint_ID> \
  --name <endpoint_name> \
  --description <endpoint_description> \
  <endpoint_parameters>

    You can view the endpoint type and parameters in the settings section for the appropriate data source or target. You can get the endpoint ID with a list of endpoints in the folder.

  1. Open the current Terraform configuration file with the endpoint description.

    To learn how to create such a file, see Create endpoint.

  2. Edit the value in the name field (endpoint name) and the endpoint parameters under settings.

  3. Make sure the settings are correct.

    1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.

    2. Run the command:

      terraform validate

      If there are errors in the configuration files, Terraform will point to them.

  4. Confirm updating the resources.

    1. Run the command to view planned changes:

      terraform plan

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

    2. If you are happy with the planned changes, apply them:

      1. Run the command:

        terraform apply

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

Note

You can edit endpoints using the API only for ClickHouse®, Apache Kafka®, MongoDB, MySQL®, and PostgreSQL sources and targets.

Use the update API method and include the following in the request:

  • Endpoint ID in the endpointId parameter.
  • Endpoint name in the name parameter.
  • Endpoint description in the description parameter.
  • Endpoint parameters in the settings parameter.

You can get the endpoint ID with a list of endpoints in the folder.

Editing the endpoint settings of a transfer with the Replication type and the Replicating status will result in restarting the transfer.

Cloning an endpoint

Note

When cloning, you cannot change the endpoint type and the database type.

  1. Go to the folder page and select Yandex Data Transfer.

  2. In the left-hand panel, select Endpoints.

  3. Select the endpoint to clone.

  4. Click Clone in the top panel.

  5. Specify a new endpoint name and edit other parameters, if required:

  6. Click Create endpoint.

Deleting an endpoint

Warning

Before deleting an endpoint, delete all the transfers it is included in.

To delete an endpoint:

  1. Go to the folder page and select Yandex Data Transfer.
  2. In the left-hand panel, select Endpoints.
  3. Select the endpoint to delete.
  4. Click Delete in the top panel.
  5. In the window that opens, click Delete.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To delete an endpoint, run the command:

yc datatransfer endpoint delete <endpoint_ID>

You can get the endpoint ID with a list of endpoints in the folder.

To delete an endpoint created using Terraform:

  1. Open the current Terraform configuration file with an infrastructure plan.

    For more information about creating this file, see Creating an endpoint.

  2. Make sure that the endpoint being deleted is not bound to a transfer.

  3. Delete the endpoint description.

  4. Make sure the settings are correct.

    1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.

    2. Run the command:

      terraform validate

      If there are errors in the configuration files, Terraform will point to them.

  5. Type the word yes, then press Enter.

    1. Run the command to view planned changes:

      terraform plan

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

    2. If you are happy with the planned changes, apply them:

      1. Run the command:

        terraform apply

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

Use the delete API method and provide the ID of the required endpoint in the endpointID request parameter.

You can get the endpoint ID with a list of endpoints in the folder.

Greenplum® and Greenplum Database® are registered trademarks or trademarks of VMware, Inc. in the United States and/or other countries.

ClickHouse® is a registered trademark of ClickHouse, Inc.

Previous
Preparing for a transfer
Next
Migrating endpoints to a different availability zone