Configuring Yandex Cloud DNS to access a Managed Service for ClickHouse® cluster from other cloud networks

In this tutorial, we will use a Managed Service for ClickHouse® cluster as an example. You can configure availability for other managed database services the same way.

Resource records for Managed Service for ClickHouse® clusters are created in DNS service zones operating within a single cloud network. This prevents clients, such as virtual machines residing in a different cloud network, from connecting to cluster hosts using their FQDNs, even with configured network connectivity between the cloud networks.

To enable clients from different cloud networks to connect to the cluster using its FQDN, configure a shared DNS zone in Yandex Cloud DNS:

1. Create a zone in Yandex Cloud DNS.
2. Check if the cluster is available from a different cloud network.

If you no longer need the resources you created, delete them.

Required paid resourcesRequired paid resources

The support cost includes:

• Managed Service for ClickHouse® cluster fee: using computing resources allocated to hosts (including ZooKeeper hosts) and disk space (see Managed Service for ClickHouse® pricing).
• VM fee: using computing resources, storage, and, optionally, public IP address (see Compute Cloud pricing).
• Fee for a DNS zone and DNS requests (see Cloud DNS pricing).

Getting startedGetting started

1. Prepare an SSH key pair to connect to VMs.

2. Set up the infrastructure:

   Manually

   Using Terraform

   1. Create two cloud networks named mch-net and another-net.
   2. Create a subnet in each network.
   3. In mch-net, create a Managed Service for ClickHouse® cluster of any suitable configuration with hosts that have no public access.
   4. Optionally, in mch-net, create a Linux-based VM named mch-net-vm. When creating it, specify the public SSH key prepared earlier.
   5. In another-net, create a Linux-based VM named another-net-vm. When creating it, specify the public SSH key prepared earlier.
   6. Configure VM and cluster security group rules by following this guide.

   1. If you do not have Terraform yet, install it.

   2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

   3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

   4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

   5. Download the nets-vm-mch.tf configuration file to the same working directory.

      This file describes:

      • Networks.
      • Subnets.
      • Security groups required for the Managed Service for ClickHouse® cluster and VMs.
      • Virtual machines.
      • Managed Service for ClickHouse® cluster.
      • Internal DNS zone.

   6. Specify the following in the nets-vm-mch.tf file:

      • ch_dbname: Name of the database in the Managed Service for ClickHouse® cluster.
      • ch_user: Managed Service for ClickHouse® cluster admin username.
      • ch_password: Managed Service for ClickHouse® cluster admin user password.
      • image_id: ID of the VM public image. For more information about how to get a list of available images, see this guide.
      • vm_username: VM user name.
      • vm_ssh_key_path: Absolute path to the VM public key you prepared earlier.
      • create_optional_vm: Parameter for creating a VM in the same network as the cluster. Optionally, set it to 1 so you can later check if the cluster is available from the same network.

   7. Run the terraform init command in the directory with the configuration files. This command initializes the provider specified in the configuration file and enables you to use the provider resources and data sources.

   8. Make sure the Terraform configuration files are correct using this command:

      terraform validate
      

      If there are any errors in the configuration files, Terraform will point them out.

   9. Create the required infrastructure:

      1. Run this command to view the planned changes:

         terraform plan
         

         If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

      2. If everything looks correct, apply the changes:

         1. Run this command:

            terraform apply
            

         2. Confirm updating the resources.

         3. Wait for the operation to complete.

      All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

3. Optionally, use SSH to connect to the mch-net-vm VM and configure cluster connection via clickhouse-client to make sure security groups are configured correctly and you can connect to the cluster using its FQDN from the same cloud network.

4. Configure network connectivity between the mch-net and another-net cloud networks, e.g., using an IPSec gateway. For other ways to configure network connectivity, see Tutorials for working with network infrastructure in Yandex Cloud.

Create a zone in Cloud DNSCreate a zone in Cloud DNS

1. Create a DNS zone:

   Manually

   Using Terraform

   Create a private DNS zone with the mdb.yandexcloud.net. address as per this guide. In the network list, specify mch-net and another-net.

   1. In the nets-vm-mch.tf file, set the create_zone parameter to 1.

   2. Make sure the Terraform configuration files are correct using this command:

      terraform validate
      

      If there are any errors in the configuration files, Terraform will point them out.

   3. Create the required infrastructure:

      1. Run this command to view the planned changes:

         terraform plan
         

         If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

      2. If everything looks correct, apply the changes:

         1. Run this command:

            terraform apply
            

         2. Confirm updating the resources.

         3. Wait for the operation to complete.

2. Make sure the cluster record appears automatically in the DNS zone.

   1. In the management console, select the folder containing the DNS zone.
   2. Select Cloud DNS.
   3. Select the zone from the list.
   4. Check that the record list contains a record in the following format: c-<cluster_ID>.rw.mdb.yandexcloud.net..

Check if the cluster is available from a different cloud networkCheck if the cluster is available from a different cloud network

1. Use SSH to connect to the another-net-vm VM.
2. Configure cluster connection via clickhouse-client and make sure you can connect to the cluster from a different cloud network using the cluster's FQDN.

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need:

ClickHouse® is a registered trademark of ClickHouse, Inc.