Configuring Yandex Cloud DNS to access a Yandex Managed Service for ClickHouse® cluster from other cloud networks

In this tutorial, we will use a Managed Service for ClickHouse® cluster as an example. You can use the same method to configure availability for other managed database services.

Managed Service for ClickHouse® cluster resource records reside in DNS service zones that are confined to one cloud network. This prevents clients, such as virtual machines residing in a different cloud network, from connecting to cluster hosts via FQDN, despite established network connectivity.

To enable clients from different cloud networks to connect to the cluster via FQDN, configure a shared DNS zone in Yandex Cloud DNS:

1. Create a zone in Yandex Cloud DNS.
2. Check if the cluster is available from a different cloud network.

If you no longer need the resources you created, delete them.

Required paid resourcesRequired paid resources

The support cost for this solution includes:

• Managed Service for ClickHouse® cluster fee: Covers the use of computing resources allocated to hosts (including ZooKeeper hosts) and disk space (see Managed Service for ClickHouse® pricing).
• VM fee: Covers the use of computational resources, storage, and, optionally, a public IP address (see Compute Cloud pricing).
• DNS zone and queries fee (see Cloud DNS pricing).

Getting startedGetting started

1. Prepare an SSH key pair for connecting to virtual machines.

2. Set up the infrastructure:

   Manually

   Using Terraform

   1. Create two cloud networks named mch-net and another-net.
   2. Create a subnet in each network.
   3. In mch-net, create a Managed Service for ClickHouse® cluster with no public access for its hosts, using any suitable configuration.
   4. Optionally, in mch-net, create a Linux-based VM named mch-net-vm. In the process, you will need to specify the public SSH key you created earlier.
   5. In another-net, create a Linux-based VM named another-net-vm. When creating it, specify the public SSH key you prepared earlier.
   6. Configure security group rules for your cluster and VMs according to this tutorial.

   1. If you do not have Terraform yet, install it.

   2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

   3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

   4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

   5. Download the nets-vm-mch.tf configuration file to your current working directory.

      This file describes:

      • Networks.
      • Subnets.
      • Security groups required for the Managed Service for ClickHouse® cluster and VMs.
      • Virtual machines.
      • Managed Service for ClickHouse® cluster.
      • Internal DNS zone.

   6. In the nets-vm-mch.tf file, specify the following:

      • ch_dbname: Managed Service for ClickHouse® cluster database name.
      • ch_user: Managed Service for ClickHouse® cluster admin username.
      • ch_password: Managed Service for ClickHouse® cluster admin password.
      • image_id: VM public image ID. For details on getting a list of available images, see this guide.
      • vm_username: VM user name.
      • vm_ssh_key_path: Absolute path to your previously created VM public key.
      • create_optional_vm: Parameter that enables VM creation in the cluster’s network. Optionally, set it to 1 to test cluster accessibility from the same network later.

   7. Run the terraform init command in the directory with your configuration files. This command initializes the provider specified in the configuration file, making its resources and data sources available for use.

   8. Validate your Terraform configuration files using this command:

      terraform validate
      

      Terraform will display any configuration errors detected in your files.

   9. Create the required infrastructure:

      1. Run this command to view the planned changes:

         terraform plan
         

         If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

      2. If everything looks correct, apply the changes:

         1. Run this command:

            terraform apply
            

         2. Confirm updating the resources.

         3. Wait for the operation to complete.

      All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

3. Optionally, connect to the mch-net-vm VM over SSH and configure cluster connection using clickhouse-client to make sure security groups are configured correctly and you can access the cluster via FQDN from within the cloud network.

4. Configure network connectivity between the mch-net and another-net cloud networks, e.g., via an IPSec gateway. For alternative methods of configuring network connectivity, see Tutorials on the use of the network infrastructure in Yandex Cloud network infrastructure tutorials.

Create a zone in Cloud DNSCreate a zone in Cloud DNS

1. Create a DNS zone:

   Manually

   Using Terraform

   Create a private DNS zone for mdb.yandexcloud.net. by following this tutorial, specifying mch-net and another-net in the network list.

   1. In the nets-vm-mch.tf file, set create_zone to 1.

   2. Validate your Terraform configuration files using this command:

      terraform validate
      

      Terraform will display any configuration errors detected in your files.

   3. Create the required infrastructure:

      1. Run this command to view the planned changes:

         terraform plan
         

         If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

      2. If everything looks correct, apply the changes:

         1. Run this command:

            terraform apply
            

         2. Confirm updating the resources.

         3. Wait for the operation to complete.

2. Verify that the cluster record has been automatically created in the DNS zone.

   1. In the management console, select the folder containing your DNS zone.
   2. Navigate to the Cloud DNS service.
   3. Select the zone from the list.
   4. Make sure the list contains a record in the following format: c-<cluster_ID>.rw.mdb.yandexcloud.net..

Check whether the cluster is available from a different cloud networkCheck whether the cluster is available from a different cloud network

1. Use SSH to connect to another-net-vm.
2. Configure cluster connection using clickhouse-client and check whether you can access the cluster via FQDN from a different cloud network.

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:

ClickHouse® is a registered trademark of ClickHouse, Inc.