Creating a ClickHouse® cluster

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

To create a Managed Service for ClickHouse® cluster:

1. Check whether the folder has any subnets for the cluster hosts:

   yc vpc subnet list
   

   If there are no subnets in the folder, create the required subnets in VPC.

2. View the description of the CLI command to create a cluster:

   yc managed-clickhouse cluster create --help
   

3. Specify cluster parameters in the create command (the list of supported parameters in the example is not exhaustive):

   yc managed-clickhouse cluster create \
     --name <cluster_name> \
     --environment <environment> \
     --network-name <network_name> \
     --host type=<host_type>,`
          `zone-id=<availability_zone>,`
          `subnet-id=<subnet_ID>,`
          `assign-public-ip=<public_access_to_host> \
     --clickhouse-resource-preset <host_class> \
     --clickhouse-disk-type <network-hdd|network-ssd|network-ssd-nonreplicated|local-ssd> \
     --clickhouse-disk-size <storage_size_in_GB> \
     --user name=<username>,password=<user_password> \
     --database name=<DB_name> \
     --security-group-ids <list_of_security_group_IDs> \
     --websql-access=<true_or_false> \
     --deletion-protection
   

   You need to specify the subnet-id if the selected availability zone has two or more subnets.

   Where:

   • --environment: Cluster environment, prestable or production.

   • --host: Host parameters:

     • type: Host type: clickhouse or zookeeper.
     • zone-id: Availability zone.
     • assign-public-ip: Internet access to the host via a public IP address, true or false.

   • --clickhouse-disk-type: Disk type.

     Warning

     You cannot change disk type after you create a cluster.

   • --user: Contains the ClickHouse® user name and password.

     The username may contain Latin letters, numbers, hyphens, and underscores but must begin with a letter or underscore. The name can be up to 32 characters long.

     The password must be from 8 to 128 characters long.

   • --websql-access: Enables SQL queries against cluster databases from the Yandex Cloud management console using Yandex WebSQL. The default value is false.

   • --deletion-protection: Cluster protection from accidental deletion, true or false.

     Even with cluster deletion protection enabled, one can still delete a user or database or connect manually and delete the database contents.

   You can manager cluster users and databases via SQL.

   Warning

   You can't disable activated settings to manage users and databases via SQL. You can enable these as required later when editing cluster settings.

   1. To enable SQL user management:

      • set --enable-sql-user-management to true.
      • Set a password for the admin user in the --admin-password parameter.

      yc managed-clickhouse cluster create \
        ...
        --enable-sql-user-management true \
        --admin-password "<admin_user_password>"
      

   2. To enable SQL database management:

      • Set --enable-sql-user-management and --enable-sql-database-management to true;
      • Set a password for the admin user in the --admin-password parameter.

      yc managed-clickhouse cluster create \
        ...
        --enable-sql-user-management true \
        --enable-sql-database-management true \
        --admin-password "<admin_user_password>"
      

   3. To allow access to the cluster from Yandex Cloud Functions, provide the --serverless-access parameter. For more information about setting up access, see the Cloud Functions documentation.

   4. To allow access to the cluster from Yandex Query, provide the --yandexquery-access=true parameter. This feature is at the Preview stage.

   5. To enable ClickHouse® Keeper in a cluster, set the --embedded-keeper parameter to true.

      yc managed-clickhouse cluster create \
        ...
        --embedded-keeper true
      

      Alert

      You can't disable ClickHouse® Keeper after you create a cluster. ZooKeeper hosts will also become unavailable.

   6. To configure hybrid storage settings:

      • Set the --cloud-storage parameter to true to enable hybrid storage.

        Note

        Once enabled, hybrid storage cannot be disabled.

      • Provide the hybrid storage settings in the relevant parameters:

        • --cloud-storage-data-cache: Allows you to cache files in cluster storage. This setting is enabled by default (set to true).
        • --cloud-storage-data-cache-max-size: Sets the maximum cache size (in bytes) allocated in cluster storage for files. The default value is 1073741824 (1 GB).
        • --cloud-storage-move-factor: Sets the minimum share of free space in cluster storage. If the actual value is less than this setting value, the data is moved to Yandex Object Storage. The minimum value is 0, the maximum one is 1, and the default one is 0.01.
        • --cloud-storage-prefer-not-to-merge: Disables data part merges in cluster and object storage. To disable merges, set the parameter to true or provide it with no value. To keep merges enabled, set the parameter to false or do not provide it in the CLI command when creating a cluster.

      yc managed-clickhouse cluster create \
         ...
         --cloud-storage=true \
         --cloud-storage-data-cache=<file_storage> \
         --cloud-storage-data-cache-max-size=<memory_size_in_bytes> \
         --cloud-storage-move-factor=<share_of_free_space> \
         --cloud-storage-prefer-not-to-merge=<merging_data_parts>
        ...
      

      Where:

      • --cloud-storage-data-cache: Store files in cluster storage, true or false.
      • --cloud-storage-prefer-not-to-merge: Disables merging of data parts in cluster and object storage, true or false.

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

Terraform is distributed under the Business Source License. The Yandex Cloud provider for Terraform is distributed under the MPL-2.0 license.

For more information about the provider resources, see the documentation on the Terraform website or mirror website.

To create a Managed Service for ClickHouse® cluster:

1. Using the command line, navigate to the folder that will contain the Terraform configuration files with an infrastructure plan. Create the directory if it does not exist.

2. If you do not have Terraform yet, install it and configure its Yandex Cloud provider.

3. Create a configuration file describing the cloud network and subnets.

   • Network: Description of the cloud network where the cluster will be hosted. If you already have a suitable network, you do not need to describe it again.
   • Subnets: Subnets to connect the cluster hosts to. If you already have suitable subnets, you do not need to describe them again.

   Example structure of a configuration file describing a single-subnet cloud network:

   resource "yandex_vpc_network" "<network_name_in_Terraform>" { name = "<network_name>" }
   
   resource "yandex_vpc_subnet" "<subnet_name_in_Terraform>" {
     name           = "<subnet_name>"
     zone           = "<availability_zone>"
     network_id     = yandex_vpc_network.<network_name_in_Terraform>.id
     v4_cidr_blocks = ["<subnet>"]
   }
   

4. Create a configuration file with a description of the cluster and its hosts.

   • Database cluster: Description of the cluster and its hosts. Also as required here:

     • Specify DBMS settings. You can specify them later.

       Using the Yandex Cloud interfaces, you can manage a limited number of settings. Using SQL queries, you can apply ClickHouse® settings at the query level.

     • Enable cluster protection against accidental deletion.

       Even with cluster deletion protection enabled, one can still delete a user or database or connect manually and delete the database contents.

   Example structure of a configuration file that describes a cluster with a single host:

   resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
     name                = "<cluster_name>"
     environment         = "<environment>"
     network_id          = yandex_vpc_network.<network_name_in_Terraform>.id
     security_group_ids  = ["<list_of_security_group_IDs>"]
     deletion_protection = <cluster_deletion_protection>
   
     clickhouse {
       resources {
         resource_preset_id = "<host_class>"
         disk_type_id       = "<disk_type>"
         disk_size          = <storage_size_in_GB>
       }
     }
   
     database {
       name = "<DB_name>"
     }
   
     user {
       name     = "<DB_user_name>"
       password = "<password>"
       permission {
         database_name = "<name_of_DB_to_create_user_in>"
       }
     }
   
     host {
       type             = "CLICKHOUSE"
       zone             = "<availability_zone>"
       subnet_id        = yandex_vpc_subnet.<subnet_name_in_Terraform>.id
       assign_public_ip = <public_access_to_host>
     }
   }
   

   Where:

   • deletion_protection: Cluster protection from accidental deletion, true or false.

   • user: User data section. Contains the ClickHouse® user name and password as well as a list of DBs the user must have access to in the permission section.

     The username may contain Latin letters, numbers, hyphens, and underscores but must begin with a letter or underscore. The name can be up to 32 characters long.

     The password must be from 8 to 128 characters long.

   • assign_public_ip: Public access to the host, true or false.

   1. To set up the maintenance window (for disabled clusters as well), add the maintenance_window block to the cluster description:

      resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
        ...
        maintenance_window {
          type = <maintenance_type>
          day  = <day_of_week>
          hour = <hour>
        }
        ...
      }
      

      Where:

      • type: Maintenance type. The possible values include:
        • anytime: Anytime.
        • weekly: By schedule.
      • day: Day of the week for the weekly type in DDD format, e.g., MON.
      • hour: Hour of the day for the weekly type in the HH format, e.g., 21.

   2. To enable access from other services and allow running SQL queries from the management console using Yandex WebSQL, add a section named access with the settings you need:

      resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
        ...
        access {
          data_lens    = <access_from_DataLens>
          metrika      = <access_from_Metrica_and_AppMetrica>
          serverless   = <access_from_Cloud_Functions>
          yandex_query = <access_from_Yandex_Query>
          web_sql      = <run_SQL_queries_from_management_console>
        }
        ...
      }
      

      Where:

      • data_lens: Access from DataLens, true or false.

      • metrika: Access from Yandex Metrica and AppMetrica, true or false.

      • serverless: Access from Cloud Functions, true or false.

      • yandex_query: Access from Yandex Query, true or false.

      • web_sql: Execution of SQL queries from the management console, true or false.

   3. You can manager cluster users and databases via SQL.

      Warning

      You can't disable activated settings to manage users and databases via SQL. You can enable these as required later when editing cluster settings.

      • To enable user management via SQL, expand the cluster description to include a sql_user_management field set to true and an admin_password field containing the password for the admin account:

        resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
          name                = "<cluster_name>"
          ...
          admin_password      = "<admin_password>"
          sql_user_management = true
          ...
        }
        

      • To enable database management via SQL, expand the cluster description to include a sql_user_management field and a sql_database_management field, both set to true, as well as the admin_password field containing the password for the admin account:

        resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
          name                    = "<cluster_name>"
          ...
          admin_password          = "<admin_password>"
          sql_database_management = true
          sql_user_management     = true
          ...
        }
        

   For more information about the resources you can create with Terraform, see the provider documentation.

5. Check that the Terraform configuration files are correct:

   1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.

   2. Run this command:

      terraform validate
      

      Terraform will show any errors found in your configuration files.

6. Create a cluster:

   1. Run this command to view the planned changes:

      terraform plan
      

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

   2. If everything looks correct, apply the changes:

      1. Run this command:

         terraform apply
         

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

   All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Use the Cluster.Create method and send the following request, e.g., via cURL:

   1. Create a file named body.json and add the following contents to it:

      Note

      This example does not use all available parameters.

      {
        "folderId": "<folder_ID>",
        "name": "<cluster_name>",
        "environment": "<environment>",
        "networkId": "<network_ID>",
        "securityGroupIds": [
          "<security_group_1_ID>",
          "<security_group_2_ID>",
          ...
          "<security_group_N_ID>"
        ],
        "configSpec": {
          "version": "<ClickHouse®>_version",
          "embeddedKeeper": <using_ClickHouse®_Keeper>,
          "clickhouse": {
            "resources": {
              "resourcePresetId": "<ClickHouse®>_host_class",
              "diskSize": "<storage_size_in_bytes>",
              "diskTypeId": "<disk_type>"
            }
          },
          "zookeeper": {
            "resources": {
              "resourcePresetId": "<ZooKeeper>_host_class",
              "diskSize": "<storage_size_in_bytes>",
              "diskTypeId": "<disk_type>"
            }
          },
          "access": {
            "dataLens": <access_from_DataLens>,
            "webSql": <run_SQL_queries_from_management_console>,
            "metrika": <access_from_Metrica_and_AppMetrica>,
            "serverless": <access_from_Cloud_Functions>,
            "dataTransfer": <access_from_Data_Transfer>,
            "yandexQuery": <access_from_Yandex_Query>
          },
          "cloudStorage": {
            "enabled": <hybrid_storage_use>,
            "moveFactor": "<share_of_free_space>",
            "dataCacheEnabled": <temporary_file_storage>,
            "dataCacheMaxSize": "<maximum_cache_size_for_file_storage>",
            "preferNotToMerge": <disabling_merge_of_data_parts>
          },
          "adminPassword": "<admin_user_password>",
          "sqlUserManagement": <user_management_via_SQL>,
          "sqlDatabaseManagement": <database_management_via_SQL>
        },
        "databaseSpecs": [
          {
            "name": "<DB_name>"
          },
          { <similar_settings_for_database_2> },
          { ... },
          { <similar_settings_for_database_N> }
        ],
        "userSpecs": [
          {
            "name": "<username>",
            "password": "<user_password>",
            "permissions": [
              {
                "databaseName": "<DB_name>"
              }
            ]
          },
          { <similar_configuration_for_user_2> },
          { ... },
          { <similar_configuration_for_user_N> }
        ],
        "hostSpecs": [
          {
            "zoneId": "<availability_zone>",
            "type": "<host_type>",
            "subnetId": "<subnet_ID>",
            "assignPublicIp": <public_access_to_host>,
            "shardName": "<shard_name>"
          },
          { <similar_configuration_for_host_2> },
          { ... },
          { <similar_configuration_for_host_N> }
        ],
        "deletionProtection": <cluster_deletion_protection>
      }
      

      Where:

      • name: Cluster name.

      • environment: Cluster environment, PRODUCTION or PRESTABLE.

      • networkId: ID of the network the cluster will be in.

      • securityGroupIds: Security group IDs as an array of strings. Each string is a security group ID.

      • configSpec: Cluster configuration:

        • version: ClickHouse® version, 24.3, 24.8, 24.9, or 24.10.

        • embeddedKeeper: Using ClickHouse® Keeper instead of ZooKeeper, true or false.

          This setting determines how replication will be managed in a cluster of multiple ClickHouse® hosts:

          • If true, the replication will be managed using ClickHouse® Keeper.

            Alert

            You can't disable ClickHouse® Keeper after you create a cluster. ZooKeeper hosts will also become unavailable.

          • If undefined or false, the replication and query distribution will be managed using ZooKeeper.

        • clickhouse: ClickHouse® configuration:

          • resources.resourcePresetId: Host class ID. You can request the list of available host classes with their IDs using the ResourcePreset.list method.
          • resources.diskSize: Disk size in bytes.
          • resources.diskTypeId: Disk type.

        • zookeeper: ZooKeeper configuration.

          • resources.resourcePresetId: Host class ID. You can request the list of available host classes with their IDs using the ResourcePreset.list method.
          • resources.diskSize: Disk size in bytes.
          • resources.diskTypeId: Disk type.

          If you enabled ClickHouse® Keeper with the help of the embeddedKeeper: true setting, you do not have to specify a ZooKeeper configuration in configSpec as it will not be applied.

        • access: Settings enabling access to the cluster from other services and SQL queries from the management console using Yandex WebSQL:

          • dataLens: Enable access from DataLens, true or false. The default value is false. For more information about setting up a connection, see Connecting from DataLens.

          • webSql: Enables SQL queries against cluster databases from the Yandex Cloud management console using Yandex WebSQL, true or false. The default value is false.

          • metrika: Enables data import from AppMetrica to your cluster: true or false. The default value is false.

          • serverless: Enable access to the cluster from Yandex Cloud Functions, true or false. The default value is false. For more information about setting up access, see the Cloud Functions documentation.

          • dataTransfer: Enable access to the cluster from Yandex Data Transfer in Serverless mode, true or false. The default value is false.

            This will enable you to connect to Yandex Data Transfer running in Kubernetes via a special network. As a result, other operations, e.g., transfer launch and deactivation, will run faster.

          • yandexQuery: Enable access to the cluster from Yandex Query, true or false. This feature is at the Preview stage.Default value: false.

        • cloudStorage: Hybrid storage settings:

          • enabled: Enable hybrid storage in the cluster if it is disabled, true or false. Default value: false (disabled).

            Note

            Once enabled, hybrid storage cannot be disabled.

          • moveFactor: Minimum share of free space in cluster storage. If the minimum share is below this value, the data will be moved to Yandex Object Storage.

            Minimum value: 0; maximum value: 1; default: 0.01.

          • dataCacheEnabled: Allow temporary storage of files in cluster storage, true or false.

            Default value: true (enabled).

          • dataCacheMaxSize: Maximum cache size (in bytes) allocated in cluster storage for temporary file storage.

            Default value: 1073741824 (1 GB).

          • preferNotToMerge: Disable merging of data parts in cluster and object storage, true or false.

            To disable merging, set to true. To leave merging enabled, set to false.

        • sql... and adminPassword: Group of settings for user and database management via SQL:

          • adminPassword: admin user password.
          • sqlUserManagement: User management via SQL, true or false.
          • sqlDatabaseManagement: Database management via SQL, true or false. For that, you also need to enable user management through SQL.

          Warning

          You can't disable activated settings to manage users and databases via SQL. You can enable these as required later when editing cluster settings.

      • databaseSpecs: Database settings as an array of name element parameters. Each parameter contains a name of a separate database.

      • userSpecs: User settings as an array of elements, one for each user. Each element has the following structure:

        • name: Username. It may contain Latin letters, numbers, hyphens, and underscores, and must start with a letter or underscore. The name can be up to 32 characters long.

        • password: User password. The password must be from 8 to 128 characters long.

        • permissions: List of DBs the user must have access to.

          The list appears as an array of databaseName parameters. Each parameter contains the name of a separate database.

      • hostSpecs: Cluster host settings as an array of elements, one for each host. Each element has the following structure:

        • type: Host type: CLICKHOUSE or ZOOKEEPER.

          If you enabled ClickHouse® Keeper with the help of the embeddedKeeper: true setting, specify only ClickHouse® host settings in hostSpecs.

        • zoneId: Availability zone.

        • subnetId: Subnet ID.

        • shardName: Shard name. The setting is only relevant for CLICKHOUSE-type hosts.

        • assignPublicIp: Internet access to the host via a public IP address, true or false.

        If you are creating a multi-host cluster without using ClickHouse® Keeper, the following rules apply to ZooKeeper hosts:

        • If the cluster cloud network has subnets in each availability zone, and ZooKeeper host settings are not specified, then one such host will automatically be added to each subnet.

        • If only some availability zones in the cluster's network have subnets, specify the ZooKeeper host settings explicitly.

      • deletionProtection: Cluster protection from accidental deletion, true or false. The default value is false.

        Even with cluster deletion protection enabled, one can still delete a user or database or connect manually and delete the database contents.

      You can request the folder ID with the list of folders in the cloud.

   2. Run this request:

      curl \
        --request POST \
        --header "Authorization: Bearer $IAM_TOKEN" \
        --header "Content-Type: application/json" \
        --url 'https://mdb.api.cloud.yandex.net/managed-clickhouse/v1/clusters' \
        --data '@body.json'
      

3. View the server response to make sure the request was successful.

1. Get an IAM token for API authentication and put it into the environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService.Create call and send the following request, e.g., via gRPCurl:

   1. Create a file named body.json and add the following contents to it:

      Note

      This example does not use all available parameters.

      {
        "folder_id": "<folder_ID>",
        "name": "<cluster_name>",
        "environment": "<environment>",
        "network_id": "<network_ID>",
        "security_group_ids": [
          "<security_group_1_ID>",
          "<security_group_2_ID>",
          ...
          "<security_group_N_ID>"
        ],
        "config_spec": {
          "version": "<ClickHouse®_version>",
          "embedded_keeper": <ClickHouse® Keeper_usage>,
          "clickhouse": {
            "resources": {
              "resource_preset_id": "<ClickHouse®>_host_class",
              "disk_size": "<storage_size_in_bytes>",
              "disk_type_id": "<disk_type>"
            }
          },
          "zookeeper": {
            "resources": {
              "resource_preset_id": "<ZooKeeper_host_class>",
              "disk_size": "<storage_size_in_bytes>",
              "disk_type_id": "<disk_type>"
            }
          },
          "access": {
            "data_lens": <access_from_DataLens>,
            "web_sql": <run_SQL_queries_from_management_console>,
            "metrika": <access_from_Metrica_and_AppMetrica>,
            "serverless": <access_from_Cloud_Functions>,
            "data_transfer": <access_from_Data_Transfer>,
            "yandex_query": <access_from_Yandex_Query>
          },
          "cloud_storage": {
            "enabled": <hybrid_storage_use>,
            "move_factor": "<share_of_free_space>",
            "data_cache_enabled": <temporary_file_storage>,
            "data_cache_max_size": "<maximum_cache_size_for_file_storage>",
            "prefer_not_to_merge": <disabling_merge_of_data_parts>
          },
          "admin_password": "<admin_user_password>",
          "sql_user_management": <user_management_via_SQL>,
          "sql_database_management": <database_management_via_SQL>
        },
        "database_specs": [
          {
            "name": "<DB_name>"
          },
          { <similar_settings_for_database_2> },
          { ... },
          { <similar_settings_for_database_N> }
        ],
        "user_specs": [
          {
            "name": "<username>",
            "password": "<user_password>",
            "permissions": [
              {
                "database_name": "<DB_name>"
              }
            ]
          },
          { <similar_configuration_for_user_2> },
          { ... },
          { <similar_configuration_for_user_N> }
        ],
        "host_specs": [
          {
            "zone_id": "<availability_zone>",
            "type": "<host_type>",
            "subnet_id": "<subnet_ID>",
            "assign_public_ip": <public_access_to_host>,
            "shard_name": "<shard_name>"
          },
          { <similar_configuration_for_host_2> },
          { ... },
          { <similar_configuration_for_host_N> }
        ],
        "deletion_protection": <cluster_deletion_protection>
      }
      

      Where:

      • name: Cluster name.

      • environment: Cluster environment, PRODUCTION or PRESTABLE.

      • network_id: ID of the network the cluster will be in.

      • security_group_ids: Security group IDs as an array of strings. Each string is a security group ID.

      • config_spec: Cluster configuration:

        • version: ClickHouse® version, 24.3, 24.8, 24.9, or 24.10.

        • embedded_keeper: Using ClickHouse® Keeper instead of ZooKeeper, true or false.

          This setting determines how replication will be managed in a cluster of multiple ClickHouse® hosts:

          • If true, the replication will be managed using ClickHouse® Keeper.

            Alert

            You can't disable ClickHouse® Keeper after you create a cluster. ZooKeeper hosts will also become unavailable.

          • If undefined or false, the replication and query distribution will be managed using ZooKeeper.

        • clickhouse: ClickHouse® configuration:

          • resources.resource_preset_id: Host class ID. You can request the list of available host classes with their IDs using the ResourcePreset.list method.
          • resources.disk_size: Disk size in bytes.
          • resources.disk_type_id: Disk type.

        • zookeeper: ZooKeeper configuration.

          • resources.resource_preset_id: Host class ID. You can request the list of available host classes with their IDs using the ResourcePreset.list method.
          • resources.disk_size: Disk size in bytes.
          • resources.disk_type_id: Disk type.

          If you enabled ClickHouse® Keeper with the help of the embedded_keeper: true setting, you do not have to specify a ZooKeeper configuration in config_spec as it will not be applied.

        • access: Settings enabling access to the cluster from other services and SQL queries from the management console using Yandex WebSQL:

          • data_lens: Enable access from DataLens, true or false. The default value is false. For more information about setting up a connection, see Connecting from DataLens.

          • web_sql: Enables SQL queries against cluster databases from the Yandex Cloud management console using Yandex WebSQL: true or false. The default value is false.

          • metrika: Enables data import from AppMetrica to your cluster: true or false. The default value is false.

          • serverless: Enable access to the cluster from Yandex Cloud Functions, true or false. The default value is false. For more information about setting up access, see the Cloud Functions documentation.

          • data_transfer: Enable access to the cluster from Yandex Data Transfer in Serverless mode, true or false. The default value is false.

            This will enable you to connect to Yandex Data Transfer running in Kubernetes via a special network. As a result, other operations, e.g., transfer launch and deactivation, will run faster.

          • yandex_query: Enable access to the cluster from Yandex Query, true or false: true or false. This feature is at the Preview stage.Default value: false.

        • cloud_storage: Hybrid storage settings:

          • enabled: Enable hybrid storage in the cluster if it is disabled, true or false. Default value: false (disabled).

            Note

            Once enabled, hybrid storage cannot be disabled.

          • move_factor: Minimum share of free space in cluster storage. If the minimum share is below this value, the data will be moved to Yandex Object Storage.

            Minimum value: 0; maximum value: 1; default: 0.01.

          • data_cache_enabled: Allow temporary storage of files in cluster storage, true or false.

            Default value: true (enabled).

          • data_cache_max_size: Maximum cache size (in bytes) allocated in cluster storage for temporary file storage.

            Default value: 1073741824 (1 GB).

          • prefer_not_to_merge: Disable merging of data parts in cluster and object storage, true or false.

            To disable merging, set to true. To leave merging enabled, set to false.

        • sql... and admin_password: Group of settings for user and database management via SQL:

          • admin_password: admin user password.
          • sql_user_management: User management via SQL, true or false.
          • sql_database_management: Database management via SQL, true or false. For that, you also need to enable user management through SQL.

          Warning

          You can't disable activated settings to manage users and databases via SQL. You can enable these as required later when editing cluster settings.

      • database_specs: Database settings as an array of name element parameters. Each parameter contains the name of a separate database.

      • user_specs: User settings as an array of elements, one for each user. Each element has the following structure:

        • name: Username. It may contain Latin letters, numbers, hyphens, and underscores, and must start with a letter or underscore. The name can be up to 32 characters long.

        • password: User password. The password must be from 8 to 128 characters long.

        • permissions: List of DBs the user must have access to.

          The list appears as an array of database_name parameters. Each parameter contains the name of a separate database.

      • host_specs: Cluster host settings as an array of elements, one for each host. Each element has the following structure:

        • type: Host type: CLICKHOUSE or ZOOKEEPER.

          If you enabled ClickHouse® Keeper with the help of the embedded_keeper: true setting, specify only ClickHouse® host settings in host_specs.

        • zone_id: Availability zone.

        • subnet_id: Subnet ID.

        • shard_name: Shard name. The setting is only relevant for CLICKHOUSE-type hosts.

        • assign_public_ip: Internet access to the host via a public IP address, true or false.

        If you are creating a multi-host cluster without using ClickHouse® Keeper, the following rules apply to ZooKeeper hosts:

        • If the cluster cloud network has subnets in each availability zone, and ZooKeeper host settings are not specified, then one such host will automatically be added to each subnet.

        • If only some availability zones in the cluster's network have subnets, specify the ZooKeeper host settings explicitly.

      • deletion_protection: Cluster protection from accidental deletion, true or false. The default value is false.

        Even with cluster deletion protection enabled, one can still delete a user or database or connect manually and delete the database contents.

      You can request the folder ID with the list of folders in the cloud.

   2. Run this request:

      grpcurl \
        -format json \
        -import-path ~/cloudapi/ \
        -import-path ~/cloudapi/third_party/googleapis/ \
        -proto ~/cloudapi/yandex/cloud/mdb/clickhouse/v1/cluster_service.proto \
        -rpc-header "Authorization: Bearer $IAM_TOKEN" \
        -d @ \
        mdb.api.cloud.yandex.net:443 \
        yandex.cloud.mdb.clickhouse.v1.ClusterService.Create \
        < body.json
      

4. View the server response to make sure the request was successful.

Create a Managed Service for ClickHouse® cluster and a network for it with the following test specifications:

• Name: mych.

• Environment: PRESTABLE.

• Cloud ID: b1gq90dgh25bebiu75o.

• Folder ID: b1gia87mbaomkfvsleds.

• New cloud network: cluster-net.

• New default security group: cluster-sg (in the cluster-net network). It must allow connections to any cluster host from any network (including the internet) on ports 8443 and 9440.

• One s2.micro class hosts in a new subnet named cluster-subnet-ru-central1-a.

  Subnet parameters:

  • Address range: 172.16.1.0/24.
  • Network: cluster-net.
  • Availability zone: ru-central1-a.

• Network SSD storage (network-ssd): 32 GB.

• Database name: db1.

• User: user1 with password user1user1.

The configuration files for this cluster are as follows:

1. Configuration file with a description of provider settings:

   provider.tf

   terraform {
     required_providers {
       yandex = {
         source = "yandex-cloud/yandex"
       }
     }
   }
   
   provider "yandex" {
     token     = "<OAuth_or_static_key_of_service_account>"
     cloud_id  = "b1gq90dgh25bebiu75o"
     folder_id = "b1gia87mbaomkfvsleds"
   }
   

   To get an OAuth token or a static access key, see the Yandex Identity and Access Management instructions.

2. Configuration file with a description of the cloud network and subnet:

   networks.tf

   resource "yandex_vpc_network" "cluster-net" { name = "cluster-net" }
   
   resource "yandex_vpc_subnet" "cluster-subnet-a" {
     name           = "cluster-subnet-ru-central1-a"
     zone           = "ru-central1-a"
     network_id     = yandex_vpc_network.cluster-net.id
     v4_cidr_blocks = ["172.16.1.0/24"]
   }
   

3. Configuration file with a description of the security group:

   security-groups.tf

   resource "yandex_vpc_default_security_group" "cluster-sg" {
     network_id = yandex_vpc_network.cluster-net.id
   
     ingress {
       description    = "HTTPS (secure)"
       port           = 8443
       protocol       = "TCP"
       v4_cidr_blocks = ["0.0.0.0/0"]
     }
   
     ingress {
       description    = "clickhouse-client (secure)"
       port           = 9440
       protocol       = "TCP"
       v4_cidr_blocks = ["0.0.0.0/0"]
     }
   
     egress {
       description    = "Allow all egress cluster traffic"
       protocol       = "TCP"
       v4_cidr_blocks = ["0.0.0.0/0"]
     }
   }
   

4. Configuration file with a description of the cluster and cluster host:

   cluster.tf

   resource "yandex_mdb_clickhouse_cluster" "mych" {
     name               = "mych"
     environment        = "PRESTABLE"
     network_id         = yandex_vpc_network.cluster-net.id
     security_group_ids = [yandex_vpc_default_security_group.cluster-sg.id]
   
     clickhouse {
       resources {
         resource_preset_id = "s2.micro"
         disk_type_id       = "network-ssd"
         disk_size          = 32
       }
     }
   
     host {
       type      = "CLICKHOUSE"
       zone      = "ru-central1-a"
       subnet_id = yandex_vpc_subnet.cluster-subnet-a.id
     }
   
     database {
       name = "db1"
     }
   
     user {
       name     = "user1"
       password = "user1user1"
       permission {
         database_name = "db1"
       }
     }
   }