Managing a custom geobase in Managed Service for ClickHouse®
A ClickHouse® geobase consists of text files containing the names and hierarchy of regions. You can add several alternative geobases to ClickHouse® to reflect different points of view on what regions belong to what countries. For more information, see this ClickHouse® guide
To add a custom geobase to a ClickHouse® cluster:
- Create a geobase.
- Upload the geobase to Yandex Object Storage.
- Add the geobase to a ClickHouse® cluster.
If you are not using your custom geobase, remove it.
Creating a geobase
-
Create a file named
regions_hierarchy.txt. It must be a TSV table without headers, containing the following columns:- Region ID (UInt32)
- Parent region ID (UInt32)
- Region type (UInt8):
1: Continent3: Country4: Federal district5: Region6: City
- Population (UInt32): Optional column.
-
To add an alternative hierarchy of regions, create the
regions_hierarchy_<suffix>.txtfiles with the same structure. To use an alternative geobase, provide this suffix when calling the function. Here is an example:regionToCountry(RegionID): Uses the default dictionary,regions_hierarchy.txt.regionToCountry(RegionID, 'alt'): Uses the dictionary with thealtsuffix,regions_hierarchy_alt.txt.
-
Create a file named
regions_names.txt. It must be a TSV table without headers, containing the following columns:- Region ID (UInt32).
- Region name (String): Cannot contain tab or newline characters, even escaped ones.
-
To add region names in other languages to your geobase, create the
regions_names_<language_code>.txtfiles with the same structure. For example, you can createregions_names_en.txtfor English andregions_names_tr.txtfor Turkish. -
Create a
tar,tar.gz, orziparchive from the geobase files.
Uploading a geobase to Yandex Object Storage
Managed Service for ClickHouse® only works with geobases uploaded to Yandex Object Storage and accessible for reading:
- To attach a service account to a cluster, assign the iam.serviceAccounts.user role or higher to your Yandex Cloud account.
- Upload the geobase archive to Yandex Object Storage.
- Attach the service account to the cluster. You will use this service account to configure access to the geobase archive.
- Assign the
storage.viewerrole to the service account. - In the bucket ACL, add the
READpermission to the service account. - Get a link to the geobase archive.
Adding a geobase to a ClickHouse® cluster
- In the management console
, select the folder the cluster is in. - Go to Managed Service for ClickHouse.
- Select your cluster and click Edit in the top panel.
- Under DBMS settings, click Settings.
- In the Geobase uri field, enter a link to the geobase archive in Yandex Object Storage.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To add a geobase:
-
View the description of the CLI command to update the cluster configuration:
yc managed-clickhouse cluster update-config --help -
Run this command, providing the link to the archive with the geobase being added in the
geobase_uriparameter:yc managed-clickhouse cluster update-config <cluster_name_or_ID> \ --set geobase_uri="<link_to_geobase_archive_in_Object_Storage>"You can request the cluster ID and name with the list of clusters in the folder.
-
Open the current Terraform configuration file describing your infrastructure.
For information on how to create such a file, see Creating a cluster.
-
In the Managed Service for ClickHouse® cluster settings, add the
geobase_uriparameter with the link to the archive containing the geobase being added in Yandex Object Storage:resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" { ... clickhouse { config { geobase_uri = "<link_to_geobase_archive_in_Object_Storage>" ... } ... } ... } -
Make sure the settings are correct.
-
In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
-
Run this command:
terraform validateTerraform will show any errors found in your configuration files.
-
-
Confirm updating the resources.
-
Run this command to view the planned changes:
terraform planIf you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
-
If everything looks correct, apply the changes:
-
Run this command:
terraform apply -
Confirm updating the resources.
-
Wait for the operation to complete.
-
-
For more information, see this Terraform provider guide.
Timeouts
The Terraform provider sets the following timeouts for Managed Service for ClickHouse® cluster operations:
- Creating a cluster, including by restoring from a backup: 60 minutes.
- Updating a cluster: 90 minutes.
- Deleting a cluster: 30 minutes.
Operations exceeding the timeout are aborted.
How do I change these limits?
Add a timeouts section to the cluster description, e.g.:
resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.Update method, e.g., via the following cURL
request:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-clickhouse/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.clickhouse.config.geobaseUri", "configSpec": { "clickhouse": { "config": { "geobaseUri": "<link>" } } } }'Where:
-
updateMask: Comma-separated list of settings you want to update.Here, we only specified a single setting,
configSpec.clickhouse.config.geobaseUri. -
configSpec.clickhouse.config.geobaseUri: Link to the geobase archive in Object Storage.
You can get the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Call the ClusterService.Update method, e.g., via the following gRPCurl
request:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/clickhouse/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.clickhouse.config.geobase_uri" ] }, "config_spec": { "clickhouse": { "config": { "geobase_uri": "<link>" } } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.clickhouse.v1.ClusterService.UpdateWhere:
-
update_mask: List of settings you want to update as an array of strings (paths[]).Here, we only specified a single setting,
config_spec.clickhouse.config.geobase_uri. -
config_spec.clickhouse.config.geobase_uri: Link to the geobase archive in Object Storage.
You can get the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
Removing a geobase
- In the management console
, select the folder the cluster is in. - Go to Managed Service for ClickHouse.
- Select your cluster and click Edit in the top panel.
- Under DBMS settings, click Settings.
- Delete the value in the Geobase uri field.
If you do not have the Yandex Cloud CLI installed yet, install and initialize it.
By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.
To remove a geobase:
-
View the description of the CLI command to update the cluster configuration:
yc managed-clickhouse cluster update-config --help -
Run this command, providing an empty value in the
geobase_uriparameter:yc managed-clickhouse cluster update-config <cluster_name_or_ID> \ --set geobase_uri=""You can get the cluster ID and name with the list of clusters in the folder.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Call the Cluster.Update method, e.g., via the following cURL
request:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
updateMaskparameter as a single comma-separated string.curl \ --request PATCH \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-clickhouse/v1/clusters/<cluster_ID>' \ --data '{ "updateMask": "configSpec.clickhouse.config.geobaseUri", "configSpec": { "clickhouse": { "config": { "geobaseUri": "" } } } }'Where:
-
updateMask: Comma-separated list of settings you want to update.Here, we only specified a single setting,
configSpec.clickhouse.config.geobaseUri. -
configSpec.clickhouse.config.geobaseUri: Link to the geobase archive in Object Storage. Set an empty value for this parameter.
You can get the cluster ID with the list of clusters in the folder.
-
-
View the server response to make sure your request was successful.
-
Get an IAM token for API authentication and put it into an environment variable:
export IAM_TOKEN="<IAM_token>" -
Clone the cloudapi
repository:cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapiBelow, we assume the repository contents are stored in the
~/cloudapi/directory. -
Call the ClusterService.Update method, e.g., via the following gRPCurl
request:Warning
The API method will assign default values to all the parameters of the object you are modifying unless you explicitly provide them in your request. To avoid this, list the settings you want to change in the
update_maskparameter as an array ofpaths[]strings.Format for listing settings
"update_mask": { "paths": [ "<setting_1>", "<setting_2>", ... "<setting_N>" ] }grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/clickhouse/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d '{ "cluster_id": "<cluster_ID>", "update_mask": { "paths": [ "config_spec.clickhouse.config.geobase_uri" ] }, "config_spec": { "clickhouse": { "config": { "geobase_uri": "" } } } }' \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.clickhouse.v1.ClusterService.UpdateWhere:
-
update_mask: List of settings you want to update as an array of strings (paths[]).Here, we only specified a single setting,
config_spec.clickhouse.config.geobase_uri. -
config_spec.clickhouse.config.geobase_uri: Link to the geobase archive in Object Storage. Set an empty value for this parameter.
You can get the cluster ID with the list of clusters in the folder.
-
-
Check the server response to make sure your request was successful.
ClickHouse® is a registered trademark of ClickHouse, Inc