Creating a Hive Metastore cluster

Written by

Updated at December 10, 2024

Note

This feature is in the Preview stage.

To learn more about Metastore clusters in Yandex MetaData Hub, see Hive Metastore clusters.

Getting started

To link your service account to a Metastore cluster, make sure your Yandex Cloud account has the iam.serviceAccounts.user role or higher.
Set up a NAT gateway in the subnet the cluster will connect to. It is needed for the cluster to interact with Yandex Cloud services.
Configure the security group.
Create a service account.
Assign the managed-metastore.integrationProvider role to the service account. This role enables the cluster to work with Yandex Cloud services, e.g., Yandex Cloud Logging and Yandex Monitoring, under a service account.

You can also add more roles. Their combination depends on your specific use case. You can view the service roles in the Metastore section and all the available roles in this reference.
Optionally, create a static access key for your service account. This key enables the cluster to access Yandex Object Storage buckets.

Save the ID and secret key. You will not be able to access them later.

To work with buckets, assign one of the Object Storage roles to the service account.
If you want to save cluster logs to a custom log group, create one.

For more information, see Transferring cluster logs.

Warning

Metastore clusters cannot be edited. Once you create a cluster, you cannot change its settings.

Management console

In the management console, select the folder where you want to create a server.
Select Yandex MetaData Hub.
In the left-hand panel, select the Metastore page.
Click Create cluster.
Enter a name for the cluster. It must be unique within the folder.
(Optional) Enter a cluster description.
(Optional) Add Yandex Cloud labels to break resources into logical groups.
Specify the service account you created earlier.
Under Network settings, select the network and subnet to host the Metastore cluster. Specify the security group you configured previously.
Optionally, configure logging settings:
1. Enable Write logs.
2. Select where to write cluster logs to:
  - Default log group: Select Folder in the Destination field and specify the folder. Logs will be stored in the selected folder's default log group.
  - Custom log group: Select Log group in the Destination field and specify the log group you created earlier.
3. Select the minimum logging level.
  
  Logs of the specified level and higher will be written to the execution log. The available levels are TRACE, DEBUG, INFO, WARN, ERROR, and FATAL. The default level is INFO.
If required, enable protection of the cluster from accidental deletion by a user.

Cluster deletion protection will not prevent a manual connection to a cluster to delete data.
Click Create.