Updating a cluster Apache Hive™ Metastore

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

To change a Apache Hive™ Metastore cluster’s settings:

1. View the description of the CLI command for updating a cluster:

   yc managed-metastore cluster update --help
   

2. Provide a list of settings to update in the update cluster command:

   yc managed-metastore cluster update <cluster_name_or_ID> \
      --new-name <cluster_name> \
      --description <cluster_description> \
      --labels <label_list> \
      --service-account-id <service_account_ID> \
      --version <Apache_Hive™_Metastore_version> \
      --security-group-ids <security_group_IDs> \
      --resource-preset-id <ID_of_computing_resources> \
      --maintenance-window type=<maintenance_type>,`
                           `day=<day_of_week>,`
                           `hour=<hour> \
      --deletion-protection \
      --log-enabled \
      --log-folder-id <folder_ID> \
      --log-min-level <logging_level>
   

   Where:

   • --new-name: New cluster name.

   • --description: Cluster description.

   • --labels: List of labels. Provide labels in <key>=<value> format.

   • --service-account-id: Service account ID.

   • --version: Apache Hive™ Metastore version.

     Available versions: 3.1 and 4.0.

     Warning

     To integrate the Apache Hive™ Metastore cluster with Yandex Managed Service for Trino and Yandex Managed Service for Apache Spark™, you need version 3.1.

     If required, you can upgrade 3.1 to 4.0, but you cannot downgrade 4.0 to 3.1.

   • --security-group-ids: List of security group IDs.

   • --resource-preset-id: Computing resource configuration.

   • --maintenance-window: Maintenance window settings (including for disabled clusters), where type is the maintenance type:

     • anytime: At any time (default).
     • weekly: On a schedule. For this value, also specify the following:
       • day: Day of week, i.e., MON, TUE, WED, THU, FRI, SAT, or SUN.
       • hour: Hour of day (UTC), from 1 to 24.

   • --deletion-protection: Enables cluster protection against accidental deletion.

   • Logging parameters:

     • --log-enabled: Enables logging. Logs generated by Apache Hive™ Metastore components will be sent to Yandex Cloud Logging.

     • --log-folder-id: Folder ID. Logs will be written to the default log group for this folder.

     • --log-group-id: Custom log group ID. Logs will be written to this group.

       Specify one of the two parameters: --log-folder-id or --log-group-id.

     • --log-min-level: Minimum logging level. Possible values: TRACE, DEBUG, INFO (default), WARN, ERROR, and FATAL.

   You can get the cluster name and ID from the folder’s cluster list.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Create a file named body.json and paste the following code into it:

   {
     "updateMask": "<list_of_parameters_to_update>",
     "name": "<cluster_name>",
     "description": "<cluster_description>",
     "labels": { "<label_list>" },
     "deletionProtection": <deletion_protection>,
     "version": "<Apache_Hive™_Metastore_version>",
     "configSpec": {
       "resources": {
       "resourcePresetId": "<resource_configuration_ID>"
       }
     },
     "serviceAccountId": "<service_account_ID>",
     "logging": {
       "enabled": <use_of_logging>,
       "folderId": "<folder_ID>",
       "minLevel": "<logging_level>"
     },
     "networkSpec": {
       "securityGroupIds": [ "<list_of_security_group_IDs>" ]
     },
     "maintenanceWindow": {
       "weeklyMaintenanceWindow": {
       "day": "<day_of_week>",
       "hour": "<hour>"
       }
     }
   }
   

   Where:

   • updateMask: List of parameters to update as a single string, separated by commas.

     Warning

     When you update a cluster, all parameters of the object you are modifying will take their defaults unless explicitly provided in the request. To avoid this, list the settings you want to change in the updateMask parameter.

   • name: Cluster name.

   • description: Cluster description.

   • labels: List of labels provided in "<key>": "<value>" format.

   • deletionProtection: Enables cluster protection against accidental deletion. The possible values are true or false.

   • version: Apache Hive™ Metastore version.

     Available versions: 3.1 and 4.0.

     Warning

     To integrate the Apache Hive™ Metastore cluster with Yandex Managed Service for Trino and Yandex Managed Service for Apache Spark™, you need version 3.1.

     If required, you can upgrade 3.1 to 4.0, but you cannot downgrade 4.0 to 3.1.

   • configSpec.resources.resourcePresetId: Computing resource configuration.

   • serviceAccountId: Service account ID.

   • logging: Logging parameters:

     • enabled: Enables logging. Logs generated by Apache Hive™ Metastore components will be sent to Yandex Cloud Logging. The possible values are true or false.

     • folderId: Folder ID. Logs will be written to the default log group for this folder.

     • logGroupId: Custom log group ID. Logs will be written to this group.

       Specify either folderId or logGroupId.

     • minLevel: Minimum logging level. The possible values are TRACE, DEBUG, INFO, WARN, ERROR, and FATAL.

   • networkSpec: Network settings:

     • securityGroupIds: List of security group IDs.

   • maintenanceWindow: Maintenance window settings (including for disabled clusters). In maintenanceWindow, provide one of the two parameters:

     • anytime: Maintenance can take place at any time.

     • weeklyMaintenanceWindow: Maintenance takes place once a week at the specified time:

       • day: Day of week, in DDD format, MON, TUE, WED, THU, FRI, SAT, or SUN.
       • hour: Time of day (UTC) in HH format, from 1 to 24.

3. Use the Cluster.Update method and send the following request, e.g., via cURL:

   curl \
       --request PATCH \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --url 'https://metastore.api.cloud.yandex.net/managed-metastore/v1/clusters/<cluster_ID>' \
       --data '@body.json'
   

   You can get the cluster ID from the folder’s cluster list.

4. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Create a file named body.json and paste the following code into it:

   {
     "cluster_id": "<cluster_ID>",
     "update_mask": "<list_of_parameters_to_update>",
     "name": "<cluster_name>",
     "description": "<cluster_description>",
     "labels": "{ <label_list> }",
     "deletion_protection": <deletion_protection>,
     "version": "<Apache_Hive™_Metastore_version>",
     "config_spec": {
       "resources": {
         "resource_preset_id": "<resource_configuration_ID>"
       }
     },
     "service_account_id": "<service_account_ID>",
     "logging": {
       "enabled": <use_of_logging>,
       "folder_id": "<folder_ID>",
       "min_level": "<logging_level>"
     },
     "network_spec": {
       "security_group_ids": [ "<list_of_security_group_IDs>" ]
     },
     "maintenance_window": {
       "weekly_maintenance_window": {
         "day": "<day_of_week>",
         "hour": "<hour>"
       }
     }
   }
   

   Where:

   • cluster_id: Cluster ID. You can get it with the list of clusters in a folder.

   • update_mask: List of parameters to update as an array of paths[] strings.

     Format for listing settings

     "update_mask": {
         "paths": [
             "<setting_1>",
             "<setting_2>",
             ...
             "<setting_N>"
         ]
     }
     

     Warning

     When you update a cluster, all parameters of the object you are modifying will take their defaults unless explicitly provided in the request. To avoid this, list the settings you want to change in the update_mask parameter.

   • name: Cluster name.

   • description: Cluster description.

   • labels: List of labels provided in "<key>": "<value>" format.

   • deletion_protection: Enables cluster protection against accidental deletion. The possible values are: true or false.

   • version: Apache Hive™ Metastore version.

     Available versions: 3.1 and 4.0.

     Warning

     To integrate the Apache Hive™ Metastore cluster with Yandex Managed Service for Trino and Yandex Managed Service for Apache Spark™, you need version 3.1.

     If required, you can upgrade 3.1 to 4.0, but you cannot downgrade 4.0 to 3.1.

   • config_spec.resources.resource_preset_id: Computing resource configuration.

   • service_account_id: Service account ID.

   • logging: Logging parameters:

     • enabled: Enables logging. Logs generated by Apache Hive™ Metastore components will be sent to Yandex Cloud Logging. The possible values are: true or false.

     • folder_id: Folder ID. Logs will be written to the default log group for this folder.

     • log_group_id: Custom log group ID. Logs will be written to this group.

       Specify one of the two parameters: folder_id or log_group_id.

     • min_level: Minimum logging level. The possible values are: TRACE, DEBUG, INFO, WARN, ERROR, and FATAL.

   • network_spec: Network settings:

     • security_group_ids: List of security group IDs.

   • maintenance_window: Maintenance window settings (including for disabled clusters). In maintenance_window, provide one of the two parameters:

     • anytime: Maintenance can take place at any time.

     • weekly_maintenance_window: Maintenance takes place once a week at the specified time:

       • day: Day of week, in DDD format, MON, TUE, WED, THU, FRI, SAT, or SUN.
       • hour: Time of day (UTC) in HH format, from 1 to 24.

4. Use the ClusterService.Update call and send the following request, e.g., via gRPCurl:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/metastore/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d @ \
       metastore.api.cloud.yandex.net:443 \
       yandex.cloud.metastore.v1.ClusterService.Update \
       < body.json
   

5. View the server response to make sure your request was successful.