Exporting and importing Hive metadata in a Apache Hive™ Metastore cluster

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

To export metadata from a Apache Hive™ Metastore cluster, run this command:

yc managed-metastore cluster export-data <cluster_name_or_ID> \
   --bucket <bucket_name> \
   --filepath <data_file>

Where:

• --bucket: Bucket you created earlier for cluster data export.
• --filepath: Path to the .sql file to which the cluster data will be written. If a file with that name already exists, it will be overwritten.

You can get the cluster ID and name with the list of clusters in the folder.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Use the Cluster.ExportData method and send the following request, e.g., via cURL:

   curl \
       --request POST \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --url 'https://metastore.api.cloud.yandex.net/managed-metastore/v1/clusters/<cluster_ID>:export' \
       --data '{
                  "bucket": "<bucket_name>",
                  "filepath": "<data_file>"
               }'
   

   Where:

   • bucket: Bucket you created earlier for cluster data export.
   • filepath: Path to the .sql file to which the cluster data will be written. If a file with that name already exists, it will be overwritten.

   You can get the cluster ID and name with the list of clusters in the folder.

3. View the server response to make sure your request was successful.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService.ExportData call and send the following request, e.g., via gRPCurl:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/metastore/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
               "cluster_id": "<cluster_ID>",
               "bucket": "<bucket_name>",
               "filepath": "<data_file>"
           }' \
       metastore.api.cloud.yandex.net:443 \
       yandex.cloud.metastore.v1.ClusterService.ExportData
   

   Where:

   • bucket: Bucket you created earlier for cluster data export.
   • filepath: Path to the .sql file to which the cluster data will be written. If a file with that name already exists, it will be overwritten.

   You can get the cluster ID with the list of clusters in the folder.

4. View the server response to make sure your request was successful.

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

To import metadata to a Apache Hive™ Metastore cluster, run this command:

yc managed-metastore cluster import-data <cluster_name_or_ID> \
   --bucket <bucket_name> \
   --filepath <data_file>

Where:

• --bucket: Bucket you created earlier to import the cluster data from.
• --filepath: Path to the .sql file to import the cluster data from.

You can get the cluster ID and name with the list of clusters in the folder.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Use the Cluster.ImportData method and send the following request, e.g., via cURL:

   curl \
       --request POST \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --url 'https://metastore.api.cloud.yandex.net/managed-metastore/v1/clusters/<cluster_ID>:import' \
       --data '{
                  "bucket": "<bucket_name>",
                  "filepath": "<data_file>"
               }'
   

   Where:

   • bucket: Bucket you created earlier to import the cluster data from.
   • filepath: Path to the .sql file to import the cluster data from.

   You can get the cluster ID and name with the list of clusters in the folder.

3. View the server response to make sure your request was successful.

1. Get an IAM token for API authentication and save it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService.ImportData call and send the following request, e.g., via gRPCurl:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/metastore/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
               "cluster_id": "<cluster_ID>",
               "bucket": "<bucket_name>",
               "filepath": "<data_file>"
           }' \
       metastore.api.cloud.yandex.net:443 \
       yandex.cloud.metastore.v1.ClusterService.ImportData
   

   Where:

   • bucket: Bucket you created earlier for cluster data export.
   • filepath: Path to the .sql file to which the cluster data will be written. If a file with that name already exists, it will be overwritten.

   You can get the cluster ID with the list of clusters in the folder.

4. View the server response to make sure your request was successful.