Contact UsGet started

Creating an external Hive data source

Written by
Updated at December 11, 2024

In Managed Service for Greenplum®, as an external data source with the Hive connection type, you can use a Hive DBMS as part of Yandex Data Processing or other third-party Hive services.

Create an external data source

  1. Go to the folder page and select Managed Service for Greenplum.
  2. Open the page of the Managed Service for Greenplum® cluster you need.
  3. In the left-hand panel, select  PXF.
  4. Click Create data source.
  5. Select the Hive connection type.
  6. Enter a source name.
  7. Configure at least one optional setting.
  8. Click Create.

  1. Get an IAM token for API authentication and put it into the environment variable:

    export IAM_TOKEN="<IAM_token>"

  2. Use the PXFDatasource.Create method and make a request, e.g., via cURL:

    curl \
    --request POST \
    --header "Authorization: Bearer $IAM_TOKEN" \
    --header "Content-Type: application/json" \
    --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters/<cluster_ID>/pxf_datasources' \
    --data '{
              "datasource": {
                "name": "<external_data_source_name>",
                "hive": {
                  "kerberos": {
                    "enable": <Kerberos_authentication>
                  },
                  ...
                }
              }
            }'

    Where:

    • name: External data source name.
    • hive: External data source settings. Configure at least one optional setting.

    You can get the cluster ID with a list of clusters in the folder.

  3. View the server response to make sure the request was successful.

  1. Get an IAM token for API authentication and put it into the environment variable:

    export IAM_TOKEN="<IAM_token>"

  2. Clone the cloudapi repository:

    cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi

    Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

  3. Use the PXFDatasourceService.Create call and make a request, e.g., via gRPCurl:

    grpcurl \
    -format json \
    -import-path ~/cloudapi/ \
    -import-path ~/cloudapi/third_party/googleapis/ \
    -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/pxf_service.proto \
    -rpc-header "Authorization: Bearer $IAM_TOKEN" \
    -d '{
          "cluster_id": "<cluster_ID>"
          "datasource": {
            "name": "<external_data_source_name>",
            "hive": {
              "kerberos": {
                "enable": <Kerberos_authentication>
              },
              ...
            }
          }
        }' \
    mdb.api.cloud.yandex.net:443 \
    yandex.cloud.mdb.greenplum.v1.PXFDatasourceService.Create

    Where:

    • name: External data source name.
    • hive: External data source settings. Configure at least one optional setting.

    You can get the cluster ID with a list of clusters in the folder.

  4. View the server response to make sure the request was successful.

Greenplum® and Greenplum Database® are registered trademarks or trademarks of VMware, Inc. in the United States and/or other countries.

Previous
HDFS
Next
Creating an external table