Viewing Greenplum® cluster logs

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

1. See the description of the CLI command for viewing cluster logs:

   yc managed-greenplum cluster list-logs --help
   

2. Run the following command to get cluster logs (our example only shows some of the available parameters):

   yc managed-greenplum cluster list-logs <cluster_name_or_ID> \
      --limit <entry_number_limit> \
      --format <output_format> \
      --service-type <service_type> \
      --columns <list_of_data_columns> \
      --filter <entry_filtration_settings> \
      --since <time_range_left_boundary> \
      --until <time_range_right_boundary>
   

   Where:

   • --limit: limits on the number of entries to output.

   • --format: Format for log entries (text, yaml, json, or json-rest).

   • --service-type: Type of service for which you want to output entries:

     • greenplum: Greenplum® operations log.
     • greenplum-pooler: Connection manager operations log.
     • greenplum-pxf: PXF protocol log.

   • --columns: List of data columns:

     • hostname: Host name.
     • level: Logging level, e.g., info.
     • pid: ID of the current session’s server process.
     • text: Message output by the component.

     Note

     The list of data columns depends on the selected --service-type. The example only shows the main columns for the greenplum-pooler type.

     To output separate columns, provide the --format parameter and specify the output format (yaml, json, or json-rest). Columns are not supported for text output format (default).

   • --filter: record filter settings, for example, message.hostname='node1.mdb.yandexcloud.net'.

   • --since: Left boundary of a time range in RFC-3339, HH:MM:SS format or a time interval relative to the current time. Examples: 2006-01-02T15:04:05Z, 15:04:05, 2h, 3h30m ago.

   • --until: right boundary of a time range, the format is similar to that of --since.

You can get the cluster name and ID with the list of clusters in the folder.

1. Get an IAM token for API authentication and place it in an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Call the Cluster.ListLogs method, e.g., via the following cURL request:

   curl \
       --request GET \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters/<cluster_ID>:logs' \
       --url-query serviceType=<service_type> \
       --url-query columnFilter=<column_name> \
       --url-query fromTime=<time_range_left_boundary> \
       --url-query toTime=<time_range_right_boundary>
   

   Where:

   • serviceType: Type of service to request logs for:

     • GREENPLUM: Greenplum® operations.
     • GREENPLUM_POOLER: Connection manager operations.
     • GREENPLUM_PXF: PXF protocol.

   • columnFilter: Name of the data column:

     • hostname: Host name.
     • level: Logging level, e.g., info.
     • pid: ID of the current session’s server process.
     • text: Message output by the component.

     Note

     The list of data columns depends on the selected serviceType. The example only shows the main columns for the GREENPLUM_POOLER type.

     You can specify only one column in the columnFilter parameter. If you want to filter logs by more than one column, provide a list of the columns in several parameters.

   • fromTime: Left boundary of a time range in RFC-3339 format, Example: 2006-01-02T15:04:05Z.
   • toTime: Right boundary of a time range, with the format analogous to that for fromTime.

   You can request the cluster ID with the list of clusters in the folder.

3. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and set it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService.ListLogs call and send the following request, e.g., via gRPCurl:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
             "cluster_id": "<cluster_ID>",
             "column_filter": [
               "<column_1>", "<column_2>", ..., "<column_N>"
             ],
             "service_type": "<service_type>",
             "from_time": "<time_range_left_boundary>",
             "to_time": "<time_range_right_boundary>"
           }' \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.greenplum.v1.ClusterService.ListLogs
   

   Where:

   • service_type: Type of service to request logs for:

     • GREENPLUM: Greenplum® operations.
     • GREENPLUM_POOLER: Connection manager operations.
     • GREENPLUM_PXF: PXF protocol.

   • column_filter: List of data columns:

     • hostname: Host name.
     • level: Logging level, e.g., info.
     • pid: ID of the current session’s server process.
     • text: Message output by the component.

     Note

     The list of data columns depends on the selected serviceType. The example only shows the main columns for the GREENPLUM_POOLER type.

     You can specify more than one column in the column_filter parameter if you want to filter logs by multiple columns.

   • from_time: Left boundary of a time range in RFC-3339 format, Example: 2006-01-02T15:04:05Z.
   • to_time: Right boundary of a time range, with the format analogous to that for from_time.

   You can request the cluster ID with the list of clusters in the folder.

4. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and set it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Call the Cluster.StreamLogs method, e.g., via the following cURL request:

   curl \
       --request GET \
       --header "Authorization: Bearer $IAM_TOKEN" \
       --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters/<cluster_ID>:stream_logs' \
       --url-query serviceType=<service_type> \
       --url-query columnFilter=<column_name> \
       --url-query fromTime=<time_range_left_boundary> \
       --url-query toTime=<time_range_right_boundary> \
       --url-query filter=<log_filter>
   

   Where:

   • serviceType: Type of service to request logs for:

     • GREENPLUM: Greenplum® operations.
     • GREENPLUM_POOLER: Connection manager operations.
     • GREENPLUM_PXF: PXF protocol.

   • columnFilter: Name of the data column:

     • hostname: Host name.
     • level: Logging level, e.g., info.
     • pid: ID of the current session’s server process.
     • text: Message output by the component.

     Note

     The list of data columns depends on the selected serviceType. The example only shows the main columns for the GREENPLUM_POOLER type.

     You can specify only one column in the columnFilter parameter. If you want to filter logs by more than one column, provide a list of the columns in several parameters.

   • fromTime: Left boundary of a time range in RFC-3339 format, Example: 2006-01-02T15:04:05Z.

   • toTime: Right boundary of a time range, with the format analogous to that for fromTime.

     If you omit this parameter, new logs will be sent to the log stream as they arrive. Semantically, this behavior is similar to tail -f.

   • filter: Log filter, e.g., message.hostname='node1.mdb.yandexcloud.net'.

     For more information about filters and their syntax, see the API reference.

   You can request the cluster ID with the list of clusters in the folder.

3. Check the server response to make sure your request was successful.

1. Get an IAM token for API authentication and set it as an environment variable:

   export IAM_TOKEN="<IAM_token>"
   

2. Clone the cloudapi repository:

   cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
   

   Below, we assume the repository contents are stored in the ~/cloudapi/ directory.

3. Use the ClusterService.StreamLogs call and send the following request, e.g., via gRPCurl:

   grpcurl \
       -format json \
       -import-path ~/cloudapi/ \
       -import-path ~/cloudapi/third_party/googleapis/ \
       -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/cluster_service.proto \
       -rpc-header "Authorization: Bearer $IAM_TOKEN" \
       -d '{
             "cluster_id": "<cluster_ID>",
             "column_filter": [
               "<column_1>", "<column_2>", ..., "<column_N>"
             ],
             "service_type": "<service_type>",
             "from_time": "<time_range_left_boundary>",
             "to_time": "<time_range_right_boundary>",
             "filter": "<log_filter>"
           }' \
       mdb.api.cloud.yandex.net:443 \
       yandex.cloud.mdb.greenplum.v1.ClusterService.StreamLogs
   

   Where:

   • service_type: Type of service to request logs for:

     • GREENPLUM: Greenplum® operations.
     • GREENPLUM_POOLER: Connection manager operations.
     • GREENPLUM_PXF: PXF protocol.

   • column_filter: List of data columns:

     • hostname: Host name.
     • level: Logging level, e.g., info.
     • pid: ID of the current session’s server process.
     • text: Message output by the component.

     Note

     The list of data columns depends on the selected service_type. The example only shows the main columns for the GREENPLUM_POOLER type.

     You can specify more than one column in the column_filter parameter if you want to filter logs by multiple columns.

   • from_time: Left boundary of a time range in RFC-3339 format, Example: 2006-01-02T15:04:05Z.

   • to_time: Right boundary of a time range, with the format analogous to that for from_time.

     If you omit this parameter, new logs will be sent to the log stream as they arrive. Semantically, this behavior is similar to tail -f.

   • filter: Log filter, e.g., message.hostname='node1.mdb.yandexcloud.net'.

     For more information about filters and their syntax, see the API reference.

   You can request the cluster ID with the list of clusters in the folder.

4. Check the server response to make sure your request was successful.