Contact UsTry it for free
© 2025 Direct Cursus Technology L.L.C.

Transferring Yandex MPP Analytics for PostgreSQL cluster logs to Yandex Cloud Logging

Written by
Updated at December 3, 2025

You can set up regular collection of Greenplum® cluster performance logs. Logs will be delivered to a log group in Cloud Logging. You can choose between these two types of log groups:

Getting started

Create a service account with the logging.writer role.

Transferring data to the default log group

  1. Create a Greenplum® cluster with active logging and a service account created earlier:

    1. In the management console, select the folder where you want to create a cluster.

    2. Select Yandex MPP Analytics for PostgreSQL.

    3. Click Create cluster.

    4. Enter a name for the cluster.

    5. Under Network settings, select:

      • Cloud network.
      • Security group.
      • Availability zone and subnet.

    6. Under User, specify the admin user credentials:

      • Username may contain Latin letters, numbers, hyphens, and underscores, but cannot start with a hyphen. It must be from 1 to 32 characters long.
      • The Password must be from 8 to 128 characters long.

    7. Under Advanced settings:

      • Select the service account with the logging.writer role.
      • Enable Write logs.
      • To write logs to the default log group, select Folder in the Destination field.
      • Select the folder whose log group you want to use.
      • Enable the Greenplum logs and Command Center logs options. Use Log min messages under DBMS settings to specify the logging level.

    8. Click Create.

    yc managed-greenplum cluster create <cluster_name> \
   --greenplum-version=6.25 \
   --environment=PRODUCTION \
   --network-name=<cluster_network_name> \
   --user-name=<VM_user_name> \
   --user-password=<cluster_user_password> \
   --master-config resource-id=s3-c8-m32,`
                  `disk-size=10,`
                  `disk-type=network-hdd \
   --segment-config resource-id=s3-c8-m32,`
                  `disk-size=93,`
                  `disk-type=network-ssd-nonreplicated \
   --zone-id=<availability_zone_ID> \
   --subnet-id=<cluster_subnet_ID> \
   --assign-public-ip=true \
   --security-group-ids=<cluster_security_group_IDs> \
   --service-account <cluster_service_account_ID> \
   --log-enabled \
   --log-greenplum-enabled \
   --log-command-center-enabled \
   --log-folder-id <folder_ID>

    Where:

    • --service-account: ID of the service account you created earlier.

    • --log-enabled: Enables log transfer. Required for other flags responsible for transferring specific logs to work, e.g., --log-greenplum-enabled.

    • --log-greenplum-enabled: Transferring Greenplum® logs.

      You can set the Greenplum® logging level in the Log min messages parameter. Learn more about Greenplum® advanced settings.

    • --log-command-center-enabled: Transferring Command Center logs.

      Command Center logs are trasferred in full; you cannot change the logging level.

    • --log-folder-id: ID of the folder the Greenplum® cluster was created in.

    1. If you do not have Terraform yet, install it.

    2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

    3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

    4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

    5. Specify the following parameters in the configuration file with the cluster description:

      resource "yandex_mdb_greenplum_cluster" "gp-cluster" {
  name               = "<cluster_name>"
  environment        = "PRODUCTION"
  network_id         = "<network_ID>"
  zone               = "<availability_zone_ID>"
  subnet_id          = "<cluster_subnet_ID>"
  assign_public_ip   = true
  version            = "6.25"
  master_host_count  = 2
  segment_host_count = 2
  segment_in_host    = 2
  service_account_id = "<service_account_ID>"

  master_subcluster {
    resources {
      resource_preset_id = "s3-c8-m32" # 8 vCPU, 32 GB RAM
      disk_size          = 10          # GB
      disk_type_id       = "network-hdd"
    }
  }

  segment_subcluster {
    resources {
      resource_preset_id = "s3-c8-m32" # 8 vCPU, 32 GB RAM
      disk_size          = 93          # GB
      disk_type_id       = "network-ssd-nonreplicated"
    }
  }

  user_name          = "<VM_user_name>"
  user_password      = "<cluster_user_password>"
  security_group_ids = ["<cluster_security_group_IDs>"]

  logging {
    enabled = true
    greenplum_enabled  = true
    command_center_enabled = true
    folder_id = <folder_ID>
  }
}

      Where:

      • service_account_id: ID of the service account you created earlier.

      • enabled: Manages log transferring. To enable parameters responsible for transferring specific logs, provide the true value.

      • greenplum_enabled: Transferring Greenplum® logs.

        You can set the Greenplum® logging level in the Log min messages parameter. Learn more about Greenplum® advanced settings.

      • command_center_enabled: Transferring Command Center logs.

        Command Center logs are trasferred in full; you cannot change the logging level.

      • folder_id: Specify the ID of the folder whose log group you want to use.

    6. Make sure the Terraform configuration files are correct using this command:

      terraform validate

      Terraform will show any errors found in your configuration files.

    7. Create the required infrastructure:

      1. Run this command to view the planned changes:

        terraform plan

        If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

      2. If everything looks correct, apply the changes:

        1. Run this command:

          terraform apply

        2. Confirm updating the resources.

        3. Wait for the operation to complete.

    Specify the following parameters in the body of the cluster create request (Cluster.Create in the REST API or ClusterService.Create in the gRPC API):

    {
  ...
  "serviceAccountId": "<service_account_ID>",
  "logging": {
    "enabled": "true",
    "greenplumEnabled": "true",
    "commandCenterEnabled": "true",
    "folderId": "<folder_ID>"
  }
  ...
}

    Where:

    • serviceAccountId: ID of the service account you created earlier.

    • enabled: Manages log transferring. To enable parameters responsible for transferring specific logs, provide the true value.

    • greenplumEnabled: Transferring Greenplum® logs.

      You can set the Greenplum® logging level in the Log min messages parameter. Learn more about Greenplum® advanced settings.

    • commandCenterEnabled: Transferring Command Center logs.

      Command Center logs are trasferred in full; you cannot change the logging level.

    • folderId: Specify the ID of the folder whose log group you want to use.

  2. Test the transfer of cluster logs to the log group:

    1. In the management console, navigate to the relevant folder.
    2. Select Cloud Logging.
    3. Click the row with the default log group.

    The page that opens will show the log group entries.

    To view the entries in JSON format, run this command:

    yc logging read --group-name=default --format=json

    Result:

    [
  {
    "uid": "22:********:15",
    "resource": {
      "type": "mdb.greenplum.cluster",
      "id": "c9qao2tkhbdt********"
    },
    "timestamp": "2025-04-25T07:50:21.871157Z",
    "ingested_at": "2025-04-25T07:50:24.553Z",
    "saved_at": "2025-04-25T07:50:26.125189Z",
    "level": "INFO",
    "message": "2025-04-25 10:50:20.209664 MSK,\"monitor\",\"postgres\",p29034,th1258748352,\"localhost\",\"47384\",2025-04-25 07:38:20 MSK,0,con40,cmd20190,seg-1,,,,sx1,\"LOG\",\"00000\",\"statement: \n\t\treset log_min_messages\n\t\",,,,,,,0,,\"postgres.c\",1664,",
    "json_payload": {
      "cid": "c9qao2tkhbdt********",
      "gp_host_type": "master",
      "gp_preferred_role": "primary",
      "hostname": "rc1d-jds7okr4********.mdb.yandexcloud.net",
      "segment_id": "-1",
      "username": "monitor"
    },
    "stream_name": "greenplum-logs-master-primary"
  }
]

    To view log group records, use the Cluster.ListLogs REST API method or the LogReadingService/Read gRPC API call.

    For more information, see Reading records.

Sending data to a custom log group

  1. Create a log group named greenplum-log-group.

  2. Create a Greenplum® cluster with active logging and a service account created earlier:

    1. In the management console, select the folder where you want to create a cluster.

    2. Select Yandex MPP Analytics for PostgreSQL.

    3. Click Create cluster.

    4. Enter a name for the cluster.

    5. Under Network settings, select:

      • Cloud network.
      • Security group.
      • Availability zone and subnet.

    6. Under User, specify the admin user credentials:

      • Username may contain Latin letters, numbers, hyphens, and underscores, but cannot start with a hyphen. It must be from 1 to 32 characters long.
      • The Password must be from 8 to 128 characters long.

    7. Under Advanced settings:

      • Select the service account with the logging.writer role.
      • Enable Write logs.
      • To write logs to a custom log group, select Log group in the Destination field.
      • Select the greenplum-log-group log group.
      • Enable the Greenplum logs and Command Center logs options. Use Log min messages under DBMS settings to specify the logging level.

    8. Click Create.

    yc managed-greenplum cluster create <cluster_name> \
   --greenplum-version=6.25 \
   --environment=PRODUCTION \
   --network-name=<cluster_network_name> \
   --user-name=<VM_user_name> \
   --user-password=<cluster_user_password> \
   --master-config resource-id=s3-c8-m32,`
                  `disk-size=10,`
                  `disk-type=network-hdd \
   --segment-config resource-id=s3-c8-m32,`
                  `disk-size=93,`
                  `disk-type=network-ssd-nonreplicated \
   --zone-id=<availability_zone_ID> \
   --subnet-id=<cluster_subnet_ID> \
   --assign-public-ip=true \
   --security-group-ids=<cluster_security_group_IDs> \
   --service-account <cluster_service_account_ID> \
   --log-enabled \
   --log-greenplum-enabled \
   --log-command-center-enabled \
   --log-group-id <log_group_ID>

    Where:

    • --service-account: ID of the service account you created earlier.

    • --log-enabled: Enables log transfer. Required for other flags responsible for transferring specific logs to work, e.g., --log-greenplum-enabled.

    • --log-greenplum-enabled: Transferring Greenplum® logs.

      You can set the Greenplum® logging level in the Log min messages parameter. Learn more about Greenplum® advanced settings.

    • --log-command-center-enabled: Transferring Command Center logs.

      Command Center logs are trasferred in full; you cannot change the logging level.

    • --log-group-id: ID of the log group to write logs to.

    1. If you do not have Terraform yet, install it.

    2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

    3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

    4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

    5. Specify the following parameters in the configuration file with the cluster description:

      resource "yandex_mdb_greenplum_cluster" "gp-cluster" {
  name               = "<cluster_name>"
  environment        = "PRODUCTION"
  network_id         = "<network_ID>"
  zone               = "<availability_zone_ID>"
  subnet_id          = "<cluster_subnet_ID>"
  assign_public_ip   = true
  version            = "6.25"
  master_host_count  = 2
  segment_host_count = 2
  segment_in_host    = 2
  service_account_id = "<service_account_ID>"

  master_subcluster {
    resources {
      resource_preset_id = "s3-c8-m32" # 8 vCPU, 32 GB RAM
      disk_size          = 10          # GB
      disk_type_id       = "network-hdd"
    }
  }

  segment_subcluster {
    resources {
      resource_preset_id = "s3-c8-m32" # 8 vCPU, 32 GB RAM
      disk_size          = 93          # GB
      disk_type_id       = "network-ssd-nonreplicated"
    }
  }

  user_name          = "<VM_user_name>"
  user_password      = "<cluster_user_password>"
  security_group_ids = ["<cluster_security_group_IDs>"]

  logging {
    enabled = true
    greenplum_enabled  = true
    command_center_enabled = true
    log_group_id = <log_group_ID>
  }
}

      Where:

      • service_account_id: ID of the service account you created earlier.

      • enabled: Manages log transferring. To enable parameters responsible for transferring specific logs, provide the true value.

      • greenplum_enabled: Transferring Greenplum® logs.

        You can set the Greenplum® logging level in the Log min messages parameter. Learn more about Greenplum® advanced settings.

      • command_center_enabled: Transferring Command Center logs.

        Command Center logs are trasferred in full; you cannot change the logging level.

      • log_group_id: ID of the log group to write logs to.

    6. Make sure the Terraform configuration files are correct using this command:

      terraform validate

      Terraform will show any errors found in your configuration files.

    7. Create the required infrastructure:

      1. Run this command to view the planned changes:

        terraform plan

        If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

      2. If everything looks correct, apply the changes:

        1. Run this command:

          terraform apply

        2. Confirm updating the resources.

        3. Wait for the operation to complete.

    Specify the following parameters in the body of the cluster create request (Cluster.Create in the REST API or ClusterService.Create in the gRPC API):

    {
  ...
  "serviceAccountId": "<service_account_ID>",
  "logging": {
    "enabled": "true",
    "greenplumEnabled": "true",
    "commandCenterEnabled": "true",
    "logGroupId": "<log_group_ID>"
  }
  ...
}

    Where:

    • serviceAccountId: ID of the service account you created earlier.

    • enabled: Manages log transferring. To enable parameters responsible for transferring specific logs, provide the true value.

    • greenplumEnabled: Transferring Greenplum® logs.

      You can set the Greenplum® logging level in the Log min messages parameter. Learn more about Greenplum® advanced settings.

    • commandCenterEnabled: Transferring Command Center logs.

      Command Center logs are trasferred in full; you cannot change the logging level.

    • logGroupId: Log group ID.

  3. Test the transfer of cluster logs to the log group:

    1. In the management console, navigate to the relevant folder.
    2. Select Cloud Logging.
    3. Click the row with the greenplum-log-group log group.

    The page that opens will show the log group entries.

    To view the entries in JSON format, run this command:

    yc logging read --group-name=greenplum-log-group --format=json

    Result:

    [
  {
    "uid": "22:********:15",
    "resource": {
      "type": "mdb.greenplum.cluster",
      "id": "c9qao2tkhbdt********"
    },
    "timestamp": "2025-04-25T07:50:21.871157Z",
    "ingested_at": "2025-04-25T07:50:24.553Z",
    "saved_at": "2025-04-25T07:50:26.125189Z",
    "level": "INFO",
    "message": "2025-04-25 10:50:20.209664 MSK,\"monitor\",\"postgres\",p29034,th1258748352,\"localhost\",\"47384\",2025-04-25 07:38:20 MSK,0,con40,cmd20190,seg-1,,,,sx1,\"LOG\",\"00000\",\"statement: \n\t\treset log_min_messages\n\t\",,,,,,,0,,\"postgres.c\",1664,",
    "json_payload": {
      "cid": "c9qao2tkhbdt********",
      "gp_host_type": "master",
      "gp_preferred_role": "primary",
      "hostname": "rc1d-jds7okr4********.mdb.yandexcloud.net",
      "segment_id": "-1",
      "username": "monitor"
    },
    "stream_name": "greenplum-logs-master-primary"
  }
]

    To view log group records, use the Cluster.ListLogs REST API method or the LogReadingService/Read gRPC API call.

    For more information, see Reading records.

Previous
Monitoring of event logging to a log group
Next
Log group