Creating a trigger for Cloud Logging that invokes Cloud Functions

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

To create a trigger that invokes a function, run this command:

yc serverless trigger create logging \
  --name <trigger_name> \
  --log-group-name <log_group_name> \
  --batch-size <message_batch_size> \
  --batch-cutoff <maximum_wait_time> \
  --resource-ids <resource_ID> \
  --resource-types <resource_type> \
  --stream-names <log_stream> \
  --log-levels <logging_level> \
  --invoke-function-id <function_ID> \
  --invoke-function-service-account-id <service_account_ID> \
  --retry-attempts <number_of_retry_attempts> \
  --retry-interval <interval_between_retry_attempts> \
  --dlq-queue-id <dead-letter_queue_ID> \
  --dlq-service-account-id <service_account_ID>

Where:

• --name: Trigger name.
• --log-group-name: Name of the log group whose new log entries will invoke the function.

• --batch-size: Message batch size. This is an optional setting. The values may range from 1 to 10. The default value is 1.
• --batch-cutoff: Maximum wait time. This is an optional setting. The values may range from 1 to 60 seconds. The default value is 1 second. The trigger groups messages within the batch-cutoff period and sends them to the function. The number of messages cannot exceed batch-size.

• --resource-ids: IDs of your resources or Yandex Cloud resources, e.g., functions in Cloud Functions. This is an optional setting.
• --resource-types: Types of resources, e.g., serverless.function in Cloud Functions. This is an optional setting.
• --stream-names: Log streams. This is an optional setting.
• --log-levels: Logging levels. This is an optional setting.
  A trigger fires when the specified log group receives entries that comply with all of the following settings: resource-ids, resource-types, stream-names, and log-levels. If the setting is not specified, the trigger fires for any value.

• --invoke-function-id: Function ID.
• --invoke-function-service-account-id: ID of the service account with permissions to invoke the function.
• --retry-attempts: Number of invocation retries before the trigger sends a message to the dead-letter queue. This is an optional setting. The values may range from 1 to 5. The default value is 1.
• --retry-interval: Time to wait before retrying the function if it fails. This is an optional setting. The values may range from 10 to 60 seconds. The default value is 10 seconds.
• --dlq-queue-id: Dead-letter queue ID. This is an optional setting.
• --dlq-service-account-id: ID of the service account with write permissions for the dead-letter queue. This is an optional setting.

Result:

id: a1sfe084v4**********
folder_id: b1g88tflru**********
created_at: "2019-12-04T08:45:31.131391Z"
name: logging-trigger
rule:
  logging:
    log-group-name: default
    resource_type:
      - serverless.functions
    resource_id:
      - d4e1gpsgam78********
    stream_name:
      - test
    levels:
      - INFO
    batch_settings:
      size: "1"
      cutoff: 1s
    invoke_function:
      function_id: d4eofc7n0m**********
      function_tag: $latest
      service_account_id: aje3932acd**********
      retry_settings:
        retry_attempts: "1"
        interval: 10s
      dead_letter_queue:
        queue-id: yrn:yc:ymq:ru-central1:aoek49ghmk**********:dlq
        service-account-id: aje3932a**********
status: ACTIVE

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

Terraform is distributed under the Business Source License. The Yandex Cloud provider for Terraform is distributed under the MPL-2.0 license.

For more information about the provider resources, see the relevant documentation on the Terraform website or its mirror.

If you do not have Terraform yet, install it and configure the Yandex Cloud provider.

To create a trigger for Cloud Logging:

1. In the Terraform configuration file, describe the resources you want to create:

   resource "yandex_function_trigger" "my_trigger" {
     name        = "<trigger_name>"
     description = "<trigger_description>"
     function {
        id                 = "<function_ID>"
        service_account_id = "<service_account_ID>"
        retry_attempts     = "<number_of_retry_attempts>"
        retry_interval     = "<interval_between_retry_attempts>"
     }
     logging {
        group_id       = "<log_group_ID>"
        resource_types = [ "<resource_type>" ]
        resource_ids   = [ "<resource_ID>" ]
        stream_names   = [ "<log_stream>" ]
        levels         = [ "<logging_level>", "<logging_level>" ]
        batch_cutoff   = "<maximum_wait_time>"
        batch_size     = "<message_batch_size>"
     }
     dlq {
       queue_id           = "<dead-letter_queue_ID>"
       service_account_id = "<service_account_ID>"
     }
   }
   

   Where:

   • name: Trigger name. Follow these naming requirements:

     • Length: between 3 and 63 characters.
     • It can only contain lowercase Latin letters, numbers, and hyphens.
     • It must start with a letter and cannot end with a hyphen.

   • description: Trigger description.

   • function: Function settings:

     • id: Function ID.
     • service_account_id: ID of the service account with permissions to invoke the function.
     • retry_attempts: Number of invocation retries before the trigger sends a message to the dead-letter queue. This is an optional setting. The values may range from 1 to 5. The default value is 1.
     • retry_interval: Time to wait before retrying the function if it fails. This is an optional setting. The values may range from 10 to 60 seconds. The default value is 10 seconds.

   • logging: Trigger settings:

     • group_id: ID of the log group whose new log entries will invoke the function.

     • resource_types: Types of resources, e.g., Cloud Functions such as resource_types = [ "serverless.function" ]. You can specify multiple types.

     • resource_ids: IDs of your resources or Yandex Cloud resources, e.g., resource_ids = [ "<function_ID>" ]. You can specify multiple IDs.

     • stream_names: Log streams. This is an optional setting.

     • levels: Logging levels, e.g., levels = [ "INFO", "ERROR"].

       A trigger fires when the specified log group receives entries that comply with all of the following settings: resource-ids, resource-types, stream-names, and levels. If the setting is not specified, the trigger fires for any value.

     • batch_cutoff: Maximum wait time. The valid values range from 0 to 60 seconds. The trigger groups messages within the specified wait time period and sends them to the function. The number of messages cannot exceed the specified batch-size.

     • batch_size: Message batch size. The valid values range from 1 to 10.

   • dlq: Dead-letter queue settings:
     • queue_id: Dead-letter queue ID.
     • service_account_id: ID of the service account with write permissions for the dead-letter queue.

   For more information about yandex_function_trigger properties, see this provider guide.

2. Create the resources:

   1. In the terminal, navigate to the configuration file directory.

   2. Make sure the configuration is correct using this command:

      terraform validate
      

      If the configuration is valid, you will get this message:

      Success! The configuration is valid.
      

   3. Run this command:

      terraform plan
      

      You will see a list of resources and their properties. No changes will be made at this step. Terraform will show any errors in the configuration.

   4. Apply the configuration changes:

      terraform apply
      

   5. Type yes and press Enter to confirm the changes.

   Terraform will create all the required resources. You can check the new resources using the management console or this CLI command:

   yc serverless trigger list
   

To create a trigger for Cloud Logging, use the create REST API method for the Trigger resource or the TriggerService/Create gRPC API call.