Creating a trigger for Object Storage that invokes a Serverless Containers container

If you do not have the Yandex Cloud (CLI) command line interface yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

To create a trigger that invokes a container, run this command:

yc serverless trigger create object-storage \
  --name <trigger_name> \
  --bucket-id <bucket_ID> \
  --prefix '<object_key_prefix>' \
  --suffix '<object_key_suffix>' \
  --events 'create-object','delete-object','update-object' \
  --batch-size <event_group_size> \
  --batch-cutoff <maximum_timeout> \
  --invoke-container-id <container_ID> \
  --invoke-container-service-account-id <service_account_ID> \
  --retry-attempts <number_of_retry_attempts> \
  --retry-interval <interval_between_retry_attempts> \
  --dlq-queue-id <dead-letter_queue_ID> \
  --dlq-service-account-id <service_account_ID>

Where:

• --name: Trigger name.
• --bucket-id: Bucket ID.
• --prefix: Bucket object key prefix. This is an optional parameter. It is used for filtering.
• --suffix: Bucket object key suffix. This is an optional parameter. It is used for filtering.
• --events: Events activating the trigger.

• --batch-size: Event batch size. This is an optional parameter. The values may range from 1 to 10. The default value is 1.
• --batch-cutoff: Maximum wait time. This is an optional parameter. The values may range from 1 to 60 seconds. The default value is 1 second. The trigger groups events for a period not exceeding batch-cutoff and sends them to a container. The number of events cannot exceed batch-size.

• --invoke-container-id: Container ID.
• --invoke-container-service-account-id: ID of the service account with permissions to invoke the container.
• --retry-attempts: Number of invocation retries before the trigger moves a message to the dead-letter queue. This is an optional parameter. The values may range from 1 to 5. The default value is 1.
• --retry-interval: Time to retry invoking the container if the current attempt fails. This is an optional parameter. The values may range from 10 to 60 seconds. The default value is 10 seconds.
• --dlq-queue-id: Dead-letter queue ID. This is an optional parameter.
• --dlq-service-account-id: ID of the service account with write permissions to the dead-letter queue. This is an optional parameter.

Result:

id: a1s5msktijh2********
folder_id: b1gmit33hgh2********
created_at: "2022-10-24T15:19:15.353909857Z"
name: os-trigger
rule:
  object_storage:
    event_type:
    - OBJECT_STORAGE_EVENT_TYPE_CREATE_OBJECT
    - OBJECT_STORAGE_EVENT_TYPE_DELETE_OBJECT
    - OBJECT_STORAGE_EVENT_TYPE_UPDATE_OBJECT
    bucket_id: s3-for-trigger
    prefix: dev
    suffix: 12.jpg
    batch_settings:
      size: "3"
      cutoff: 20s
    invoke_container:
      container_id: bba5jb38o8h2********
      service_account_id: aje3932acdh2********
      retry_settings:
        retry_attempts: "1"
        interval: 10s
      dead_letter_queue:
        queue-id: yrn:yc:ymq:ru-central1:aoek49ghmkh2********:dlq
        service-account-id: aje3932acdh2********
status: ACTIVE

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

Terraform is distributed under the Business Source License. The Yandex Cloud provider for Terraform is distributed under the MPL-2.0 license.

For more information about the provider resources, see the documentation on the Terraform website or mirror website.

If you do not have Terraform yet, install it and configure its Yandex Cloud provider.

To create a trigger for Object Storage:

1. In the configuration file, describe the trigger parameters:

   resource "yandex_function_trigger" "my_trigger" {
     name = "<trigger_name>"
     container {
       id                 = "<container_ID>"
       service_account_id = "<service_account_ID>"
       retry_attempts     = "<number_of_retry_attempts>"
       retry_interval     = "<interval_between_retry_attempts>"
     }
     object_storage {
       bucket_id    = "<bucket_ID>"
       prefix       = "<object_key_prefix>"
       suffix       = "<object_key_suffix>"
       create       = true
       update       = true
       delete       = true
       batch_cutoff = "<maximum_timeout>"
       batch_size   = "<event_group_size>"
     }
     dlq {
       queue_id           = "<dead-letter_queue_ID>"
       service_account_id = "<service_account_ID>"
     }
   }
   

   Where:

   • name: Trigger name. The name format is as follows:

     • It must be from 2 to 63 characters long.
     • It may contain lowercase Latin letters, numbers, and hyphens.
     • It must start with a letter and cannot end with a hyphen.

   • container: Container parameters:

     • id: Container ID.
     • service_account_id: ID of the service account with rights to invoke the container.
     • retry_attempts: Number of invocation retries before the trigger moves a message to the dead letter queue. This is an optional parameter. The values may range from 1 to 5. The default value is 1.
     • retry_intervall: Time to retry invoking the container if the current attempt fails. This is an optional parameter. The values may range from 10 to 60 seconds. The default value is 10 seconds.

   • object_storage: Trigger parameters:

     • bucket_id: Bucket ID.

     • prefix: Bucket object key prefix. This is an optional parameter. It is used for filtering.

     • suffix: Bucket object key suffix. This is an optional parameter. It is used for filtering.

     • Events activating the trigger:

       • create: Trigger will invoke the container if a new object is created in the storage. It can either be true or false.
       • update: Trigger will invoke the container if an object is updated in the storage. It can either be true or false.
       • delete: Trigger will invoke the container if an object is deleted from the storage. It can either be true or false.
     • batch_cutoff: Maximum wait time. This is an optional parameter. The values may range from 1 to 60 seconds. The default value is 1 second. The trigger groups events for a period not exceeding batch_cutoff and sends them to a container. The number of events cannot exceed batch_size.
     • batch_size: Event batch size. This is an optional parameter. The values may range from 1 to 10. The default value is 1.
   • dlq: Dead-letter queue message parameters:
     • queue_id: Dead-letter queue ID. This is an optional parameter.
     • service_account_id: ID of the service account with write permissions to the dead-letter queue. This is an optional parameter.

   For more information about the yandex_function_trigger resource parameters, see the provider documentation.

2. Create resources:

   1. In the terminal, change to the folder where you edited the configuration file.

   2. Make sure the configuration file is correct using the command:

      terraform validate
      

      If the configuration is correct, the following message is returned:

      Success! The configuration is valid.
      

   3. Run the command:

      terraform plan
      

      The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

   4. Apply the configuration changes:

      terraform apply
      

   5. Confirm the changes: type yes in the terminal and press Enter.

   Terraform will create all the required resources. You can check the new resources using the management console or this CLI command:

   yc serverless trigger list
   

To create a trigger for Object Storage, use the create REST API method for the Trigger resource or the TriggerService/Create gRPC API call.