Creating a trigger for Object Storage that invokes a Serverless Containers container
Create a trigger for Object Storage that invokes a Serverless Containers container when you create, move, or delete an object in a bucket.
Getting started
To create a trigger, you need:
-
A container that the trigger will invoke. If you do not have a container:
-
(Optional) A Dead Letter Queue where messages that could not be processed by a container will be redirected. If you do not have a queue, create one.
-
Service account with rights to invoke the container and (optionally) write messages to the Dead Letter Queue. You can use the same service account or different ones. If you do not have a service account, create one.
- Bucket whose object events will fire the trigger. If you do not have a bucket, create one with restricted access.
Creating a trigger
Note
The trigger is initiated within 5 minutes of being created.
-
In the management console
, select the folder you want to create a trigger in. -
Open Serverless Containers.
-
In the left-hand panel, select
Triggers. -
Click Create trigger.
-
Under Basic settings:
- Enter a name and description for the trigger.
- In the Type field, select
Object Storage
. - In the Launched resource field, select
Container
.
-
Under Object Storage settings:
- In the Bucket field, select the bucket whose object events you want to create a trigger for.
- In the Event types field, select the events that will fire the trigger.
- (Optional) In the Object key prefix field, enter a prefix for filtering.
- (Optional) In the Object key suffix field, enter a suffix for filtering.
-
Under Batch message settings, specify:
- Waiting time, s. The values may range from 1 to 60 seconds. The default value is 1 second.
- Batch size. The values may range from 1 to 100. The default value is 1.
The trigger groups events for a period not exceeding the specified timeout and sends them to a container. The number of events cannot exceed the specified batch size.
-
Under Container settings, select a container and specify a service account to invoke it under.
-
(Optional) Under Repeat request settings:
- In the Interval field, specify the time interval to retry invoking the container if the current attempt fails. The values may range from 10 to 60 seconds. The default value is 10 seconds.
- In the Number of attempts field, specify the number of invocation retries before the trigger moves a message to the Dead Letter Queue. The values may range from 1 to 5. The default value is 1.
-
Optionally, under Dead Letter Queue settings, select the dead-letter queue and the service account with write permissions for this queue.
-
Click Create trigger.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To create a trigger that invokes a container, run this command:
yc serverless trigger create object-storage \
--name <trigger_name> \
--bucket-id <bucket_ID> \
--prefix '<object_key_prefix>' \
--suffix '<object_key_suffix>' \
--events 'create-object','delete-object','update-object' \
--batch-size <event_group_size> \
--batch-cutoff <maximum_timeout> \
--invoke-container-id <container_ID> \
--invoke-container-service-account-id <service_account_ID> \
--retry-attempts <number_of_repeat_attempts> \
--retry-interval <time_between_repeat_attempts> \
--dlq-queue-id <dead_letter_queue_ID> \
--dlq-service-account-id <service_account_ID>
Where:
--name
: Trigger name.--bucket-id
: Bucket ID.--prefix
: Bucket object key prefix. This is an optional parameter. It is used for filtering.--suffix
: Bucket object key suffix. This is an optional parameter. It is used for filtering.--events
: Events activating the trigger.
--batch-size
: Event batch size. This is an optional parameter. The values may range from 1 to 10. The default value is 1.--batch-cutoff
: Maximum wait time. This is an optional parameter. The values may range from 1 to 60 seconds. The default value is 1 second. The trigger groups events for a period not exceedingbatch-cutoff
and sends them to a container. The number of events cannot exceedbatch-size
.
--invoke-container-id
: Container ID.--invoke-container-service-account-id
: ID of the service account with permissions to invoke the container.--retry-attempts
: Number of invocation retries before the trigger moves a message to the dead-letter queue. This is an optional parameter. The values may range from 1 to 5. The default value is 1.--retry-interval
: Time to retry invoking the container if the current attempt fails. This is an optional parameter. The values may range from 10 to 60 seconds. The default value is 10 seconds.--dlq-queue-id
: Dead-letter queue ID. This is an optional parameter.--dlq-service-account-id
: ID of the service account with write permissions to the dead-letter queue. This is an optional parameter.
Result:
id: a1s5msktijh2********
folder_id: b1gmit33hgh2********
created_at: "2022-10-24T15:19:15.353909857Z"
name: os-trigger
rule:
object_storage:
event_type:
- OBJECT_STORAGE_EVENT_TYPE_CREATE_OBJECT
- OBJECT_STORAGE_EVENT_TYPE_DELETE_OBJECT
- OBJECT_STORAGE_EVENT_TYPE_UPDATE_OBJECT
bucket_id: s3-for-trigger
prefix: dev
suffix: 12.jpg
batch_settings:
size: "3"
cutoff: 20s
invoke_container:
container_id: bba5jb38o8h2********
service_account_id: aje3932acdh2********
retry_settings:
retry_attempts: "1"
interval: 10s
dead_letter_queue:
queue-id: yrn:yc:ymq:ru-central1:aoek49ghmkh2********:dlq
service-account-id: aje3932acdh2********
status: ACTIVE
Terraform
For more information about the provider resources, see the documentation on the Terraform
If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.
If you don't have Terraform, install it and configure the Yandex Cloud provider.
To create a trigger for Object Storage:
-
In the configuration file, describe the trigger parameters:
resource "yandex_function_trigger" "my_trigger" { name = "<trigger_name>" container { id = "<container_ID>" service_account_id = "<service_account_ID>" retry_attempts = "<number_of_repeat_attempts>" retry_interval = "<time_between_repeat_attempts>" } object_storage { bucket_id = "<bucket_ID>" prefix = "<object_key_prefix>" suffix = "<object_key_suffix>" create = true update = true delete = true batch_cutoff = "<maximum_timeout>" batch_size = "<event_group_size>" } dlq { queue_id = "<dead_letter_queue_ID>" service_account_id = "<service_account_ID>" } }
Where:
-
name
: Trigger name. The name format is as follows:- The name must be from 3 to 63 characters long.
- It may contain lowercase Latin letters, numbers, and hyphens.
- The first character must be a letter and the last character cannot be a hyphen.
-
container
: Container parameters:id
: Container ID.service_account_id
: ID of the service account with rights to invoke the container.
retry_attempts
: Number of invocation retries before the trigger moves a message to the dead letter queue. This is an optional parameter. The values may range from 1 to 5. The default value is 1.retry_intervall
: Time to retry invoking the container if the current attempt fails. This is an optional parameter. The values may range from 10 to 60 seconds. The default value is 10 seconds.
-
object_storage
: Trigger parameters:-
bucket_id
: Bucket ID. -
prefix
: Bucket object key prefix. This is an optional parameter. It is used for filtering. -
suffix
: Bucket object key suffix. This is an optional parameter. It is used for filtering. -
Events activating the trigger:
create
: Trigger will invoke the container if a new object is created in the storage. It can either betrue
orfalse
.update
: Trigger will invoke the container if an object is updated in the storage. It can either betrue
orfalse
.delete
: Trigger will invoke the container if an object is deleted from the storage. It can either betrue
orfalse
.
batch_cutoff
: Maximum wait time. This is an optional parameter. The values may range from 1 to 60 seconds. The default value is 1 second. The trigger groups events for a period not exceedingbatch_cutoff
and sends them to a container. The number of events cannot exceedbatch_size
.batch_size
: Event batch size. This is an optional parameter. The values may range from 1 to 10. The default value is 1.
-
dlq
: Dead-letter queue message parameters:queue_id
: Dead-letter queue ID. This is an optional parameter.service_account_id
: ID of the service account with write permissions to the dead-letter queue. This is an optional parameter.
For more information about the
yandex_function_trigger
resource properties, see the provider documentation . -
-
Create resources:
-
In the terminal, change to the folder where you edited the configuration file.
-
Make sure the configuration file is correct using the command:
terraform validate
If the configuration is correct, the following message is returned:
Success! The configuration is valid.
-
Run the command:
terraform plan
The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.
-
Apply the configuration changes:
terraform apply
-
Confirm the changes: type
yes
in the terminal and press Enter.
Terraform will create all the required resources. You can check the new resources using the management console
or this CLI command:yc serverless trigger list
-
To create a trigger for Object Storage, use the create REST API method for the Trigger resource or the TriggerService/Create gRPC API call.
Checking the result
Make sure the trigger is working properly. To do this, view container logs that show information about invocations.