Creating a trigger for Object Storage that invokes Cloud Functions
Create a trigger for Object Storage that invokes Cloud Functions when you create, move, or delete a bucket object.
Getting started
To create a trigger, you will need:
-
Function the trigger will invoke. If you do not have a function:
-
Optionally, a dead-letter queue where to redirect the messages the function failed to process. If you do not have a queue, create one.
-
Service accounts with permissions to invoke the function and, optionally, write to the dead letter queue. You can use the same service account or different ones. If you do not have a service account, create one.
- Bucket whose object events will set off the trigger. If you do not have a bucket, create one with restricted access.
Creating a trigger
Note
The trigger is initiated within five minutes after it is created.
-
In the management console
, select the folder where you want to create a trigger. -
Go to Cloud Functions.
-
In the left-hand panel, select
Triggers. -
Click Create trigger.
-
Under Basic settings:
- Enter a name and description for the trigger.
- In the Type field, select Object Storage.
- In the Launched resource field, select Function.
-
Under Object Storage settings:
- In the Bucket field, select the bucket whose object events you want to create a trigger for.
- In the Event types field, select events that will set off the trigger.
- Optionally, in the Object key prefix field, enter a prefix for filtering.
- Optionally, in the Object key suffix field, enter a suffix for filtering.
-
Under Batch message settings, specify:
- Waiting time, s. The values may range from 1 to 60 seconds. The default value is 1 second.
- Batch size. The values may range from 1 to 100. The default value is 1.
The trigger groups events within the specified wait time period and sends them to the function. The number of events cannot exceed the specified batch size.
-
Under Function settings, select a function and specify:
- Function version tag.
- Service account to use for invoking the function.
-
Optionally, under Repeat request settings:
- In the Interval field, specify how long to wait before retrying the function if it fails. The values may range from 10 to 60 seconds. The default value is 10 seconds.
- In the Number of attempts field, specify the number of function retries before the trigger sends a message to the dead letter queue. The values may range from 1 to 5. The default value is 1.
-
Optionally, under Dead Letter Queue settings, select a dead-letter queue and a service account with write permissions for that queue.
-
Click Create trigger.
If you do not have the Yandex Cloud CLI yet, install and initialize it.
The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.
To create a trigger that invokes a function, run this command:
yc serverless trigger create object-storage \
--name <trigger_name> \
--bucket-id <bucket_ID> \
--prefix '<object_key_prefix>' \
--suffix '<object_key_suffix>' \
--events 'create-object','delete-object','update-object' \
--batch-size <event_batch_size> \
--batch-cutoff <maximum_wait_time> \
--invoke-function-id <function_ID> \
--invoke-function-service-account-id <service_account_ID> \
--retry-attempts <number_of_retry_attempts> \
--retry-interval <interval_between_retry_attempts> \
--dlq-queue-id <dead-letter_queue_ID> \
--dlq-service-account-id <service_account_ID>
Where:
--name: Trigger name.--bucket-id: Bucket ID.--prefix: Bucket object key prefix. This is an optional setting used for filtering.--suffix: Bucket object key suffix. This is an optional setting used for filtering.--events: Events that set off the trigger.
--batch-size: Event batch size. This is an optional setting. The values may range from 1 to 10. The default value is 1.--batch-cutoff: Maximum wait time. This is an optional setting. The values may range from 1 to 60 seconds. The default value is 1 second. The trigger groups events within thebatch-cutoffperiod and sends them to the function. The number of events cannot exceedbatch-size.
--invoke-function-id: Function ID.--invoke-function-service-account-id: ID of the service account with permissions to invoke the function.--retry-attempts: Number of invocation retries before the trigger sends a message to the dead-letter queue. This is an optional setting. The values may range from 1 to 5. The default value is 1.--retry-interval: Time to wait before retrying the function if it fails. This is an optional setting. The values may range from 10 to 60 seconds. The default value is 10 seconds.--dlq-queue-id: Dead-letter queue ID. This is an optional setting.--dlq-service-account-id: ID of the service account with write permissions for the dead-letter queue. This is an optional setting.
Result:
id: a1s92agr8mpg********
folder_id: b1g88tflru0e********
created_at: "2019-12-18T09:47:50.079103Z"
name: os-trigger
rule:
object_storage:
event_type:
- OBJECT_STORAGE_EVENT_TYPE_CREATE_OBJECT
- OBJECT_STORAGE_EVENT_TYPE_DELETE_OBJECT
- OBJECT_STORAGE_EVENT_TYPE_UPDATE_OBJECT
bucket_id: s3-for-trigger
prefix: dev
suffix: 12.jpg
batch_settings:
size: "3"
cutoff: 20s
invoke_function:
function_id: d4eofc7n0m03********
function_tag: $latest
service_account_id: aje3932acd0c********
retry_settings:
retry_attempts: "1"
interval: 10s
dead_letter_queue:
queue-id: yrn:yc:ymq:ru-central1:aoek49ghmknn********:dlq
service-account-id: aje3932acd0c********
status: ACTIVE
With Terraform
Terraform is distributed under the Business Source License
For more information about the provider resources, see the relevant documentation on the Terraform
If you do not have Terraform yet, install it and configure the Yandex Cloud provider.
To create a trigger for Object Storage:
-
In the Terraform configuration file, describe the resources you want to create:
resource "yandex_function_trigger" "my_trigger" { name = "<trigger_name>" description = "<trigger_description>" function { id = "<function_ID>" service_account_id = "<service_account_ID>" retry_attempts = "<number_of_retry_attempts>" retry_interval = "<interval_between_retry_attempts>" } object_storage { bucket_id = "<bucket_ID>" prefix = "<object_key_prefix>" suffix = "<object_key_suffix>" create = true update = true delete = true batch_cutoff = "<maximum_wait_time>" batch_size = "<event_batch_size>" } dlq { queue_id = "<dead-letter_queue_ID>" service_account_id = "<service_account_ID>" } }Where:
-
name: Trigger name. Follow these naming requirements:- Length: between 3 and 63 characters.
- It can only contain lowercase Latin letters, numbers, and hyphens.
- It must start with a letter and cannot end with a hyphen.
-
description: Trigger description. -
function: Function settings:id: Function ID.service_account_id: ID of the service account with permissions to invoke the function.retry_attempts: Number of invocation retries before the trigger sends a message to the dead-letter queue. This is an optional setting. The values may range from 1 to 5. The default value is 1.retry_interval: Time to wait before retrying the function if it fails. This is an optional setting. The values may range from 10 to 60 seconds. The default value is 10 seconds.
-
object_storage: Trigger settings:-
bucket_id: Bucket ID. -
prefix: Bucket object key prefix. This is an optional setting used for filtering. -
suffix: Bucket object key suffix. This is an optional setting used for filtering. -
Events that set off the trigger:
create: Trigger will invoke the function when a new object is created in the storage. It can either betrueorfalse.update: Trigger will invoke the function when an object is updated in the storage. It can either betrueorfalse.delete: Trigger will invoke the function when an object is deleted from the storage. It can either betrueorfalse.
batch_cutoff: Maximum wait time. This is an optional setting. The values may range from 1 to 60 seconds. The default value is 1 second. The trigger groups events within thebatch-cutoffperiod and sends them to the function. The number of events cannot exceedbatch-size.batch_size: Event batch size. This is an optional setting. The values may range from 1 to 10. The default value is 1.
-
dlq: Dead-letter queue settings:queue_id: Dead-letter queue ID.service_account_id: ID of the service account with write permissions for the dead-letter queue.
For more information about
yandex_function_triggerproperties, see this provider guide. -
-
Create the resources:
-
In the terminal, navigate to the configuration file directory.
-
Make sure the configuration is correct using this command:
terraform validateIf the configuration is valid, you will get this message:
Success! The configuration is valid. -
Run this command:
terraform planYou will see a list of resources and their properties. No changes will be made at this step. Terraform will show any errors in the configuration.
-
Apply the configuration changes:
terraform apply -
Type
yesand press Enter to confirm the changes.
Terraform will create all the required resources. You can check the new resources using the management console
or this CLI command:yc serverless trigger list -
To create a trigger for Object Storage, use the create REST API method for the Trigger resource or the TriggerService/Create gRPC API call.
Checking the result
Check that the trigger works correctly. To do this, view function logs that show information on invocations.