Creating a rule with a Yandex Data Streams target

If you do not have the Yandex Cloud CLI installed yet, install and initialize it.

By default, the CLI uses the folder specified when creating the profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also set a different folder for any specific command using the --folder-name or --folder-id parameter.

1. See the description of the CLI command for creating a rule:

   yc serverless eventrouter rule create --help
   

2. Create a rule with a Yandex Data Streams target:

   yc serverless eventrouter rule create \
     --bus-id <bus_ID> \
     --filter '<jq_template>' \
     --yds-target \
   database=<path_to_database>,\
   stream-name=<data_stream_name>,\
   service-account-id=<stream_service_account>,\
   retry-attempts=<number_of_attempts>,\
   maximum-age=<interval>,\
   transformer=<jq_template>,\
   dlq-arn=<DLQ>,\
   dlq-service-account-id=<DLQ_service_account_ID> \
     --name <rule_name> \
     --description "<rule_description>" \
     --deletion-protection \
     --labels <label_list>
   

   Where:

   • --bus-id: EventRouter bus ID.
   • --filter: jq template for event filtering.

   • --yds-target: Parameter used to configure the Yandex Data Streams target and its properties:

     • database: Path to the Managed Service for YDB database specified in the Data Streams data stream settings. This stream will receive the messages matching the rule.
     • stream-name: Name of the data stream to receive the messages matching the rule.
     • service-account-id: ID of the service account with the yds.writer role or higher for the selected data stream.
     • retry-attempts: Number of message retries before EventRouter moves the messages to the dead-letter queue (DLQ). Valid values range from 0 to 1,000. This is an optional parameter.
     • maximum-age: Time interval before a retry attempt to send the messages if the current attempt fails. Valid values range from 0 to 60 hours. This is an optional parameter.
     • transformer: jq template to transform events. If no template is specified, the event will not be transformed. This is an optional parameter.
     • dlq-arn: ARN of the DLQ to which messages that the target could not process will be redirected. This is an optional parameter.
     • dlq-service-account-id: ID of the service account to which the ymq.writer role or higher is assigned for the selected DLQ. This is an optional parameter.

   • --name: Rule name. Follow these naming requirements:

     • The name must be from 3 to 63 characters long.
     • It may contain uppercase and lowercase Latin and Cyrillic letters, numbers, hyphens, underscores, and spaces.
     • The first character must be a letter. The last character cannot be a hyphen, underscore, or space.

   • --description: Rule description. This is an optional parameter.

   • --deletion-protection: Rule deletion protection. By default, protection is disabled. You cannot delete a rule with this option enabled. To disable deletion protection, specify --no-deletion-protection. This is an optional parameter.

   • --labels: List of labels. This is an optional parameter.

     You can specify one or more labels separated by commas in <key1>=<value1>,<key2>=<value2> format.

   Result:

   id: f66vfpjrkc35********
   bus_id: f66epjc9llqt********
   folder_id: b1g681qpemb4********
   cloud_id: b1gia87mbaom********
   created_at: "2025-02-26T14:04:47.710918Z"
   name: new-rule
   description: created via cli
   labels:
     owner: admin
     version: beta
   filter:
     jq_filter: .firstName == "Ivan"
   targets:
     - yds:
         database: /ru-central1/b1gia87mbaom********/etnvqsesnr5g********
         stream_name: new-stream
         service_account_id: ajelprpohp7r********
       transformer:
         jq_transformer: .
       retry_settings:
         retry_attempts: "3"
         maximum_age: 600s
       dead_letter_queue:
         queue_arn: yrn:yc:ymq:ru-central1:b1g681qpemb4********:dlq-42
         service_account_id: ajelprpohp7r********
       status: ENABLED
   status: ENABLED
   

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

Terraform is distributed under the Business Source License. The Yandex Cloud provider for Terraform is distributed under the MPL-2.0 license.

For more information about the provider resources, see the relevant documentation on the Terraform website or its mirror.

If you do not have Terraform yet, install it and configure the Yandex Cloud provider.

To create a rule with a Yandex Data Streams target:

1. In the configuration file, describe the resources you want to create:

   resource "yandex_serverless_eventrouter_rule" "example_rule" {
     bus_id    = "<bus_ID>"
     jq_filter = "<jq_template>"
   
     yds {
       database           = "<path_to_database>"
       stream_name        = "<data_stream_name>"
       service_account_id = "<service_account_ID>"
     }
   
     name        = "<rule_name>"
     description = "<rule_description>"
   
     labels = {
       <key_1> = "<value_1>"
       <key_2> = "<value_2>"
       ...
       <key_n> = "<value_n>"
     }
   }
   

   Where:

   • bus_id: EventRouter bus ID.

   • jq_filter: jq template for event filtering.

   • yds: Section to configure the Yandex Data Streams target and its parameters:

     • database: Path to the Managed Service for YDB database specified in the Data Streams data stream settings. This stream will receive the messages matching the rule.
     • stream_name: Name of the data stream to receive the messages matching the rule.
     • service_account_id: ID of the service account with the yds.writer role or higher for the selected data stream.

   • name: Rule name. The naming requirements are as follows:

     • The name must be from 3 to 63 characters long.
     • It may contain uppercase and lowercase Latin and Cyrillic letters, numbers, hyphens, underscores, and spaces.
     • The first character must be a letter. The last character cannot be a hyphen, underscore, or space.

   • description: Rule description. This is an optional parameter.

   • labels: List of labels. Provide labels in <key> = "<value>" format. This is an optional parameter.

   For more information about yandex_serverless_eventrouter_rule properties, see this Terraform article.

2. Create the resources:

   1. In the terminal, go to the directory where you edited the configuration file.

   2. Make sure the configuration file is correct using this command:

      terraform validate
      

      If the configuration is correct, you will get this message:

      Success! The configuration is valid.
      

   3. Run this command:

      terraform plan
      

      You will see a detailed list of resources. No changes will be made at this step. If the configuration contains any errors, Terraform will show them.

   4. Apply the changes:

      terraform apply
      

   5. Type yes and press Enter to confirm the changes.

   Terraform will create all the required resources. You can check the new resources in the management console or using this CLI command:

   yc serverless eventrouter rule list
   

To create a rule with a Yandex Data Streams target, use the Create REST API method for the rule resource or the rule/Create gRPC API call.