Creating a connector for Data Streams

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder specified when creating the CLI profile is used by default. To change the default folder, use the yc config set folder-id <folder_ID> command. You can specify a different folder using the --folder-name or --folder-id parameter.

1. See the description of the CLI command for creating a connector:

   yc serverless eventrouter connector create data-stream --help
   

2. Create a connector for Data Streams:

   yc serverless eventrouter connector create data-stream \
     --bus-id <bus_ID> \
     --database <DB_path>
     --stream-name <stream_name>
     --consumer <consumer_name>
     --service-account-id <service_account_ID> \
     --name <connector_name> \
     --description "<connector_description>" \
     --labels <label_list> \
     --deletion-protection
   

   Where:

   • --bus-id: EventRouter bus ID.

   • --database: Path to the Yandex Managed Service for YDB database specified in the data stream settings, e.g., /ru-central1/b1gia87mbaom********/etnudu2n9ri3********.

   • --stream-name: Stream name.

   • --consumer: Name of the dedicated data consumer.

   • --service-account-id: ID of the service account that has read permissions for the data stream.

   • --name: Connector name. Follow these naming requirements:

     • The name must be from 3 to 63 characters long.
     • It may contain uppercase and lowercase Latin and Cyrillic letters, numbers, hyphens, underscores, and spaces.
     • The first character must be a letter. The last character cannot be a hyphen, underscore, or space.

   • --description: Connector description. This is an optional parameter.

   • --labels: List of labels. This is an optional parameter.

     You can specify one or more labels separated by commas in <key1>=<value1>,<key2>=<value2> format.

   • --deletion-protection: Connector deletion protection. By default, protection is disabled. You cannot delete a connector with this option enabled. To disable deletion protection, specify --no-deletion-protection. This is an optional parameter.

   Result:

   id: f66g6jcnc5no********
   bus_id: f66qn4p7uk6p********
   folder_id: b1g681qpemb4********
   cloud_id: b1gia87mbaom********
   created_at: "2025-02-21T10:53:25.994124Z"
   name: yds-via-cli
   description: created via cli
   labels:
     interface: yccli
     owner: admin
   source:
     data_stream:
       database: /ru-central1/b1gia87mbaom********/etntd0p5tauu********
       stream_name: my-ydb-flow
       consumer: consumer-one
       service_account_id: ajelprpohp7r********
   deletion_protection: true
   status: RUNNING
   

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

Terraform is distributed under the Business Source License. The Yandex Cloud provider for Terraform is distributed under the MPL-2.0 license.

For more information about the provider resources, see the documentation on the Terraform website or mirror website.

If you do not have Terraform yet, install it and configure its Yandex Cloud provider.

To create a connector for Data Streams:

1. In the configuration file, define the parameters of the resources you want to create:

   resource "yandex_serverless_eventrouter_connector" "example_connector" {
     bus_id              = "<bus_ID>"
     name                = "<connector_name>"
     description         = "<connector_description>"
     deletion_protection = <true|false>
   
     labels = {
       <key_1> = "<value_1>"
       <key_2> = "<value_2>"
       ...
       <key_n> = "<value_n>"
     }
   
     ymq {
       database           = "<DB_path>"
       stream_name        = "<stream_name>"
       consumer           = "<consumer_name>"
       service_account_id = "<service_account_ID>"
     }
   }
   

   Where:

   • bus_id: EventRouter bus ID.

   • name: Connector name. Follow these naming requirements:

     • The name must be from 3 to 63 characters long.
     • It may contain uppercase and lowercase Latin and Cyrillic letters, numbers, hyphens, underscores, and spaces.
     • The first character must be a letter. The last character cannot be a hyphen, underscore, or space.

   • description: Connector description. This is an optional parameter.

   • deletion_protection: Connector deletion protection, true or false. You cannot delete a connector with this option enabled. This is an optional parameter.

   • labels: List of labels. Provide labels in <key> = "<value>" format. This is an optional parameter.

   • database: Path to the Yandex Managed Service for YDB database specified in the data stream settings, e.g., /ru-central1/b1gia87mbaom********/etnudu2n9ri3********.

   • stream_name: Stream name.

   • consumer: Name of the dedicated data consumer.

   • service_account_id: ID of the service account that has read permissions for the data stream.

   For more information about the yandex_serverless_eventrouter_connector resource parameters, see this Terraform article.

2. Create the resources:

   1. In the terminal, change to the folder where you edited the configuration file.

   2. Make sure the configuration file is correct using the command:

      terraform validate
      

      If the configuration is correct, the following message is returned:

      Success! The configuration is valid.
      

   3. Run the command:

      terraform plan
      

      The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

   4. Apply the configuration changes:

      terraform apply
      

   5. Confirm the changes: type yes in the terminal and press Enter.

   Terraform will create all the required resources. You can check the new resources using the management console or this CLI command:

   yc serverless eventrouter connector list
   

To create a connector for Data Streams, use the Create REST API method for the connector resource or the connector/Create gRPC API call.