Contact UsGet started
© 2025 Direct Cursus Technology L.L.C.

Streaming Yandex Cloud Postbox events to Yandex Data Streams and analyzing them with Yandex DataLens

Written by
Updated at November 24, 2025

In this tutorial, you will set up streaming of Yandex Cloud Postbox events to Yandex Data Streams and their visualization in Yandex DataLens for further analysis. Sending emails generates events.

You can implement this solution using the Yandex Cloud management console even if you have no coding skills. To process and store data, you will use Yandex Cloud Functions and Yandex Managed Service for YDB.

To set up event streaming and visualization:

  1. Get your cloud ready.
  2. Create service accounts.
  3. Create a static access key.
  4. Set up a Managed Service for YDB database.
  5. Create a data stream in Data Streams.
  6. Set up Yandex Cloud Postbox resources.
  7. Set up Cloud Functions resources.
  8. Send emails.
  9. Configure visualization in DataLens.

If you no longer need the resources you created, delete them.

Get your cloud ready

Sign up for Yandex Cloud and create a billing account:

  1. Navigate to the management console and log in to Yandex Cloud or create a new account.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure.

Learn more about clouds and folders here.

The infrastructure support costs include:

Create service accounts

Create two service accounts:

  • yds-functions to call Cloud Functions and write data to the YDB database.
  • postbox-user to send emails via Yandex Cloud Postbox.

  1. Create a yds-functions service account:

    1. In the management console, select the folder where you are going to create your infrastructure.
    2. In the list of services, select Identity and Access Management and click Create service account.
    3. In the Name field, enter yds-functions for name.
    4. Click Add role and select the yds.editor and functions.functionInvoker roles.
    5. Click Create.

  2. Similarly, create a service account named postbox-user and assign the postbox.sender role to it.

Create a static access key

Create a static access key to use for sending emails:

  1. In the management console, select the folder where you are deploying your infrastructure.

  2. In the list of services, select Identity and Access Management and then select postbox-user from the list of service acounts. In the window that opens, do the following:

    1. In the top panel, click Create new key and select Create static access key.

    2. Click Create.

    3. Save the ID and secret key.

      Alert

      After you close this dialog, the key value will no longer be available.

Set up a Managed Service for YDB database

You can use any DBMS suitable for analytical tasks as event storage. In this tutorial, we use a Managed Service for YDB serverless database. For other DBMS, the SQL query for creating a table and code for loading events into that table may be different.

Create a database

  1. In the management console, select the folder where you are deploying your infrastructure.
  2. In the list of services, select Managed Service for YDB and click Create a database.
  3. In the Name field, specify postbox-events-ydb.
  4. In the Database type field, select Serverless.
  5. Do not edit the other database settings. Click Create a database.

Wait for the database to start. While being created, your database will have the Provisioning status. Once it is ready for use, its status will change to Running.

Create a table

  1. On the Databases page, select the new postbox-events-ydb DB.

  2. To open the DB root directory, navigate to the Navigation tab.

  3. To create a DB query, click New SQL query in the top-right corner.

  4. In the Query box that opens, enter the following:

    CREATE TABLE postbox_events
(
    saved_datetime Datetime NOT NULL,
    eventid String NOT NULL,
    eventtype String,
    mail_timestamp Timestamp,
    mail_messageid String,
    mail_ch_from String,
    mail_ch_to String, 
    mail_ch_messageid String,
    mail_ch_subject String,
    delivery_timestamp Timestamp,
    delivery_time_ms Uint64,
    delivery_recipients String,
    bounce_bounceType String,
    bounce_bounceSubType String,
    bounce_bouncedRecipients String,
    bounce_timestamp Timestamp,
    -- message Json,
    PRIMARY KEY (saved_datetime, eventid)
)

  5. Click Run.

    After you run the query, a new postbox_events table will appear in your database.

Create a data stream in Data Streams

Create a data stream to log Yandex Cloud Postbox events:

  1. In the management console, select the folder where you are deploying your infrastructure.
  2. In the list of services, select Data Streams and click Create stream.
  3. In the Database field, select the postbox-events-ydb database you created earlier.
  4. In the Name field, enter the stream name: postbox-events-stream.
  5. In the Pricing Mode field, select Actual usage.
  6. Do not edit the other data stream settings. Click Create.

Wait for the stream to start. Once the stream is ready for use, its status will change from Creating to Active.

Create Yandex Cloud Postbox resources

In Yandex Cloud Postbox, configure event logging to Data Streams, create a sender address, and verify your domain ownership.

Create a Yandex Cloud Postbox configuration

  1. In the management console, select the folder where you are deploying your infrastructure.

  2. In the list of services, select Cloud Postbox.

  3. In the left-hand panel, select Configurations and click Create configuration.

  4. In the Name field, specify the configuration name: postbox-events-config.

  5. Under Subscriptions, click Add and fill out the form that opens:

    1. In the Name field, specify the subscription name: postbox-events-subscribe.
    2. In the Data stream field, select the data stream named postbox-events-stream.
    3. Toggle the Enabled option on to activate the subscription.

  6. Do not edit the other configuration settings. Click Create configuration.

Create a Yandex Cloud Postbox address

  1. Generate the privatekey.pem key to create a DKIM signature by running this command in the terminal:

    openssl genrsa -out privatekey.pem 2048

    Note

    openssl comes preinstalled in Linux and macOS. If using Windows, you need to install it manually. For more information, visit the project's website.

  2. Create a Yandex Cloud Postbox address:

    1. In the management console, select the folder where you are deploying your infrastructure.
    2. In the list of services, select Cloud Postbox.
    3. In the left-hand panel, select Addresses and click Create address.
    4. In the Domain field, specify the domain you will use to send emails. You can use a domain of any level, as long as you own it.
    5. In the Selector field, specify a selector, e.g., postbox. Make sure to only use this selector in the resource record you create when verifying domain ownership.
    6. In the Configuration field, select postbox-events-config you previously created.
    7. In the Private key field, paste the contents of the privatekey.pem file from the previous step.
    8. Do not change the other address settings. Click Create address.
    9. Click the newly created address in the list that opens.
    10. Under Signature verification on the address info page, copy and save the TXT record from the Value field.

Pass a domain ownership check

To send emails, confirm domain ownership. After creating an address, DKIM signature settings will be generated on its page. Specify them as the values of the resource record you need to add to your domain zone. You can add a record with your registrar or in Yandex Cloud DNS if you have delegated your Yandex Cloud domain.

Example of creating a resource record in Yandex Cloud DNS

  1. In the management console, select the folder containing the address and your domain zone.

    If you do not have a public DNS zone yet, create one:

  2. Select Cloud DNS.

  3. Select your domain zone.

  4. Click Create record.

  5. In the Name field, specify the name portion generated when creating the address, omitting the domain in <selector>._domainkey format, e.g., postbox._domainkey.

    Note

    For other DNS services, you may need to copy the entire record. The final record must look like this: <selector>._domainkey.<domain>., e.g., postbox._domainkey.example.com..

  6. In the Type field, select TXT.

  7. Copy the contents of the Value field from the Signature verification section and paste it into the Data field. Note that the record value must be enclosed in quotes, for example:

    "v=DKIM1;h=sha256;k=rsa;p=M1B...aCA8"

  8. Click Create.

  9. Go to Cloud Postbox.

  10. Select the address you created.

  11. Click Verify address. If the record is correct, the verification status on the address page will change to Success.

DNS server responses are cached, so delays may occur when updating a resource record.

Create Cloud Functions resources

Create a function to send data from the stream to the DB, and a trigger to invoke the function when events are logged in the data stream.

Get the function code

Download the postbox-events.zip archive to your computer. The archive contains the index.py and requirements.txt files with the function code.

On your computer, do the following:

  1. Clone the repository with the function code:

    git clone https://github.com/yandex-cloud-examples/yc-postbox-events/blob/main/build/postbox-events.git

    Navigate to the repository directory. Make sure it contains the index.py and requirements.txt files with the function code.

  2. Create an archive named postbox-events.zip and add index.py and requirements.txt to it.

Get the DB connection details

  1. In the management console, select the folder where you are deploying your infrastructure.

  2. In the list of services, select Managed Service for YDB.

  3. Select the DB for which you need to get the endpoint and path.

    • The DB endpoint is specified under Connection in the first part of the Endpoint field value (preceding /?database=):

      For example, the endpoint of a serverless DB is grpcs://ydb.serverless.yandexcloud.net:2135.

    • The DB path is specified under Connection in the second part of the Endpoint field value (following /?database=).

      Here is an example of a DB path: /ru-central1/b1gia87mbaomkfvs6rgl/etnudu2n9ri35luqe4h1.

Create a function

To create a function, you will need the function code and DB connection details.

  1. In the management console, select the folder where you are deploying your infrastructure.

  2. In the list of services, select Cloud Functions.

  3. Create a function:

    1. Click Create function.
    2. In the Name field, enter a name for the function, e.g., postbox-events-function.
    3. Click Create.

  4. Create a function version:

    1. In the Editor window that opens, select Python 3.12.

    2. Disable Add files with code examples.

    3. Click Continue.

    4. In the Method field, select ZIP archive.

    5. Click Attach file and select postbox-events.zip.

    6. In the Entry point field, specify index.handler.

    7. Under Parameters, specify:

      • Timeout: 10 seconds.

      • Memory: 128 MB.

      • Service account: yds-functions.

      • Environment variables:

        Key Description Value (example)
        YDB_DATABASE DB path /ru-central1/b1go123e9vjq********/etnu15kr22********
        YDB_ENDPOINT DB endpoint grpcs://ydb.serverless.yandexcloud.net:2135
        YDB_TABLE Table name postbox_events

    8. Click Save changes.

Create a trigger

  1. In the management console, select the folder where you are deploying your infrastructure.
  2. In the list of services, select Cloud Functions.
  3. In the left-hand panel, select Triggers and click Create trigger.
  4. In the Name field, enter the trigger name: postbox-events-trigger.
  5. In the Type field, select Data Streams.
  6. In the Launched resource field, select Function.
  7. Under Data Streams settings, select the data stream named postbox-events-stream and the yds-functions service account.
  8. Under Function settings, select the postbox-events-function function and the yds-functions service account.
  9. Click Create trigger.

Send emails

  1. Use the Yandex Cloud Postbox address and static access key you created earlier to send several test emails in any way you prefer.

  2. Make sure the emails were delivered by checking the mailbox you sent them to.

  3. Make sure the data about sent emails reaches the services:

    • Check the data stream:

      1. In the management console, select the folder where you are deploying your infrastructure.

      2. In the list of services, select Data Streams.

      3. Select the data stream named postbox-events-stream.

      4. Navigate to the Data viewer tab.

        The charts should show email sending events.

    • Check the DB:

      1. In the management console, select the folder where you are deploying your infrastructure.

      2. In the list of services, select Managed Service for YDB.

      3. Select the postbox-events-ydb database.

      4. Go to the Navigation tab.

      5. Select the postbox_events table.

        The table should show new records. Some columns may have the NULL value: this depends on the type of notification received from Yandex Cloud Postbox.

Configure visualization in DataLens

To monitor the emails you send, set up a connection, create a dataset, charts, and a dashboard.

Create a connection

  1. Go to the DataLens home page.

  2. In the left-hand panel, select Connections and click Create connection.

  3. Select the YDB connector.

  4. Configure the connection as follows:

    • Cloud and folder: Select the folder you are using to complete this tutorial.
    • Service account: yds-functions.
    • Database: postbox-events-ydb.
    • Cache TTL in seconds: Default.
    • Enable the SQL query access level option and select Allow subqueries in datasets.

  5. Click Create connection.

  6. In the window that opens, select a workbook for your new connection and click Create.

  7. Enter a connection name, e.g., postbox-events-connection, and click Create.

Create a dataset

  1. Go to the DataLens home page.
  2. In the left-hand panel, select Datasets and click Create dataset.
  3. In the window that opens, select the workbook your new connection is in and click Create.
  4. Under Connections, click Add and select the postbox-events-connection connection you created earlier.
  5. Under Tables, select the postbox-events table and drag it to the right.
  6. Click Save.
  7. In the window that opens, specify postbox-events-dataset and click Create.

Create charts

In this tutorial, we will create the Events by day and Event list charts as an example. You can create other charts in the same way.

Events by day

  1. Go to the DataLens home page.

  2. In the left-hand panel, select Charts. Click Create chart and select Chart in Wizard.

  3. In the window that opens, select the workbook your new connection is in and click Create.

  4. In the left-hand section, click Select dataset and select postbox-events-dataset.

  5. Select Column chart as the chart type.

  6. Drag the dimensions as follows:

    • delivery_timestamp to the X section: Dates will be plotted along the X axis.
    • eventid to the Y section: Number of events will be plotted on the Y axis.
    • eventtype to the Colors section: This enables using a different color for each event type.

  7. Click Save.

  8. In the window that opens, specify the chart name, Events by day, and click Save.

Event list

  1. Go to the DataLens home page.

  2. In the left-hand panel, select Charts. Click Create chart and select Chart in Wizard.

  3. In the window that opens, select the workbook your new connection is in and click Create.

  4. In the left-hand section, click Select dataset and select postbox-events-dataset.

  5. Select Table as the chart type.

  6. Drag the following dimensions to the Columns section:

    • saved_datetime: Date and time of saving the event.
    • eventid: Event type.
    • mail_ch_to: Recipient's email address.
    • delivery_timestamp: Email delivery date and time.
    • delivery_time_ms: Delivery time in milliseconds.
    • mail_ch_subject: Email subject.
    • bounce_bounceType: Delivery error type.
    • bounce_bounceSubType: Non-delivery report.

  7. Drag the delivery_timestamp dimension to the Sorting section to sort the table by delivery time.

  8. Click Save.

  9. In the window that opens, specify the chart name, Event list, and click Save.

Create a dashboard

  1. Go to the DataLens home page.

  2. In the left-hand panel, select Dashboards and click Create dashboard.

  3. In the window that opens, select the workbook your new connection is in and click Create.

  4. Add Events by day, Event list, and other charts you created earlier to your dashboard:

    1. In the bottom panel, click Chart.
    2. In the window that opens, click Select and select the Events by day chart.
    3. Click Add.
    4. Repeat these steps to add to the dashboard the Event list chart and other charts as needed.

  5. Use your mouse to resize and move the charts around.

  6. Click Save.

  7. In the window that opens, name the dashboard: postbox-events-dashboard.

  8. Click Create.

After completing this tutorial, you can use charts and tables to analyze events.

How to delete the resources you created

To stop paying for the resources you created:

  1. Delete the trigger invoking the function in Cloud Functions.
  2. Delete the function in Cloud Functions.
  3. Delete the Managed Service for YDB database.
  4. Delete the stream in Data Streams.
  5. Delete the Yandex Cloud Postbox address and configuration.
  6. Optionally, delete the service accounts.
  7. Optionally, delete the resource record and public DNS zone.
Previous
Terraform
Next
Building an interactive serverless application using WebSocket