Rules for estimating the cost of topic operations

Pricing modesPricing modes

The pricing for the operations with data in YDB topics using YDB request units (RUs) applies to the topics with the on-demand pricing mode. In this mode, the topics are by default created via the YDB CLI/SDK and while creating a CDC stream.

If you create a topic as storage for a Data Streams stream, pricing based on allocated resources applies to it by default. For the topics with pricing for allocated resources, no YDB RUs are charged. You pay for the resource usage on an hourly basis within Yandex Data Streams.

You can change any topic's pricing mode by explicitly running the ydb topic alter YDB CLI command or SDK method.

Calculation of RU usageCalculation of RU usage

For topics with on-demand pricing, YDB calculates the cost of operations in RUs as follows.

TopicAPITopicAPI

The TopicAPI is used when working with the YDB SDK and CLI. It provides methods for streaming data reads and writes. The cost of calling these methods in RUs is calculated as explained below:

1. Each data read or write method call, i.e., opening a data read or write stream, costs 1 RU.

2. Within each streaming data transfer method (session) opened, a certain amount of transferred (written or read) data is accumulated. Once this amount exceeds the established block limits, 1 RU is additionally charged per block. The block sizes for reads and writes are different:

   Type	Block size
   Reads	8 KB
   Writes	4 KB

Calculation example

1. Let’s assume a new streaming write method is called. At that point, 1 RU is charged.
2. Within this method, a 1 KB batch of data to write is transferred. Since the 4 KB write block limit is not exceeded, no additional RUs are charged.
3. Within this method, a 8 KB batch of data to write is transferred. The total amount of data transferred within the method call is now 9 KB, including 2 blocks of 4 KB each. This means 2 RUs are charged, 1 RU per block.
4. Within this method, a 6 KB batch of data to write is transferred. The total amount of data transferred within the method call is now 15 KB, including 3 data blocks of 4 KB each. This costs 3 RUs. As 2 RUs were transferred before, the difference is transferred now: 3 RUs - 2 RUs = 1 RU.

The data reads are calculated in a similar way; the only difference is that the block size is 8 KB.

DataStreamsAPIDataStreamsAPI

DataStreamsAPI is used when accessing a topic through the data streams interface that is compatible with AWS Kinesis. This interface does not support streaming reads and writes. Therefore, to transfer each data block, you need to call an individual unary method (request-response). The cost of calling these methods in RUs is calculated as explained below:

1. Each data read or write method call for transferring or receiving a new data block costs 1 RU.

2. What is calculated here is the amount of data blocks transferred in a request to the write method or received in response to the read method call. The block sizes for reads and writes are different:

   Type	Block size
   Reads	8 KB
   Writes	4 KB

3. 1 RU is charged for each complete data block transferred.

Calculation example

1. Let’s assume the getRecords KinesisAPI method is called.
2. A 20 KB batch of data is received in response. The batch contains two complete read blocks, 8 KB each.
3. The cost of method call in RU is 1 RU per call plus 2 RUs for two complete data blocks received, which equals 3 RUs.

KafkaAPIKafkaAPI

The KafkaAPI does not support streaming reads and writes. To transfer each data block, a separate unary method (request-response) has to be called. The cost of calling these methods in RUs is calculated as explained below:

1. Each data read or write method call to transfer or receive the next data block costs 1 RU (effective as of July 1, 2024).

2. What is calculated here is the amount of data blocks transferred in a request to the write method or received in response to the read method call. The block sizes for read and write operations are different:

   Type	Block size
   Reads	8 KB
   Writes	4 KB

3. 1 RU is charged for each complete data block transferred.

Calculation example

1. Let’s assume the FETCH KafkaAPI method is called.
2. A 20 KB batch of data is received in response. The batch contains two complete read blocks, 8 KB each.
3. The cost of method call in RU is 1 RU per call plus 2 RUs for two complete data blocks received, which equals 3 RUs.

The data writes are calculated in a similar way; the only difference is that the block size is 4 KB.