Troubleshooting slow writes in Yandex Data Streams

If you notice a decrease in write speed in Yandex Data Streams, the most common cause is reaching the write limit.

There are two main limits that can slow down writes:

• Maximum write speed per partition; the default is 1 MB/sec.
• Maximum storage space used per topic; depends on YDB database limits.

Partition write limitPartition write limit

DiagnosticsDiagnostics

If the chart shows that the write quota usage is approaching 100%, this must be what limits your data stream: the system intentionally slows writes down to stay within the allowed maximum.

ResolutionResolution

You cannot increase the write limit for one partition. If you have reached this limit, you can increase the number of partitions in the topic. As a result, data will be written concurrently to multiple partitions, thereby increasing overall throughput.

To increase the number of partitions, edit the data stream settings according to this guide.

Limit on storage space used in the YDB databaseLimit on storage space used in the YDB database

DiagnosticsDiagnostics

If the chart shows that the storage space is close to 100% full, write requests most likely fail due to lack of storage space.

ResolutionResolution

To free up space and re-enable writes, do one of the following:

• Manually delete the data you no longer need from the YDB database.
• Reduce the message retention period for your data stream. This will automatically delete old data.
• Increase the storage size limit in YDB to expand the space available for writes. Follow these steps:
  1. In the management console, select the folder where you want to change the stream settings.
  2. Select Data Streams.
  3. Select the data stream you want to increase the database storage limit for.
  4. Navigate to the data stream's Overview page.
  5. Under Access, click the link in the Database field.
  6. Increase the storage limit according to this guide.