OLTP or OLAP operating modes
When working with databases, there are two main scenarios: On-line Transaction Processing (OLTP) and On-line Analytical Processing (OLAP).
OLTP is focused on quick execution of a large stream of transactions, each of which processes a relatively small amount of data. OLAP is mainly designed to analyze data with a lower number of transactions relative to OLTP, but with a much higher volume of processed data.
Analytics β
Yandex Managed Service for YDB Serverless OLTP capabilities are utilized in the production phase by large numbers of users. The OLAP scenario's capabilities are currently at the Preview stage. To reduce the mutual influence of workload types, a separate database type for OLAP workloads is deployed called Analytics β.
Analytics β databases are fully managed (serverless) YDB databases designed for analytical data processing. With their help, you can conveniently address periodic data analysis tasks, build data marts, or perform any other types of analysis.
Note
There is currently no charge for using Analytics β databases. You only pay for using data storage space based on Yandex Managed Service for YDB Serverless pricing plans.
Yandex Managed Service for YDB dedicated databases
OLAP data processing is available at Preview stage in Yandex Managed Service for YDB dedicated databases.
Warning
When operating within the same database, OLAP and OLTP workloads share common database resources. Yet the database resource utilization profiles for OLAP and OLTP workloads differ considerably. It is not recommended to combine production workloads of different types within the same database because of the mutual influence of the workload profiles.
Column-oriented tables
It is recommended to store the data used in OLAP workloads in column tables.
YDB column tables store the data of each column separately (independently) from other columns. This data storage principle is optimized for use in OLAP scenarios, as oftentimes only a few columns of source data are used to build analytical reports, and columnar storage allows reading from disks only the columns directly involved in the request. Another benefit of this approach lies in the high degree of data compression, since columns often store repetitive or similar data. The downside is that operations on full rows become costlier due to the need to reconstruct them on the fly from individual cells.
The current basic use case of YDB column tables is writing data with increasing primary key (e.g., event time), analyzing such data, and deleting obsolete data based on TTL. The optimal way to add data to YDB column tables is batch writing in blocks of several megabytes.
Note
Currently, managing column table data through INSERT INTO, UPSERT, UPDATE, REPLACE, DELETE SQL constructions is not supported. To record data, use data delivery from external systems (ydb import, Data Transfer, Fluent Bit, Logstash), or the BulkUpsert SDK API method. To delete data, use TTL.