Batch processing

Batch processing is a technology for processing data that involves preparing aggregated information based on large arrays of data. This type of data analysis is traditional and is used for processing data stored, for example, in a DBMS.

Data volumes increase with time, but that doesn't mean all data is accessed often. So, rarely used data is usually transferred to storage systems like Yandex Object Storage which are much more cost-effective than DBMS.

Data is stored in Yandex Object Storage as a file structure with directories and files. To store data in files, standard storage formats like CSV and JSON are usually used.

Yandex Query lets you access data stored in Yandex Object Storage in the same way as data stored in a DBMS by making queries in an SQL-like language called YQL.

Yandex Object Storage is usually used to manage large amounts of data. Yandex Query analyzes the volume of data to handle and runs dozens to hundreds of concurrent data processing jobs in a computing cluster. This ensures a high processing rate even if data volumes are large.