Building a data platform based on Yandex Cloud
Written by
Updated at January 19, 2026
Apache Kafka®
- Unassisted deployment of the Apache Kafka® web interface
- Upgrading a Managed Service for Apache Kafka® cluster to migrate from ZooKeeper to KRaft
- Migrating a database from a third-party Apache Kafka® cluster to Yandex Managed Service for Apache Kafka®
- Moving data between Managed Service for Apache Kafka® clusters using Data Transfer
- Delivering data from Yandex Managed Service for MySQL® to Yandex Managed Service for Apache Kafka® using Yandex Data Transfer
- Delivering data from Yandex Managed Service for MySQL® to Yandex Managed Service for Apache Kafka® using Debezium
- Delivering data from Yandex Managed Service for PostgreSQL to Yandex Managed Service for Apache Kafka® using Yandex Data Transfer
- Delivering data from Yandex Managed Service for PostgreSQL to Yandex Managed Service for Apache Kafka® using Debezium
- Delivering data from Yandex Managed Service for YDB to Yandex Managed Service for Apache Kafka® using Yandex Data Transfer
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for ClickHouse® using Yandex Data Transfer
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex MPP Analytics for PostgreSQL using Yandex Data Transfer
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex StoreDoc using Yandex Data Transfer
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for MySQL® using Yandex Data Transfer
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for OpenSearch using Yandex Data Transfer
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for PostgreSQL using Yandex Data Transfer
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for YDB using Yandex Data Transfer
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Data Streams using Yandex Data Transfer
- Delivering data from a Data Streams queue to Managed Service for Apache Kafka® using Yandex Data Transfer
- Configuring Kafka Connect to work with a Yandex Managed Service for Apache Kafka® cluster
- Syncing data from Apache Kafka® topics to an Object Storage bucket without using the internet
- Using a schema registry with Yandex Managed Service for Apache Kafka®:
- Managing data schemas in Managed Service for Apache Kafka®
- Using Managed Schema Registry with Yandex Managed Service for Apache Kafka®
- Using Managed Schema Registry with Yandex Managed Service for Apache Kafka® through the REST API
- Using Confluent Schema Registry with Yandex Managed Service for Apache Kafka®
- Monitoring message loss in an Apache Kafka® topic
Apache Airflow™
- Automating Yandex Query tasks with Yandex Managed Service for Apache Airflow™
- Sending requests to the Yandex Cloud API via the Yandex Cloud Python SDK
- Configuring an SMTP server to send e-mail notifications
- Running a PySpark job using Yandex Managed Service for Apache Airflow™
ClickHouse®
- Adding data to ClickHouse®
- Migrating data to Managed Service for ClickHouse® using ClickHouse®
- Migrating data to Yandex Managed Service for ClickHouse® using Yandex Data Transfer
- Migrating databases from MySQL® to ClickHouse® using Yandex Data Transfer
- Asynchronously replicating data from PostgreSQL to ClickHouse®
- Exchanging data between Yandex Managed Service for ClickHouse® and Yandex Data Processing
- Configuring Yandex Managed Service for ClickHouse® for Graphite
- Fetching data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for ClickHouse®
- Delivering data to ksqlDB
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for ClickHouse® using Yandex Data Transfer
- Fetching data from RabbitMQ to Yandex Managed Service for ClickHouse®
- Saving a Yandex Data Streams data stream in Yandex Managed Service for ClickHouse®
- Using hybrid storage in Yandex Managed Service for ClickHouse®
- Sharding Yandex Managed Service for ClickHouse® tables
- Loading data from Yandex Direct to a Yandex Managed Service for ClickHouse® data mart using Yandex Cloud Functions, Yandex Object Storage, and Yandex Data Transfer
- Loading data from Yandex Object Storage to Yandex Managed Service for ClickHouse® using Yandex Data Transfer
- Copying data from Managed Service for OpenSearch to Managed Service for ClickHouse® using Yandex Data Transfer
- Loading data from Yandex Managed Service for YDB to Yandex Managed Service for ClickHouse® using Yandex Data Transfer
- Migrating databases from Google BigQuery to Yandex Managed Service for ClickHouse®
- Yandex Managed Service for ClickHouse® integration with external Microsoft SQL Server database via ClickHouse® JDBC Bridge
- Yandex Managed Service for ClickHouse® integration with Oracle via ClickHouse® JDBC Bridge
Greenplum®
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex MPP Analytics for PostgreSQL using Yandex Data Transfer
- Migrating data from Yandex Managed Service for MySQL® to Yandex MPP Analytics for PostgreSQL using Yandex Data Transfer
- Migrating databases from Greenplum® to ClickHouse®
- Migrating databases from Greenplum® to PostgreSQL
- Exporting Greenplum® data to cold storage in Yandex Object Storage
- Loading data from Yandex Object Storage to Yandex MPP Analytics for PostgreSQL using Yandex Data Transfer
- Copying data from Managed Service for OpenSearch to Yandex MPP Analytics for PostgreSQL using Yandex Data Transfer
- Creating an external table from a Yandex Object Storage bucket table using a configuration file
- Getting data from external sources using named queries
MongoDB/Yandex StoreDoc
- Migrating collections from MongoDB to Yandex StoreDoc
- Migrating data to Yandex StoreDoc
- Migrating a Yandex StoreDoc cluster from version 4.4 to 6.0 using Yandex Data Transfer
- Sharding Yandex StoreDoc collections
- Performance analysis and tuning of Yandex StoreDoc
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex StoreDoc using Yandex Data Transfer
MySQL®
- Migrating a database from a third-party MySQL® cluster to a Yandex Managed Service for MySQL® cluster
- Managed Service for MySQL® performance analysis and tuning
- Syncing data from a third-party MySQL® cluster to Yandex Managed Service for MySQL® using Yandex Data Transfer
- Migrating databases from Yandex Managed Service for MySQL® to a third-party MySQL® cluster
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for MySQL® using Yandex Data Transfer
- Migrating databases from MySQL® to ClickHouse® using Yandex Data Transfer
- Migrating databases from Yandex Managed Service for MySQL® to Yandex Object Storage
- Migrating data from Yandex Object Storage to Yandex Managed Service for MySQL® using Yandex Data Transfer
- MySQL® change data capture and delivery to YDS
- Migrating data from Managed Service for MySQL® to Managed Service for PostgreSQL using Data Transfer
- Migrating data from Yandex Managed Service for MySQL® to Yandex MPP Analytics for PostgreSQL using Yandex Data Transfer
- Importing data from Yandex Managed Service for MySQL® to Yandex Data Processing using Sqoop
- Delivering data from Yandex Managed Service for MySQL® to Yandex Managed Service for Apache Kafka® using Yandex Data Transfer
- Delivering data from Yandex Managed Service for MySQL® to Yandex Managed Service for Apache Kafka® using Debezium
- Migrating databases from Yandex Managed Service for MySQL® to Yandex Managed Service for YDB using Yandex Data Transfer
OpenSearch
- Configuring an index policy in Yandex Managed Service for OpenSearch
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for OpenSearch using Yandex Data Transfer
- Migrating data from a third-party OpenSearch cluster to Yandex Managed Service for OpenSearch using Yandex Data Transfer
- Loading data from Yandex Managed Service for OpenSearch to Yandex Object Storage using Yandex Data Transfer
- Migrating data from Yandex Managed Service for OpenSearch to Yandex Managed Service for YDB using Yandex Data Transfer
- Copying data from Managed Service for OpenSearch to Yandex MPP Analytics for PostgreSQL using Yandex Data Transfer
- Copying data from Managed Service for OpenSearch to Managed Service for ClickHouse® using Yandex Data Transfer
- Migrating data from Yandex Managed Service for PostgreSQL to Yandex Managed Service for OpenSearch using Yandex Data Transfer
- Authenticating a Yandex Managed Service for OpenSearch cluster in OpenSearch Dashboards using Keycloak
- Using the yandex-lemmer plugin in Yandex Managed Service for OpenSearch
PostgreSQL
- Creating a PostgreSQL cluster for 1C:Enterprise
- Searching for Managed Service for PostgreSQL cluster performance issues
- Managed Service for PostgreSQL performance analysis and tuning
- Logical replication PostgreSQL
- Migrating a database from a third-party PostgreSQL cluster to Managed Service for PostgreSQL
- Migrating a database from Managed Service for PostgreSQL
- Asynchronously replicating data from PostgreSQL to ClickHouse®
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for PostgreSQL using Yandex Data Transfer
- Delivering data from Yandex Managed Service for PostgreSQL to Yandex Managed Service for Apache Kafka® using Yandex Data Transfer
- Delivering data from Yandex Managed Service for PostgreSQL to Yandex Managed Service for Apache Kafka® using Debezium
- Importing data from Yandex Managed Service for PostgreSQL to Yandex Data Processing using Sqoop
- Delivering data from Yandex Managed Service for PostgreSQL to Yandex Managed Service for YDB using Yandex Data Transfer
- Migrating databases from Managed Service for PostgreSQL to Object Storage
- Migrating data from Yandex Object Storage to Yandex Managed Service for PostgreSQL using Yandex Data Transfer
- Migrating data from Managed Service for PostgreSQL to Managed Service for MySQL® using Data Transfer
- PostgreSQL change data capture and delivery to YDS
- Migrating data from AWS RDS for PostgreSQL to Yandex Managed Service for PostgreSQL using Yandex Data Transfer
- Migrating data from Yandex Managed Service for PostgreSQL to Yandex Managed Service for OpenSearch using Yandex Data Transfer
- Fixing string sorting issues in PostgreSQL after upgrading glibc
Valkey™
- Migrating a database from a third-party Valkey™ cluster to Yandex Managed Service for Valkey™
- Using a Yandex Managed Service for Valkey™ cluster as a PHP session storage
YDB
- Delivering data from Yandex Managed Service for YDB to Yandex Managed Service for Apache Kafka® using Yandex Data Transfer
- Delivering data from Yandex Managed Service for Apache Kafka® to Yandex Managed Service for YDB using Yandex Data Transfer
- Migrating databases from Yandex Managed Service for MySQL® to Yandex Managed Service for YDB using Yandex Data Transfer
- Delivering data from Yandex Managed Service for PostgreSQL to Yandex Managed Service for YDB using Yandex Data Transfer
- Loading data from Yandex Object Storage to Yandex Managed Service for YDB using Yandex Data Transfer
- Loading data from Yandex Managed Service for YDB to Yandex Object Storage using Yandex Data Transfer
- Loading data from Yandex Managed Service for YDB to Yandex Managed Service for ClickHouse® using Yandex Data Transfer
- Migrating data from Yandex Managed Service for OpenSearch to Yandex Managed Service for YDB using Yandex Data Transfer
- Delivering data from an Data Streams queue to Managed Service for YDB using Yandex Data Transfer
- Yandex Managed Service for YDB change data capture and delivery to Yandex Data Streams
Yandex Cloud DNS
Yandex Data Processing
- Migrating an HDFS Yandex Data Processing cluster to a different availability zone
- Exchanging data between Yandex Managed Service for ClickHouse® and Yandex Data Processing
- Importing data from Yandex Managed Service for MySQL® to Yandex Data Processing using Sqoop
- Importing data from Yandex Managed Service for PostgreSQL to Yandex Data Processing using Sqoop
- Mounting Yandex Object Storage buckets to the file system of Yandex Data Processing hosts
- Working with Apache Kafka® topics using PySpark jobs in Yandex Data Processing
- Automating operations with Yandex Data Processing using Yandex Managed Service for Apache Airflow™
- Shared use of Yandex Data Processing tables through Apache Hive™ Metastore
- Transferring metadata between Yandex Data Processing clusters using Apache Hive™ Metastore
- Importing data from Yandex Object Storage, processing and exporting to Yandex Managed Service for ClickHouse®
Basic examples of working with jobs
- Working with Hive jobs
- Working with MapReduce jobs
- Working with PySpark jobs
- Working with Spark jobs
Advanced examples of working with jobs
- Running Apache Hive jobs
- Launching and managing applications for Spark and PySpark
- Running jobs from remote hosts that are not part of the Yandex Data Processing cluster
Yandex Query
- Processing Yandex Audit Trails events
- Processing Yandex Cloud Logging logs
- Processing Debezium CDC streams
- Analyzing data with Jupyter
- Processing files with usage details in Yandex Cloud Billing
Yandex Data Streams
- Ingesting data into storage systems
- Smart log processing
- Data transfer in microservice architectures
- Migrating data to Yandex Object Storage using Yandex Data Transfer
Yandex Data Transfer
- Migrating data from a third-party Greenplum® or PostgreSQL cluster to Yandex MPP Analytics for PostgreSQL using Yandex Data Transfer
- Migrating MongoDB clusters
- Migrating MySQL® clusters
- Migrating to a third-party MySQL® cluster
- Migrating PostgreSQL clusters
- Creating a schema registry to deliver data in Debezium CDC format from Apache Kafka®
Yandex Managed Service for Apache Spark™
- Automating operations using Yandex Managed Service for Apache Airflow™
- Working with an Object Storage table from a PySpark job using Apache Hive™ Metastore and Apache Iceberg™
- Yandex Managed Service for Apache Spark™ integration with Apache Hive™ Metastore
- Running a PySpark job using Yandex Managed Service for Apache Airflow™
- Using Yandex Object Storage in Yandex Managed Service for Apache Spark™
ClickHouse® is a registered trademark of ClickHouse, Inc.