Transferring data from a BigQuery source endpoint

When creating or updating an endpoint, configure access to BigQuery.

Yandex Data Transfer enables you to upload data from BigQuery storage to Yandex Cloud managed databases and implement various data processing and transformation scenarios. To implement a transfer:

1. Explore possible data transfer scenarios.
2. Prepare the BigQuery database for the transfer.
3. Set up an endpoint source in Yandex Data Transfer.
4. Set up one of the supported data targets.
5. Create a transfer and start it.
6. In case of any issues, use ready-made solutions to resolve them.

Scenarios for transferring data from BigQueryScenarios for transferring data from BigQuery

You can upload data from BigQuery cloud storage to managed databases for further storage in the cloud, processing and loading into data marts for further visualization.

For a detailed description of possible Yandex Data Transfer data transfer scenarios, see Tutorials.

Preparing your BigQuery databasePreparing your BigQuery database

To prepare a BigQuery data source:

1. Create a Google Cloud service account.
2. Add the service account as a participant to the Google Cloud project with the BigQuery User role.
3. Create a Google Cloud service account key.

For more information, see the Airbyte® documentation.

Configuring the BigQuery source endpointConfiguring the BigQuery source endpoint

Configuring the data targetConfiguring the data target

Configure one of the supported data targets:

• MySQL®.
• MongoDB.
• ClickHouse®.
• Greenplum®.
• Yandex Managed Service for YDB.
• Apache Kafka®.
• YDS.
• PostgreSQL.

For a complete list of supported sources and targets in Yandex Data Transfer, see Available Transfers.

Make sure that the network hosting the target cluster is configured to allow connections from the internet. To enable internet access, set up routing.

After configuring the data source and target, create and start the transfer.