Transferring data from an AWS CloudTrail source endpoint

Yandex Data Transfer enables you to upload audit logs the AWS CloudTrail database to Yandex Cloud managed databases and implement various data processing and transformation scenarios. To implement a transfer:

1. Explore possible data transfer scenarios.
2. Prepare the AWS CloudTrail database for the transfer.
3. Set up an endpoint source in Yandex Data Transfer.
4. Set up one of the supported data targets.
5. Create a transfer and start it.
6. In case of any issues, use ready-made solutions to resolve them.

Scenarios for transferring data from AWS CloudTrailScenarios for transferring data from AWS CloudTrail

You can upload logs to various databases for further storage in the cloud, processing and loading into data marts for further visualization.

For a detailed description of possible Yandex Data Transfer data transfer scenarios, see Tutorials.

Preparing the AWS CloudTrail databasePreparing the AWS CloudTrail database

Get an AWS key ID and secret access key by following the AWS guide.

For more information, see the Airbyte® documentation.

Configuring an AWS CloudTrail source endpointConfiguring an AWS CloudTrail source endpoint

When creating or updating an endpoint, configure access to AWS. For more information, see the Airbyte® documentation.

Configuring the data targetConfiguring the data target

Configure one of the supported data targets:

• MySQL®.
• MongoDB.
• ClickHouse®.
• Greenplum®.
• Yandex Managed Service for YDB.
• Apache Kafka®.
• YDS.
• PostgreSQL.

For a complete list of supported sources and targets in Yandex Data Transfer, see Available Transfers.

Make sure that the network hosting the target cluster is configured to allow connections from the internet. To enable internet access, set up routing.

After configuring the data source and target, create and start the transfer.

Airbyte® is a registered trademark of Airbyte, Inc in the United States and/or other countries.