Contact UsGet started
© 2024 Iron Hive LLC Belgrade

Transferring data from a Yandex Metrica source endpoint

Written by
Updated at August 5, 2024

Data Transfer enables you to transfer data from a Yandex Metrica tag to a Yandex Cloud managed database.

  1. Explore possible data transfer scenarios.
  2. Set up an endpoint source in Yandex Data Transfer.
  3. Set up one of the supported data targets.
  4. Create a transfer and start it.
  5. Perform required operations with the database and control the transfer.
  6. In case of any issues, use ready-made solutions to resolve them.

Scenarios for transferring data from Yandex Metrica

With Data Transfer, you can transfer data from a Yandex Metrica tag to a ClickHouse® cluster. Such a transfer ensures the data transferred from Yandex Metrica are complete and allows:

  • Processing data with ClickHouse® tools.
  • Streaming data from ClickHouse® to other locations.
  • Visualizing data using Yandex DataLens or other services.

See this tutorial on how to transfer Yandex Metrica tag data to a ClickHouse® cluster.

For a detailed description of other possible Yandex Data Transfer data transfer scenarios, see Tutorials.

Configuring the Metrica source endpoint

Note

You can transfer data from a Yandex Metrica source if the Metrica Pro package is enabled or when using Data Streaming from Yandex Metrica.

When creating or updating an endpoint, configure access to Yandex Metrica.

  • Tag numbers: Specify the numbers of the tags the data will be coming from.
  • Token: Specify the token required to access the tags. To obtain it, follow this link.
  • Hits: Select whether information about hits should be transferred.
    • Fields: Select the hit fields to export. In addition to the fields you select, there are certain mandatory fields that are always exported.
  • Sessions: Select whether information about sessions (visits) should be transferred.
    • Fields: Select the session fields to export. In addition to the fields you select, there are certain mandatory fields that are always exported.

  • Endpoint type: metrika_source.

  • counter_ids: Numbers of the tags that will send data.

  • token.raw: Token required to access the tags. To obtain it, follow this link.

  • streams: Parameters of transmitted data:

    • type: Data type. Specify one of these values:

      • METRIKA_STREAM_TYPE_HITS: Hits (obsolete, not updated)
      • METRIKA_STREAM_TYPE_HITS_V2: Hits
      • METRIKA_STREAM_TYPE_VISITS: Visits

    • columns: List of fields to export. You can view them in the management console. In addition to the fields you select, there are certain required fields that are always exported.

Here is an example of the configuration file structure:

resource "yandex_datatransfer_endpoint" "<endpoint_name_in_Terraform>" {
  name = "<endpoint_name>"
  settings {
    metrika_source {
      counter_ids = [<tag_numbers>]
      token {
        raw = "<token>"
      }
      streams {
        type    = "<data_type>"
        columns = ["<field_list>"]
      }
    }
  }
}

  • counterIds: Numbers of the tags that will send data.

  • token.raw: Token required to access the tags. To obtain it, follow this link.

  • streams: Parameters of transmitted data:

    • type: Data type. Specify one of these values:

      • METRIKA_STREAM_TYPE_HITS: Hits (obsolete, not updated)
      • METRIKA_STREAM_TYPE_HITS_V2: Hits
      • METRIKA_STREAM_TYPE_VISITS: Visits

    • columns: List of fields to export. You can view them in the management console. In addition to the fields you select, there are certain required fields that are always exported.

For more information about the settings, see the service documentation.

Configuring the data target

Configure the supported data target:

For a complete list of supported Yandex Data Transfer sources and targets, see Available Transfers.

After configuring the data source and target, create and start the transfer.

ClickHouse® is a registered trademark of ClickHouse, Inc.

Previous
Target
Next
Source