Yandex Tracker: data export and visualization

Visualizing data from Yandex Tracker to Yandex DataLens allows you to build more advanced analytics than when using the Tracker tools.

To visualize data from Tracker to DataLens:

• Set up regular export of data to external storage.
• Visualize the required metrics and data using DataLens.

To visualize the data, follow these steps:

1. Get your cloud ready.
2. Create a DB for storing the Tracker data.
3. Create an OAuth token for Tracker access.
4. Create a Cloud Functions function for importing the data.
5. Create a connection to DataLens.
6. Create a dataset.
7. Create a chart.
8. Create a dashboard in DataLens and add charts to it.

Getting startedGetting started

Sign up for Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or create a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can create or select a folder for your infrastructure on the cloud page.

Learn more about clouds and folders here.

Required paid resourcesRequired paid resources

• Continuously running Managed Service for ClickHouse® cluster (see Managed Service for ClickHouse® pricing).
• Cloud Functions function usage (see Cloud Functions pricing).

If you no longer need the resources you created, delete them.

Create a DB for storing the Tracker dataCreate a DB for storing the Tracker data

1. Navigate to the management console.
2. Go to Managed Service for ClickHouse.
3. Click Create ClickHouse cluster.
4. Specify the cluster parameters:
   • Basic parameters:
     • Environment: PRODUCTION
     • Version: 22.8 LTS
   • Resources:
     • Platform: Intel Ice Lake
     • Type: standart
     • Host class: s3-c2-m8 (2 vCPU, 8 GB)
   • Storage size: 30 GB
   • Hosts:
     • Public access: Enabled
   • DBMS settings:
     • User management via SQL: Disabled
     • Managing databases via SQL: Disabled
     • Username: tracker_data
     • DB name: db1
   • Service settings:
     • DataLens access: Enabled
     • Serverless access: Enabled
       For a full list of settings, see Managed Service for ClickHouse® settings.
5. Click Create cluster. Wait for the new cluster status to change to Alive.
6. Copy and save the host name for further Cloud Functions setup.

Create an OAuth token for Tracker accessCreate an OAuth token for Tracker access

1. Go to the Create an app page.

2. In the What app do you want to create? window, leave For user authorization and click Proceed to creation. If you see a verification window for your Public Services Portal account, close it.

3. In the Create app window:

   • Enter your Service name.
   • Add an icon for your service.
   • Specify the email address for notifications about changes in external authorization.

   Click Next.

4. In the App platforms window, select Web services. In the Redirect URI field, specify the URL where to redirect the user after they have allowed or denied access to the app, or enter https://oauth.yandex.ru/verification_code for debugging.

   Click Next.

5. In the User data access permissions window, under Additional, specify:

   • Read from tracker
   • Write to tracker

   Click Next.

6. Check the data and click Everything is correct, create app.

7. Copy the app ID from the ClientID field.

8. In the window that opens, enter the following URL in the browser search bar:

   https://oauth.yandex.ru/authorize?response_type=token&client_id=<app_ID>
   

   Where client_id is the created app ID copied from the ClientID field.

9. Log in under the Tracker account to be used for data visualization.

10. Save the received OAuth token.

Create a Cloud Functions function for importing the dataCreate a Cloud Functions function for importing the data

1. Navigate to the management console.
2. Go to Cloud Functions.
3. Click Create function.
4. Specify a name for the function and click Create.
5. In the Editor window that opens, select the Python runtime environment.
6. Click Continue.
7. In the Code source field, click ZIP archive.
8. Attach a test archive.
9. In the Entry point field, specify tracker_import.handler.
10. Under Parameters, specify:
    • Timeout: 60
    • Memory: 1024
    • Environment variables:

      • TRACKER_ORG_ID: ID of the Yandex 360 for Business organization.

        Note

        If you are using a Yandex Identity Hub organization (you can check this on the administration page), replace the X-Org-ID header with X-Cloud-Org-Id in the tracker_import.py function code.

      • TRACKER_OAUTH_TOKEN: Tracker account OAuth token.

      • CH_HOST: Host name.

      • CH_DB: Database name.

      • CH_USER: Username.

      • CH_PASSWORD: Password.

      • CH_ISSUES_TABLE: tracker_issues.

      • CH_CHANGELOG_TABLE: tracker_changelog.

      • TRACKER_INITIAL_HISTORY_DEPTH: 1d.

      • CH_STATUSES_VIEW: v_tracker_statuses.

11. Click Save changes.
12. In the Testing tab, click Run test.
13. The test result is a data import log:

    {
        "statusCode": 200,
        "headers": {
        "Content-Type": "text/plain"
        },
        "isBase64Encoded": false,
        "body": "OK"
    }
    
14. Create a trigger to regularly export new data to the DB:
    1. Open the Cloud Functions section.
    2. Click → Create trigger.
    3. Set the trigger type to Timer.
    4. In the Cron expression field, select Every day.
    5. Under Function settings, click Create.
    6. Enter the account name. By default, the account is assigned the functions.functionInvoker role to work with the trigger.
    7. Click Create.
    8. Click Create trigger.

Create a connection in DataLensCreate a connection in DataLens

1. Open the Managed Service for ClickHouse® cluster.

2. On the left side of the window, select DataLens.

3. Click Create connection.

4. Specify the connection settings:

   • Connection: Select in folder.

   • Cluster: Cluster specified when creating the database.

   • Hostname: Host specified when creating the database.

   • HTTP interface port: 8443.

   • Username: Username specified when creating the database.

   • Password: Password specified when creating the database.

   • Cache TTL in seconds: Default.

   • Raw SQL level: Forbid.

   • HTTPS: Enabled.

     

5. Click Create connection.

Create a datasetCreate a dataset

1. Go to the DataLens home page.
2. Click Start in cloud.
3. Go to the connections page.
4. Select a connection.
5. In the top-right corner, click Create dataset.
6. Drag one or more tables to the workspace:
   • db1.v_tracker_issues: Current (most recent) issue cross-section.
   • db1.v_tracker_changelog: Issue parameter change history.
   • Db1.v_tracker_statuses: Status transition time based on the issue change history.
7. Click Save.

Create a chartCreate a chart

1. Go to the DataLens home page.

2. In the left-hand panel, click Charts.

3. Click Create chart → Chart in Wizard.

4. In the top-left corner, click Select dataset.

5. In the Datasets drop-down list, select the dataset you created in the previous step.

6. On the top panel, select a visualization type. By default, the Column chart type is selected.

Create a dashboard and add charts to itCreate a dashboard and add charts to it

1. In the left-hand panel, click Dashboards.

2. Click Create dashboard.

3. At the top of the dashboard page, click Add→ Chart.

4. Fill in the widget parameters. Pay close attention to the following fields:

   • Name: Sets the name of the widget. It is displayed at the top of the widget.
   • Chart: Sets the widget you are adding.
   • Description: Sets the description of the widget. It is displayed at the bottom of the widget.
   • Auto height: Sets the automatic height for Table and Markdown widgets. If this parameter is disabled, you can set the height of the widget on the page using the mouse.

5. Click Add. The widget will be displayed on the dashboard.

6. Save the dashboard:

   1. In the top-right corner of the dashboard, click Save.
   2. Enter a name for the dashboard and click Create.

   For more information about setting up dashboards, see Yandex DataLens dashboard.

How to delete the resources you createdHow to delete the resources you created

To stop paying for the resources you created:

• Delete the ClickHouse® cluster.
• Delete the function in Cloud Functions.

ClickHouse® is a registered trademark of ClickHouse, Inc.