Visualizing Yandex Object Storage data in Yandex DataLens

By integrating Yandex Query with Yandex DataLens, you can visualize data stored in Yandex Object Storage. DataLens generates a YQL query, while Yandex Query runs the query and returns the results, which are visualized using charts.

For the solution architecture, see below.

As an example, let's visualize how the time of day affects the number and average cost of New York City yellow taxi trips. The data was pre-uploaded to a public Yandex Object Storage bucket named yq-sample-data, the nyc_taxi_csv folder.

To visualize and explore data, set up your cloud and follow the steps below:

1. Connect to data in Object Storage.
2. Create a connection in Yandex DataLens.
3. Configure the dataset fields.
4. Configure visualization.

Getting startedGetting started

Sign up for Yandex Cloud and create a billing account:

1. Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Connect to data in Object StorageConnect to data in Object Storage

1. In the management console, select the folder where you want to create a connection.

   

2. In the list of services, select Yandex Query.

   

3. In the left-hand panel, select Tutorial.

4. Click Create connection. In the Create connection section, all parameter fields are pre-filled. Enter a description (optional) and click Create. This will take you to the screen for creating a data binding.

5. Enter the data binding description (optional) and click Create. Appropriate training resources will be created.

Create a connection in Yandex DataLensCreate a connection in Yandex DataLens

To create a Yandex Query connection:

1. Go to the DataLens home page.

2. In the left-hand panel, select Connections and click Create connection.

3. Select a Yandex Query connection.

4. Specify the connection parameters:

   • Cloud and folder: Select the folder where your service account is located.
   • Service account: Select an existing service account or create a new one.
   • Cache TTL in seconds: Specify the cache time-to-live or leave the default value.
   • Raw SQL level: Select Allow subqueries in datasets and queries from charts.

5. Click Create connection.

6. Enter a connection name, e.g., yq-connect, and click Create.

Configure the dataset fieldsConfigure the dataset fields

1. In the top-right corner of the page where you created the connection, click Create dataset.

   

2. In the window that opens, enter the query text provided below and click Create.

   SELECT 
       *
   FROM 
       bindings.`tutorial-analytics`
   

   If everything is set up correctly, the Preview section in the lower part of the screen will contain the following data:

   

3. Create a calculated field with the pick-up time data:

   1. Go to the Fields tab and click Add field.

      

   2. In Field name, specify hour_trip.

   3. In the formula line, enter:

      HOUR([tpep_pickup_datetime])
      

   4. Click Create. The new hour_trip field will appear in the list of dataset fields.

4. In the Aggregation column, select the Average aggregation type for the total_amount field.

   

5. In the top-right corner, click Save. Enter the dataset name, yellow-taxi-dataset, and click Create.

6. When the dataset is saved, click Create chart in the top-right corner.

   

Configure visualizationConfigure visualization

To visualize and analyze the data, use a combined chart.

1. For the visualization type, choose Combined chart.

   

2. Drag the hour_trip dimension to the X section. In the X section, the dimension will be common for all layers.

3. For the first layer, choose Bar chart.

   

4. Drag the total_amount measure to the Y section. You will see a bar chart in the visualization area.

5. Add a layer. To do this, click the icon to the right of the first layer name.

6. For the second layer, choose Line chart.

   

7. Add a field to the chart:

   1. At the top left, click and select Field.

      

   2. In the window that opens:

      • Under Field name, enter count_trip.

      • In the formula line, enter COUNT().

        

   3. Click Create.

8. Drag the new count_trip measure to the Y2 section. You will see a line chart on top of the first chart in the visualization area.

9. In the top-right corner, click Save. Enter yellow-taxi-combo-chart as the chart name and click Save.

The chart you created shows how the time of day affects the number of trips (line chart) and the average trip cost (bar chart).

In the same way, you can create additional charts and dashboards or share the results with your teammates.

Visualizing Yandex Object Storage data with Yandex Query: