Visualizing Yandex Object Storage data in Yandex DataLens

By integrating Yandex Query with Yandex DataLens, you can visualize data stored in Yandex Object Storage. DataLens generates a YQL query, while Yandex Query runs it and returns the results, which are visualized using charts.

Below is the architecture used thereby.

As an example, let's visualize how the time of day affects the number and average cost of New York City yellow taxi rides. The data was pre-uploaded to a public Yandex Object Storage bucket named yq-sample-data, in the nyc_taxi_csv folder.

To visualize and explore data, get your cloud ready and follow these steps:

1. Connect to the data in Object Storage.
2. Create a connection in Yandex DataLens.
3. Configure the dataset fields.
4. Configure visualization.

Getting startedGetting started

Sign up in Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or register a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Connect to data in Object StorageConnect to data in Object Storage

1. In the management console, select the folder where you want to create a connection.

   

2. From the list of services, select Yandex Query.

   

3. In the left-hand panel, select Tutorial.

4. Click Create connection. In the Create connection section, all parameter fields have been pre-filled. Optionally, enter a description; then click Create. This will take you to the screen for creating a data binding.

5. Optionally, enter the data binding description; then click Create. This will create the appropriate training resources.

Create a connection in Yandex DataLensCreate a connection in Yandex DataLens

To create a Yandex Query connection:

1. Go to the DataLens home page.

2. In the left-hand panel, select Connections and click Create connection.

3. Select a Yandex Query connection.

4. Specify the connection settings:

   • Cloud and folder. Select the folder where you keep your service account.
   • Service account. Select an existing service account or create a new one.
   • Cache TTL in seconds. Specify the cache time-to-live or leave the default value.
   • Raw SQL level. Select Allow subqueries from datasets and queries from charts.

5. Click Create connection.

6. Enter a connection name, e.g., yq-connect, and click Create.

Configure the dataset fieldsConfigure the dataset fields

1. In the top-right corner of the page where you created the connection, click Create dataset.

   

2. In the window that opens, enter the query text below and click Create.

   SELECT 
       *
   FROM 
       bindings.`tutorial-analytics`
   

   If the configuration is correct, the Preview section in the lower part of the screen will contain the following data:

   

3. Create a calculated field with the pick-up time data:

   1. Navigate to the Fields tab and click Add field.

      

   2. In Field name, specify hour_trip.

   3. In the formula line, enter:

      HOUR([tpep_pickup_datetime])
      

   4. Click Create. The new hour_trip field will appear in the list of dataset fields.

4. In the Aggregation column, select the Average aggregation type for the total_amount field.

   

5. In the top-right corner, click Save. Enter the dataset name, yellow-taxi-dataset, and click Create.

6. Once the dataset is saved, click Create chart in the top-right corner.

   

Configure visualizationConfigure visualization

To visualize and analyze the data, use a combined chart.

1. Select Combined chart as the visualization type.

   

2. Drag the hour_trip dimension to the X section. The dimension in the X section will be common for all layers.

3. For the first layer, select Bar chart.

   

4. Drag the total_amount measure to the Y section. You will see a bar chart in the visualization area.

5. Add a layer. To do this, click to the right of the first layer's name.

6. For the second layer, select Line chart.

   

7. Add a field to the chart:

   1. At the top left, click and select Field.

      

   2. In the window that opens:

      • Under Field name, enter count_trip.

      • In the formula line, enter COUNT().

        

   3. Click Create.

8. Drag the new count_trip measure to the Y2 section. You will see a line chart on top of the first chart in the visualization area.

9. In the top-right corner, click Save. Enter yellow-taxi-combo-chart as the chart name and click Save.

The chart you created shows how the time of day affects the number of trips (line chart) and the average ride cost (bar chart).

In the same way, you can create additional charts and dashboards or share the results with your teammates.