Contact UsGet started

Delivering data from Yandex Forms to Yandex DataLens using Yandex Cloud Functions and Yandex Query via Terraform

Written by
Updated at May 7, 2025

To set up Forms and DataLens integration via Terraform:

  1. Set up your infrastructure.
  2. Create and configure a function in Yandex Cloud Functions.
  3. Create and set up a form in Forms.
  4. Configure a connection and binding to data in Yandex Query.
  5. Set up fetching data in DataLens.
  6. Test the new resource integration.

If you no longer need the resources you created, delete them.

Get your cloud ready

Sign up in Yandex Cloud and create a billing account:

  1. Navigate to the management console and log in to Yandex Cloud or register a new account.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

The infrastructure support cost includes:

Set up your infrastructure

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

Terraform is distributed under the Business Source License. The Yandex Cloud provider for Terraform is distributed under the MPL-2.0 license.

For more information about the provider resources, see the documentation on the Terraform website or mirror website.

To set up your infrastructure via Terraform:

  1. If you do not have Terraform yet, install it.

  2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

  3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

  4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

  5. Download the forms-and-datalens-integration.tf configuration file to the same working directory.

    This file describes:

    • Network and subnet for deploying the infrastructure for your function in Cloud Functions.
    • Service account with the following roles:
      • lockbox.payloadViewer: For reading Yandex Lockbox secrets.
      • functions.functionInvoker: For calling the function in Cloud Functions.
      • storage.admin: For reading data from and writing data to an Object Storage bucket, as well as managing the bucket’s ACL.
      • yq.viewer and yq.invoker: For integrating DataLens and Query.
    • Static key for the service account used for creating the Object Storage bucket.
    • Static key for the service account used for creating the function in Cloud Functions.
    • Yandex Lockbox secret for storing data on the service account static key for the function from Cloud Functions.
    • data resource for the Yandex Lockbox secret the function in Cloud Functions uses to retrieve the secret version ID.
    • Object Storage bucket to store function results.
    • Publicly accessible function in Cloud Functions.

  6. In the local variables section of the forms-and-datalens-integration.tf file, specify the following:

    • Your folder ID in the sa_folder_id variable.
    • Name of the Object Storage bucket in the bucket-name variable.

  7. Make sure the Terraform configuration files are correct using this command:

    terraform validate

    If there are any errors in the configuration files, Terraform will point them out.

  8. Create the required infrastructure:

    1. Run this command to view the planned changes:

      terraform plan

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

    2. If everything looks correct, apply the changes:

      1. Run this command:

        terraform apply

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

    All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

Create and configure a function in Cloud Functions

  1. Download the function code archive to the directory with the forms-and-datalens-integration.tf file.

  2. Under the local variables section of the forms-and-datalens-integration.tf file, specify the following values:

    • content_path = function.zip.
    • create-function = 1.

  3. Make sure the Terraform configuration files are correct using this command:

    terraform validate

    If there are any errors in the configuration files, Terraform will point them out.

  4. Create the required infrastructure:

    1. Run this command to view the planned changes:

      terraform plan

      If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.

    2. If everything looks correct, apply the changes:

      1. Run this command:

        terraform apply

      2. Confirm updating the resources.

      3. Wait for the operation to complete.

    All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

Create and set up a Forms form

Create an API key

You need an API key to set up integration with Forms.

  1. In the management console, select the appropriate folder.
  2. From the list of services, select Identity and Access Management.
  3. Select the forms-integration service account.
  4. Go to Overview.
  5. Click Create new key and select Create API key.
  6. Click Create.
  7. In the window that opens, New key, you will see Key ID and Your secret key. Save them, as you will need them later.

Create a form

  1. Go to Forms.

  2. Click Create form.

  3. Add at least two fields to the form and set up any questions in them.

  4. Click Publish.

  5. Save the link to the published form: you will need it later.

Set up integration with a Cloud Functions function

  1. Go to Forms.

  2. Open the form you created earlier.

  3. Go to the Settings tab.

  4. Open the Additional section.

  5. Under Cloud function key, enter the data of the previously created API key and click Save.

  6. Open the Integrations tab.

  7. Click Cloud Functions to add a condition for the integration.

  8. In the Function code field, enter the Cloud Functions function ID. You can copy the ID in the Overview section of the Cloud Functions function in the management console.

  9. In the Show outcome notification field, select Show.

  10. Click Save.

Test the integration you created

  1. Follow the public link to the previously created form.

  2. Complete the form and click Submit.

  3. Go to Forms.

  4. Open the form you created earlier.

  5. Open the Integrations tab.

  6. Go to the Completed integrations page.

  7. Open the completed action record and make sure that the Response section contains the 200 — OK response.

  8. Navigate to the management console and open the bucket created earlier. Make sure the bucket now contains a JSON file with the data from the form you completed.

    The name of the directory the file will reside in matches the form's internal ID. Save this ID for the next steps.

    All the future form completion outcomes will be saved in this directory.

  9. Complete the form several times more, and make sure you fill one of the fields in the same way in at least two forms. This will improve data visualization when testing the integration.

Configure a connection and binding to data in Query

  1. Navigate to the management console and select Query.
  2. In the Connections tab, click Create new.
  3. Create a connection with the following parameters:
    • Name: forms-connection.
    • Type: Object Storage.
    • Bucket: Name of the bucket you created earlier.
    • Service account: forms-integration-sa.
  4. In the window that opens, set the data binding parameters:
    • Type: Object Storage.
    • Connection: forms-connection.
    • Path: /<form_ID>/.
      You can copy the ID:
      • Under Objects in the Object Storage bucket. The name of the folder containing the file with results of the completed form matches the form ID.
      • In the address bar of the Forms interface, on the form viewing or editing page.
    • Format: json_each-row.
    • Columns: Create columns for the fields you specified in the form.
      If you want Query to define columns automatically, click Detect columns.

Set up fetching data in DataLens

  1. Go to DataLens.
  2. In the left-hand panel, select Connections and click Create connection.
  3. Select Query. In the window that opens, set the below parameters as follows:
    • Cloud and folder: Select the cloud and folder where you created the rest of the resources.
    • Service account: forms-integration-sa
  4. Click Save changes. In the window that opens, specify forms-datalens-connection as the connection name, then click Create.
  5. Go back to the DataLens main page and click Create dataset.
  6. In the connection panel, click Add and select the forms-datalens-connection connection.
  7. Under Tables, select the table and drag it to your DataLens workspace.
    After loading, the data from the table will appear in the Preview panel.
  8. Click Save and enter forms-integration-dataset as the dataset name, then click Create.

Test the new resource integration

To check how the created resources work together, open the forms-integration-dataset dataset and click Create chart. In the window that opens, create a chart using the data from the form. For better visualization, create a chart based on a field with repeating values.

Delete the resources you created

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them.

  1. In the terminal window, go to the directory containing the infrastructure plan.

    Warning

    Make sure the directory has no Terraform manifests with the resources you want to keep. Terraform deletes all resources that were created using the manifests in the current directory.

  2. Delete resources:

    1. Run this command:

      terraform destroy

    2. Confirm deleting the resources and wait for the operation to complete.

    All the resources described in the Terraform manifests will be deleted.

Previous
Management console
Next
All tutorials