Delivering data from Yandex Forms to Yandex DataLens using Yandex Cloud Functions and Yandex Query via the management console

• lockbox.payloadViewer: For reading Yandex Lockbox secrets.
• functions.functionInvoker: For invoking the Cloud Functions function.
• storage.editor: For reading data from, and writing data to, an Object Storage bucket.
• yq.viewer and yq.invoker: For integrating DataLens and Query.

In the management console, select the appropriate folder.

From the list of services, select Object Storage.

Open the bucket you created earlier.

Go to Objects.

Click and select Configure ACL.

In the ACL editing window that opens:

1. Start typing the service account name, forms-integration, and select it from the drop-down list.
2. Select the READ and WRITE access permissions.
3. Click Add.
4. Click Save.

1. Select Secret type: Custom.
2. Create the following key-value pairs:

   • Static key ID:

     • Key: static-key-id.
     • Value: Select Text and enter the ID of the static key you created earlier.

   • Static key value:

     • Key: static-key-value.
     • Value: Select Text and enter the value of the static key you created earlier.

In the management console, select the appropriate folder.

From the list of services, select Cloud Functions.

Create a function:

1. Click Create function in the top-right corner.
2. Enter the function Name: forms-function.
3. Click Create.

Create a function version:

1. In the Editor window that opens, select the runtime environment: Python.

2. Disable the Add files with code examples option.

3. Click Continue.

4. Select the function creation Method: Code editor.

5. In the function editor window, click Create file.

6. In the window that opens, enter the file Name: forms-integration.py. Click Create.

7. Paste the function code to the forms-integration.py file:

   import boto3
   import os
   import json
   from botocore.config import Config
   
   def get_answers(raw_answers: json) -> json:
       result = {}
       for name, value in raw_answers.get('answer').get('data').items():
           if name.startswith('answer_choices_'):
               choices = []
               for c in value.get('value'):
                   choices.append(c.get('text'))
               result[name] = ','.join(choices)
           else:
               result[name] = value.get('value')
       return result
   
   def handler(event, context):
       headers = event.get('headers')
       form_id = headers.get('X-Form-Id')
       answer_id = headers.get('X-Form-Answer-Id')
   
       body_json = json.loads(event.get('body'))
       answers_json = get_answers(body_json)
   
       aws_access_key_id = os.environ['KEY']
       aws_secret_access_key = os.environ['SECRET_KEY']
       bucket_name = os.environ['BUCKET']
   
       s3 = boto3.client(service_name='s3',
                         aws_access_key_id=aws_access_key_id,aws_secret_access_key=aws_secret_access_key,
                         endpoint_url='https://storage.yandexcloud.net', region_name='ru-central1',
                         config=Config(request_checksum_calculation="when_required"))
   
       s3.put_object(Bucket=bucket_name, Key=f"{form_id}/{answer_id}.json", Body=json.dumps(answers_json))
   
       return {
           'statusCode': 200,
           'body': "OK",
           'headers': {
               'Content-Type': 'text/plain',
           }
       }
   
   

8. In the function editor window, click Create file.

9. In the window that opens, enter the file Name: requirements.txt. Click Create.

10. In requirements.txt, specify the boto3 library for accessing Object Storage from your function:

    boto3
    

11. In the Entry point field, specify forms-integration.handler.

12. In the Service account field, select the previously created account named forms-integration-sa.

13. Under Environment variables, create a variable containing the bucket name where the function results will be stored:

    1. Key: BUCKET.
    2. Value: Name of the bucket you created earlier.

14. Under Lockbox secrets:

    1. Create a KEY variable for the static key ID. Set the parameters as follows:

       • Environment variable: KEY.
       • Secret ID: Select the static-key-id secret.
       • Version ID: Select the version marked latest.
       • Secret key: Select static-key-id.

    2. Create a SECRET_KEY variable for the static key ID. Set the parameters as follows:

       • Environment variable: SECRET_KEY.
       • Secret ID: Select the static-key-value secret.
       • Version ID: Select the version marked latest.
       • Secret key: Select static-key-value.

15. Once your function is created, on the Overview tab, enable Public function:

Go to Forms.

Click Create form.

Add at least two fields to the form and set up any questions in them.

Click Publish.

Save the link to the published form: you will need it later.

Go to Forms.

Open the form you created earlier.

Go to the Settings tab.

Open the Additional section.

Under Cloud function key, enter the data of the previously created API key and click Save.

Open the Integrations tab.

Click Cloud Functions to add a condition for the integration.

In the Function code field, enter the Cloud Functions function ID. You can copy the ID in the Overview section of the Cloud Functions function in the management console.

In the Show outcome notification field, select Show.

Click Save.

• Name: forms-connection.
• Type: Object Storage.
• Bucket: Name of the bucket you created earlier.
• Service account: forms-integration-sa.

• Type: Object Storage.
• Connection: forms-connection.
• Path: /<form_ID>/.
  You can copy the ID:
  • Under Objects in the Object Storage bucket. The name of the folder containing the file with results of the completed form matches the form ID.
  • In the address bar of the Forms interface, on the form viewing or editing page.
• Format: json_each-row.
• Columns: Create columns for the fields you specified in the form.
  If you want Query to define columns automatically, click Detect columns.

• Cloud and folder: Select the cloud and folder where you created the rest of the resources.
• Service account: forms-integration-sa