Uploading a variable from Yandex Lockbox

When working with Yandex Managed Service for Apache Airflow™, you can use Yandex Lockbox to store artifacts that can be used in DAG files: connections, variables, and configuration data. Yandex Lockbox integrates into Managed Service for Apache Airflow™ via the Yandex Lockbox Secret Backend provider. As a result, access to the secret storage is configured automatically.

You can upload a variable from Yandex Lockbox using a directed acyclic graph (DAG). Data for connecting to the DB is stored in Yandex Lockbox and automatically substituted into the graph.

Getting startedGetting started

Issue the lockbox.payloadViewer role to your service account.

You do not have to assign the lockbox.payloadViewer role for the whole folder. It is enough to assign it for a specific Yandex Lockbox secret once you create it.

Create a Yandex Lockbox secretCreate a Yandex Lockbox secret

Create a Yandex Lockbox secret with the following parameters:

• Name: airflow/variables/var_query
• Secret type: Custom
• Key: value
• Value: SELECT 2

The airflow/variables/var_query secret will store the value variable with SELECT 2 as its value.

Prepare the DAG file and run the graphPrepare the DAG file and run the graph

1. Create a local file named load_variable_from_lockbox.py and copy the following script to it:

   from airflow.decorators import dag, task
   from airflow.models import Variable
   
   
   @dag(schedule=None)
   def load_variable_from_lockbox():
       @task
       def print_var_query():
           query = Variable.get_variable_from_secrets("var_query")
           print("query: ", query)
      
       print_var_query()
   
   
   load_variable_from_lockbox()
   

2. Upload the load_variable_from_lockbox.py DAG file to the bucket you created earlier. This will automatically create a graph with the same name in the Apache Airflow™ web interface.

3. Open the Apache Airflow™ web interface.

4. Make sure a new graph named load_variable_from_lockbox has appeared in the DAGs section.

   It may take a few minutes to upload a DAG file from the bucket.

5. To run the graph, click in the line with its name.

Check the resultCheck the result

To check the result in the Apache Airflow™ web interface:

1. In the DAGs section, open the load_variable_from_lockbox graph.
2. Go to the Graph section.
3. Select the print_var_query job.
4. Go to Logs.
5. Make sure the logs contain the query: SELECT 2 string. This means the query was successful.