Impersonation in Yandex Managed Service for Apache Airflow™
Impersonation in Managed Service for Apache Airflow™ occurs when an Apache Airflow™ cluster performs certain actions with user resources on behalf of a user's service account.
By default, an Apache Airflow™ cluster does not have permissions to access user resources. To provide access to such resources, create a service account with the required roles and attach it to the Apache Airflow™ cluster when creating or updating the cluster. After that, you will be able to authenticate on behalf of the service account from the code of DAG files.
Impersonation enables an Apache Airflow™ cluster to support integration with other Yandex Cloud services via the Yandex Cloud Python SDK and Airflow Yandex Provider.
Managed Service for Apache Airflow™ also supports integration with Yandex Lockbox as an Apache Airflow™ secret storage.
Integration with Yandex Cloud via the Python SDK
The Yandex Cloud Python SDK allows you to manage Yandex Cloud resources from DAG files of the Apache Airflow™ cluster on behalf of a service account. To do this, create the yandexcloud.SDK() object in a DAG file without specifying authentication parameters. The DAG file will be authenticated using the IAM token of the service account attached to the cluster.
For an integration example, see Sending requests to the Yandex Cloud API via the Yandex Cloud Python SDK.
Integration with Yandex Cloud via the Airflow Provider
Managed Service for Apache Airflow™ clusters with enabled impersonation have an automatically configured yandexcloud_default connection that all operators in the Airflow Yandex Provider use by default. To use it, create a DAG file without specifying the yandex_conn_id parameter. The operator will be authenticated using the IAM token of the service account attached to the cluster.
For an integration example, see Automating Yandex Query tasks using Yandex Managed Service for Apache Airflow™.
Integration with Yandex Lockbox
Yandex Lockbox is a service for centralized storage of secrets. In Yandex Lockbox, you can store data to use in DAG files, such as configuration data, variables and Apache Airflow™ connection parameters.
By default, Apache Airflow™ stores sensitive data in a metadata storage. In this case, you have to manage secrets for every Apache Airflow™ cluster manually via the UI or API. To automate management of secrets, store them in Yandex Lockbox. For more information, see the Apache Airflow™ documentation.
To use Yandex Lockbox features in an Apache Airflow™ cluster, create a secret with the required data and provide the permission to access it to the service account attached to the cluster. After that, you will be able to use data from the secret in the cluster's DAG files.
For an integration example, see Storing Apache Airflow™ connections and variables in Yandex Lockbox.