General questions about Managed Service for Apache Airflow™

How do I fix the IP address range overlap error when creating a cluster?How do I fix the IP address range overlap error when creating a cluster?

When creating a Managed Service for Apache Airflow™ cluster you may get the following error:

user subnet overlaps with service network range 10.248.0.0/13, see documentation for details

This error means that when creating the cluster, you selected a subnet whose IP address range overlaps with the 10.248.0.0/13 address range of the service subnet. In this subnet, Yandex Cloud manages the Managed Service for Apache Airflow™ cluster components.

To fix this error, select another subnet whose IP address range does not overlap with the service subnet range. Learn more about cluster subnet requirements in Network.

How can I fix the no permission error when assigning a service account to a cluster?How can I fix the no permission error when assigning a service account to a cluster?

Error message:

ERROR: rpc error: code = PermissionDenied desc = you do not have permission to access the requested service account or service account does not exist

This error occurs if you link a service account to a cluster while creating or modifying it.

Solution
Assign the iam.serviceAccounts.user role or higher to your Yandex Cloud account.

How do I fix the error?How do I fix the No module named 'airflow.providers.postgres.operators' error?

When accessing a Yandex Managed Service for PostgreSQL cluster from an Apache Airflow™ cluster, you may get this error:

Broken DAG: [/opt/airflow/dags/postgre.py] Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/airflow/dags/postgre.py", line 2, in <module>
    from airflow.providers.postgres.operators.postgres import PostgresOperator
ModuleNotFoundError: No module named 'airflow.providers.postgres.operators'

apache-airflow-providers-postgres 5.13.1 is used by default. The error may occur if using PostgresOperator with newer provider versions.

Solution:

If using a newer provider version, use SQLExecuteQueryOperator instead of PostgresOperator. For more information, see the official documentation.

How do I fix the error?How do I fix the AirflowException: Unknown hook type "postgres" error?

When accessing a Yandex Managed Service for PostgreSQL cluster from an Apache Airflow™ cluster, you may get this error:

Task failed with exception: source="task"
AirflowException: Unknown hook type "postgres"

By default, the apache-airflow-providers-postgres provider is not installed in Yandex Managed Service for Apache Airflow™ clusters with Apache Airflow™ versions above 3.0.

Solution:

When creating or updating a Managed Service for Apache Airflow™ cluster, add the apache-airflow-providers-postgres pip package under Dependencies.

How do I fix the error?How do I fix the No module named 'airflow_clickhouse_plugin' error?

When accessing a Yandex Managed Service for ClickHouse® cluster from an Apache Airflow™ cluster, you may get this error:

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/airflow/dags/dags/clickhouse.py", line 2, in <module>
    from airflow_clickhouse_plugin.hooks.clickhouse import ClickHouseHook
ModuleNotFoundError: No module named 'airflow_clickhouse_plugin'

By default, the airflow-clickhouse-plugin plugin is not installed in Yandex Managed Service for Apache Airflow™.

Solution:

When creating or updating a Managed Service for Apache Airflow™ cluster, add the airflow-clickhouse-plugin pip package under Dependencies.