Reading a file from Yandex Object Storage

Use a directed acyclic graph to read files from Yandex Object Storage of the service account attached to your Apache Airflow™ cluster.

Prepare the DAG file and run the graphPrepare the DAG file and run the graph

1. Create a local file named read_file_from_dags_bucket.py and paste the following script to it:

   from airflow.settings import DAGS_FOLDER
   from airflow.decorators import dag, task
   
   
   @dag(schedule=None)
   def read_file_from_dags_bucket():
       @task
       def read_file():
           with open(f'{DAGS_FOLDER}/data/airflow.txt') as file:
               content = file.read()
               print(f"file content: {content}")
   
       read_file()
   
   
   read_file_from_dags_bucket()
   

2. Upload the read_file_from_dags_bucket.py DAG file to the bucket you created earlier. This will automatically create a graph with the same name in the Apache Airflow™ web interface.

3. Open the Apache Airflow™ web interface.

4. Make sure a new graph named read_file_from_dags_bucket has appeared in the DAGs section.

   It may take a few minutes to load a DAG file from the bucket.

5. To run the graph, click in the line with its name.

Check the resultCheck the result

To check the result in the Apache Airflow™ web interface: