Example of using Yandex Message Queue with Celery

Celery is a task queue for Python that you can use to work with Message Queue.

InstallationInstallation

Install Celery and the necessary dependencies:

pip install celery
pip install celery[sqs]
pip install boto3
pip install pycurl

Set the environment variables:

export AWS_ACCESS_KEY_ID="<access_key_ID>"
export AWS_SECRET_ACCESS_KEY="<secret_key>"
export AWS_DEFAULT_REGION="ru-central1"

Getting startedGetting started

1. Create a service account.
2. Assign a role to a service account.
3. Create a static access key.

ExampleExample

In this example:

1. A task is enqueued.
2. The enqueued tasks are executed.

To run the example:

1. Copy the example to a file named mq_example.py:

   from celery import Celery
   import logging
   import boto3
   
   ENDPOINT = 'message-queue.api.cloud.yandex.net:443'
   
   broker='sqs://{}'.format(ENDPOINT)
   broker_transport_options = {
       'is_secure': True,
   }
   
   app = Celery('mq_example', broker=broker)
   app.conf.broker_transport_options = broker_transport_options
   
   @app.task
   def add(a, b):
       res = a + b
       app.log.get_default_logger().info('{} + {} = {}'.format(a, b, res))
       return res
   
   if __name__ == '__main__':
       add.delay(2, 3)
       print("Task scheduled, now run 'celery worker -A mq_example' to execute it")
   

2. Run the task handler with the command:

   celery worker -A mq_example
   

3. Enqueue a task with the command:

   python mq_example.py
   

By default, Celery creates a Message Queue queue named celery in the folder that the service account belongs to.