This post looks at how to implement several asynchronous task queues using the Python multiprocessing library and Redis.
Task queues manage background work asynchronously outside of a user request. With regard to web development, they are used to process tasks outside the typical request/response cycle. They are quite popular in microservice architectures for inter-microservice communication. Celery and RQ are the two most popular Python-based task queues.
The tutorials and articles on TestDriven focus on setting up and configuring task queues to work with Docker and various Python web Frameworks like Django and Flask.
Latest Posts (8)
Making Celery work nicely with Django Database Transactions.
This blog post looks at how to send confirmation emails to newly registered users with Flask, Redis Queue, and Amazon SES.
Automatically retry failed Celery tasks.
This post looks at how to manage periodic tasks with Django, Celery, and Docker.
This post looks at how to configure Celery to handle long-running tasks in a Django app.
This post looks at how to configure Redis Queue (RQ) to handle long-running tasks in a Flask app.
This post shows how to integrate Celery into the Python-based Falcon web framework.