The Definitive Guide to Celery and Django

  • Updated December 15th, 2021
  • v1.2.0
Michael Yin Michael Yin

In this course, you will learn how to add Celery to a Django application, allowing you to handle asynchronous tasks, which is vital for smooth user experiences. With this powerful combo, you will be able to do things like:

  • Run machine learning models
  • Send bulk emails
  • Process images or PDFs
  • Generate exports of user data
  • Perform backups

These functions are increasingly common in modern web applications, but with Celery they are easy to implement. Because so many of these activities may be critical to the operation of your application, this course will walk you through configuring, testing, deploying, and monitoring of Celery tasks. With this setup, you can be assured that your tasks are firing and working as expected, outside the HTTP request/response flow.

Django + Celery

What will you learn?

Select a Part

In this first part, you'll learn how to configure Celery to work with Django and Docker in order to process tasks in the background asynchronously. We'll then look at how to retry failed tasks, debug tasks, and ensure tasks work with Django database transactions correctly. Finally, we'll add WebSocket support to Django with Django Channels.

Learning Objectives

  1. Containerize Django, Celery, and Redis with Docker
  2. Ensure Celery tasks execute correctly with validation and debugging
  3. Create and manage Celery tasks
  4. Configure Django Channels to add WebSocket support to a Django application

In part 2, we'll dive into testing with the standard Django testing framework and pytest. We'll also look at a Test-Driven Development (TDD) workflow. You'll then deploy the application to DigitalOcean and learn how to properly monitor Celery tasks with Flower, Container Advisor, and Prometheus. Finally, we'll look at some Celery best practices and tips for performance tuning.

Learning Objectives

  1. Set up logging with Papertrail
  2. Practice Test-Driven Development
  3. Deploy Django, Celery, and RabbitMQ to DigitalOcean
  4. Configure monitoring

What do you need to know?

This course is targeted at advanced-beginners -- someone with at least 6 months of web development experience. Before beginning, you should have some familiarity with the following topics:

Docker and Docker Compose

Django

Meet the Author

Michael Yin

Michael Yin

Michael Yin is a full-stack developer from China with experience in Django, Python, and Javascript. He also loves writing blog posts to teach people programming.

What developers are saying

The TestDriven.io courses are some of the best courses I've ever done for any language, any platform, any price range... just some of the most thorough and well-sourced courses around.

Just a word of thanks for doing such a great job with these training courses. The thorough, entire-lifecycle approach -- from implementation through test, coverage, quality, CI/CD, and all the rest -- is what separates these courses from other training material that I've completed. I'll be able to walk away from here with knowledge and skills that I can apply immediately at work -- and for that I'm grateful. It's a rare gift in an environment where so much 'training' is really just lightweight treatment that doesn't begin to scratch the surface of real, end-to-end software development. Really well done!

The TestDriven.io courses are worth 10 times what I paid for them.

I'm writing to thank you for all the tutorials and the work you've put out there. I'm new to DevOps and I found TestDriven.io while looking for Django and DevOps related topics. One of the best collections of tutorials and guides I've seen -- very well-written, clear, and concise. You have saved me so much time and energy. Thanks from the bottom of my heart.

I am very much into buying and purchasing any course by you and your team. I've never felt like a better programmer ready to show my coding chops to the world.

Frequently Asked Questions

What tools and technologies are used in this course?

This course covers a variety of technologies and services:

Core

  1. Python
  2. Django
  3. Celery
  4. Django Channels
  5. Flower
  6. Docker
  7. Postgres
  8. Redis
  9. RabbitMQ
  10. Gunicorn
  11. Uvicorn

Testing and Monitoring

  1. pytest
  2. factory_boy
  3. Coverage.py
  4. Container Advisor
  5. Prometheus

Services

  1. Papertrail
  2. DigitalOcean

What can I learn to build with Django and Celery?

Celery is perfect for managing background and periodic tasks. In Django applications, it's commonly used for:

  • Performing long running tasks, like processing image uploads such as cropping, resizing, compressing, or building various thumbnails.
  • Similarly, generating PDFs of large datasets, perhaps as part of an export.
  • Sending bulk emails to users, perhaps as part of a mailing list, email-based game, or facilitating redundancy in forum software.
  • Performing tasks on a schedule, such as a regular backups, generating activity digests, or scraping web content during low traffic periods.

Celery is a must-have for many Django web applications because of these common use cases.

What support does TestDriven.io offer?

Since the courses mimic real-world development, support is provided via Stack Overflow. Helpful users, including the developers of the courses, read and respond to messages on Stack Overflow. If you get stuck and you can't find an answer via Stack Overflow, feel free to reach out via email directly. Just be sure to detail what you've tried. For more, review Support and Consulting.

How long does it take to complete the course?

It's dependent on your current skill level. On average, it takes approximately 12 hours to complete.