Running Flower in Production

Last updated November 8th, 2020

In this tutorial, we'll look at how to configure Flower to run behind Nginx with Docker. We'll also set up basic authentication.

Contents

Docker

Assuming you're using using Redis as your message broker, you're Docker Compose config will look similar to:

version: '3.7'

services:

  redis:
    image: redis
    expose:
      - 6379

  worker:
    build:
      context: .
      dockerfile: Dockerfile
    command: ['celery', 'worker', '-A', 'app.app', '-l', 'info']
    environment:
      - BROKER_URL=redis://redis:6379
      - RESULT_BACKEND=redis://redis:6379
    depends_on:
      - redis

  flower:
    image: mher/flower:0.9.5
    command: ['flower', '--broker=redis://redis:6379', '--port=5555']
    ports:
      - 5557:5555
    depends_on:
      - redis

This tutorial uses Flower version 0.9.5 with Celery 4.4.7.

You can view this sample code in the celery-flower-docker repo on GitHub.

app.py:

import os

from celery import Celery


os.environ.setdefault('CELERY_CONFIG_MODULE', 'celery_config')

app = Celery('app')
app.config_from_envvar('CELERY_CONFIG_MODULE')


@app.task
def add(x, y):
    return x + y

celery_config.py:

from os import environ

broker_url = environ['BROKER_URL']
result_backend = environ['RESULT_BACKEND']

Dockerfile:

FROM python:3.9

WORKDIR /usr/src/app

RUN pip install celery[redis]==4.4.7

COPY app.py .
COPY celery_config.py .

Quick test:

$ docker-compose up -d --build
$ docker-compose exec worker python

>> from app import add
>> task = add.delay(5, 5)
>>>
>>> task.status
'SUCCESS'
>>> task.result
10

You should be able to view the dashboard at http://localhost:5557/.

Nginx

To run flower behind Nginx, first add Nginx to the Docker Compose config:

version: '3.7'

services:

  redis:
    image: redis
    expose:
      - 6379

  worker:
    build:
      context: .
      dockerfile: Dockerfile
    command: ['celery', 'worker', '-A', 'app.app', '-l', 'info']
    environment:
      - BROKER_URL=redis://redis:6379
      - RESULT_BACKEND=redis://redis:6379
    depends_on:
      - redis

  flower:
    image: mher/flower:0.9.5
    command: ['flower', '--broker=redis://redis:6379', '--port=5555']
    expose:  # new
      - 5555
    depends_on:
      - redis

  # new
  nginx:
    image: nginx:latest
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    ports:
      - 80:80
    depends_on:
      - flower

nginx.conf:

events {}

http {
  server {
    listen 80;
    # server_name your.server.url;

    location / {
        proxy_pass http://flower:5555;
        proxy_set_header Host $host;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
  }
}

Quick test:

$ docker-compose down
$ docker-compose up -d --build
$ docker-compose exec worker python

>> from app import add
>> task = add.delay(7, 7)
>>>
>>> task.status
'SUCCESS'
>>> task.result
14

This time the dashboard should be viewable at http://localhost/.

Authentication

To add basic authentication, first create a htpasswd file. For example:

$ htpasswd -c htpasswd michael

Next, add another volume to the nginx service to mount the htpasswd from the host to /etc/nginx/.htpasswd in the container:

nginx:
  image: nginx:latest
  volumes:
    - ./nginx.conf:/etc/nginx/nginx.conf
    - ./htpasswd:/etc/nginx/.htpasswd  # new
  ports:
    - 80:80
  depends_on:
    - flower

Finally, to protect the "/" route, add auth_basic and auth_basic_user_file directives to the location block:

events {}

http {
  server {
    listen 80;
    # server_name your.server.url;

    location / {
        proxy_pass http://flower:5555;
        proxy_set_header Host $host;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        auth_basic  "Restricted";
        auth_basic_user_file  /etc/nginx/.htpasswd;
    }
  }
}

For more on setting up basic authentication with Nginx, review the Restricting Access with HTTP Basic Authentication guide.

Final test:

$ docker-compose down
$ docker-compose up -d --build
$ docker-compose exec worker python

>> from app import add
>> task = add.delay(9, 9)
>>>
>>> task.status
'SUCCESS'
>>> task.result
18

Navigate to http://localhost/ again. This time you should be prompted to enter your username and password.

Featured Course

Test-Driven Development with Django, Django REST Framework, and Docker

In this course, you'll learn how to set up a development environment with Docker in order to build and deploy a RESTful API powered by Python, Django, and Django REST Framework.

Featured Course

Test-Driven Development with Django, Django REST Framework, and Docker

In this course, you'll learn how to set up a development environment with Docker in order to build and deploy a RESTful API powered by Python, Django, and Django REST Framework.