flask and aws

Serving Static Files from Flask with WhiteNoise and Amazon CloudFront

Serving Static Files from Flask with WhiteNoise and Amazon CloudFront




WhiteNoise drastically simplifies static file management since it enables your Flask app to serve up its own static files. Couple it with a CDN like CloudFront or Cloudflare, and it's a convenient solution -- e.g., a good balance between simplicity and performance -- for handling static files on a Platform as a Service (PaaS) like Heroku or PythonAnywhere.

This tutorial details how to manage static files with Flask and WhiteNoise. We'll also configure Amazon CloudFront to get the best possible performance.

Why?

  1. The official WhiteNoise documentation -- Using WhiteNoise with Flask -- is, as of writing, outdated and inaccurate.
  2. There are very few tutorials out there on Flask and WhiteNoise.
  3. There are no tutorials on using Flask with WhiteNoise and CloudFront.

It's worth noting that this tutorial does not cover how to handle user-uploaded media files. Feel free to set this up as well as you work your way through the tutorial. Refer to the Storing Django Static and Media Files on Amazon S3 blog post for more info.

Contents

WhiteNoise

Assuming you have a Flask project set up that uses the Application Factory function pattern, import and configure WhiteNoise:

import os
from flask import Flask, jsonify
from whitenoise import WhiteNoise


def create_app(script_info=None):

    app = Flask(__name__, static_folder="staticfiles")

    WHITENOISE_MAX_AGE = 31536000 if not app.config["DEBUG"] else 0

    # configure WhiteNoise
    app.wsgi_app = WhiteNoise(
        app.wsgi_app,
        root=os.path.join(os.path.dirname(__file__), "staticfiles"),
        prefix="assets/",
        max_age=WHITENOISE_MAX_AGE,
    )

    @app.route("/")
    def hello_world():
        return jsonify(hello="world")

    return app

Configuration:

  • root is the absolute path to the directory of static files.
  • prefix is the prefix string for all static URLs. In other words, based on the above configuration, a main.css static file will be available at http://localhost:5000/assets/main.css.
  • max_age is the length of time in seconds that browsers and proxies should cache the static files.

Review the Configuration attributes section, from the official WhiteNoise documentation, for more info on optional arguments.

Add a "static" directory in the project root and, for testing purpose, download a copy of boostrap.css and add it to that newly created directory. Add a "staticfiles" directory to the project root as well.

Your project structure should now look something like this:

├── app.py
├── static
│   └── bootstrap.css
└── staticfiles

Next, add the following script -- called compress.py -- to your project root that compresses the files from the "static" directory and then copies them over to the "staticfiles" directory:

import os
import gzip


INPUT_PATH = os.path.join(os.path.dirname(__file__), "static")
OUTPUT_PATH = os.path.join(os.path.dirname(__file__), "staticfiles")
SKIP_COMPRESS_EXTENSIONS = [
    # Images
    ".jpg",
    ".jpeg",
    ".png",
    ".gif",
    ".webp",
    # Compressed files
    ".zip",
    ".gz",
    ".tgz",
    ".bz2",
    ".tbz",
    ".xz",
    ".br",
    # Flash
    ".swf",
    ".flv",
    # Fonts
    ".woff",
    ".woff2",
]


def remove_files(path):
    print(f"Removing files from {path}")
    for filename in os.listdir(path):
        file_path = os.path.join(path, filename)
        try:
            if os.path.isfile(file_path):
                os.unlink(file_path)
        except Exception as e:
            print(e)


def main():
    # remove all files from "staticfiles"
    remove_files(OUTPUT_PATH)

    for dirpath, dirs, files in os.walk(INPUT_PATH):
        for filename in files:
            input_file = os.path.join(dirpath, filename)
            with open(input_file, "rb") as f:
                data = f.read()
            # compress if file extension is not part of SKIP_COMPRESS_EXTENSIONS
            name, ext = os.path.splitext(filename)
            if ext not in SKIP_COMPRESS_EXTENSIONS:
                # save compressed file to the "staticfiles" directory
                compressed_output_file = os.path.join(OUTPUT_PATH, f"{filename}.gz")
                print(f"\nCompressing {filename}")
                print(f"Saving {filename}.gz")
                output = gzip.open(compressed_output_file, "wb")
                try:
                    output.write(data)
                finally:
                    output.close()
            else:
                print(f"\nSkipping compression of {filename}")
            # save original file to the "staticfiles" directory
            output_file = os.path.join(OUTPUT_PATH, filename)
            print(f"Saving {filename}")
            with open(output_file, "wb") as f:
                f.write(data)


if __name__ == "__main__":
    main()

This script:

  1. Removes any existing files in the "staticfiles" directory
  2. Iterates through the files in the "static" directory and compresses then saves the compressed version to the "staticfiles" directory along with the original, uncompressed version

By having both the compressed and uncompressed versions available, WhiteNoise will serve up the compressed version when a client specifically asks for it. You'll see an example of this shortly.

To test, first install WhiteNoise, if you haven't already done so:

$ pip install whitenoise

Next, add a dummy PNG file to the "static" directory, to ensure that it gets skipped in the compress script, and then run the script:

$ touch static/test.png
$ python compress.py

You should see:

Removing files from staticfiles

Compressing bootstrap.css
Saving bootstrap.css.gz
Saving bootstrap.css

Skipping compression of test.png
Saving test.png

The "staticfiles" directory should now be populated:

├── app.py
├── compress.py
├── static
│   ├── bootstrap.css
│   └── test.png
└── staticfiles
    ├── bootstrap.css
    ├── bootstrap.css.gz
    └── test.png

To verify that this worked, install then run Gunicorn:

$ pip install gunicorn
$ gunicorn "app:create_app()" -b 127.0.0.1:5000

Now, to test out WhiteNoise's gzip functionality with cURL, run:

$ curl -I -H "Accept-Encoding: gzip" http://localhost:5000/assets/bootstrap.css

You should see the following response:

HTTP/1.1 200 OK
Server: gunicorn/19.9.0
Date: Sun, 03 Nov 2019 21:57:41 GMT
Connection: close
Content-Type: text/css; charset="utf-8"
Cache-Control: max-age=31536000, public
Access-Control-Allow-Origin: *
Vary: Accept-Encoding
Last-Modified: Sun, 03 Nov 2019 21:52:35 GMT
ETag: "5dbfae93-2b7e8"
Content-Length: 22200
Content-Encoding: gzip

Take note of Content-Encoding: gzip. This indicates that the gzipped version of the file was served.

CloudFront

Although it's not required, using a Content Delivery Network (CDN) is highly recommended since it will store cached versions of your static files on multiple geographic edge locations. Your visitors will then be served your static content from the location closest to them, which will improve the web server's overall response time. CloudFront, in particular, provides a number of additional features as well like protection against DDoS attacks and access control permissions, to name a few.

To set up, log in to the AWS Console and navigate to the CloudFront dashboard. Click "Create Distribution" and choose "Get Started" under the "Web" section. Add your domain (without http or https) in the "Origin Domain Name" field and leave the remaining defaults. Then, click "Create Distribution".

If you don't have a domain name configured, feel free to test this setup locally with ngrok. With your Gunicorn server up and running on port 5000, download (if necessary) then start ngrok:

$ ngrok http 5000

Once started, you should see a public URL that you can use with CloudFront.

Want to see a demo of this in action? Check out the video below.

It generally takes about 15 minutes for CloudFront to fully configure your distribution. You can test it out before it's been fully distributed out to all edge locations while the creation status is still "In Progress", though. It still may take a few minutes before you can begin testing.

To test, grab the URL associated with the CloudFront distribution and run:

$ curl -I -H "Accept-Encoding: gzip" https://d1tmz03ynvjh5g.cloudfront.net/assets/bootstrap.css

You should see something similar to:

HTTP/1.1 200 OK
Content-Type: text/css; charset="utf-8"
Content-Length: 22200
Connection: keep-alive
Server: gunicorn/19.9.0
Date: Sun, 03 Nov 2019 22:16:07 GMT
Cache-Control: max-age=31536000, public
Access-Control-Allow-Origin: *
Last-Modified: Sun, 03 Nov 2019 21:52:35 GMT
ETag: "5dbfae93-2b7e8"
Content-Encoding: gzip
Vary: Accept-Encoding
X-Cache: Miss from cloudfront
Via: 1.1 68aef80d7f290793b86c83a688a0464f.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: DEN50-C1
X-Amz-Cf-Id: yyZAZoyHfFXLLlHol94Jgk8Qy2-wWKolAPfysXNCuVMQWnVFpHzBYQ==

You can now use the provided CloudFront domain in the Flask app to handle static file requests:

import os
from urllib.parse import urljoin

from flask import Flask, jsonify, render_template
from whitenoise import WhiteNoise


def create_app(script_info=None):

    app = Flask(__name__, static_folder="staticfiles")

    WHITENOISE_MAX_AGE = 31536000 if not app.config["DEBUG"] else 0
    CDN = "https://d1tmz03ynvjh5g.cloudfront.net"

    app.config["STATIC_URL"] = CDN if not app.config["DEBUG"] else ""

    # configure WhiteNoise
    app.wsgi_app = WhiteNoise(
        app.wsgi_app,
        root=os.path.join(os.path.dirname(__file__), "staticfiles"),
        prefix="assets/",
        max_age=WHITENOISE_MAX_AGE,
    )

    @app.template_global()
    def static_url(prefix, filename):
        return urljoin(app.config["STATIC_URL"], f"{prefix}/{filename}")

    @app.route("/")
    def hello_world():
        return jsonify(hello="world")

    return app

The static_url should be used instead of url_for in your templates.

Sanity Check

Let's configure a template to test this out.

Add a new handler:

import os
from urllib.parse import urljoin

from flask import Flask, jsonify, render_template
from whitenoise import WhiteNoise


def create_app(script_info=None):

    app = Flask(__name__, static_folder="staticfiles")

    WHITENOISE_MAX_AGE = 31536000 if not app.config["DEBUG"] else 0
    CDN = "https://d1tmz03ynvjh5g.cloudfront.net"

    app.config["STATIC_URL"] = CDN if not app.config["DEBUG"] else ""

    # configure WhiteNoise
    app.wsgi_app = WhiteNoise(
        app.wsgi_app,
        root=os.path.join(os.path.dirname(__file__), "staticfiles"),
        prefix="assets/",
        max_age=WHITENOISE_MAX_AGE,
    )

    @app.template_global()
    def static_url(prefix, filename):
        return urljoin(app.config["STATIC_URL"], f"{prefix}/{filename}")

    @app.route("/")
    def hello_world():
        return jsonify(hello="world")

    @app.route("/hi")
    def index():
        return render_template("index.html")

    return app

Create a new directory called "templates" in the project root and add an index.html file to that directory:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta http-equiv="X-UA-Compatible" content="ie=edge">
  <link rel="stylesheet" href="{{ static_url('assets', filename='bootstrap.css') }}">
  <title>Hello, world!</title>
</head>
<body>
  <div class="container" style="padding-top:100px">
    <h1>Hello, world!</h1>
  </div>
</body>
</html>

Restart the Gunicorn server and then test it out at http://localhost:5000/hi.

You should see:

cloudfront http response

Try running a WebPageTest to ensure static files are being compressed and cached correctly:

webpagetest results

Demo video:





Join our mailing list to be notified about course updates and new tutorials.

 

Authentication with Flask, React, and Docker

Get the full course. Learn how to add authentication to a Flask and React microservice.

View the Course

Authentication with Flask, React, and Docker

Get the full course. Learn how to add authentication to a Flask and React microservice.


Table of Contents