Application Skeleton

Part 1, Chapter 4

« Git Repository Tests and Code Quality »


What Are We Building?

Let's dive a bit deeper into the the application that we're building. Say you're an international speaker. You're quite popular so you get hundreds of emails a week with requests for you to speak. It's hard to stay organized. Thus, you've decided to build yourself an application that will allow people to request that you speak at their conference or event.

Your initial idea is to cover these cases:

  • users can request talks
  • you can accept or reject talk requests
  • you can list all talk requests

Where to Start?

The most natural instinct is to open your favorite code editor and start coding. After that you'd start thinking about deployment. You'd deploy it to some server. And then what? New ideas would come. You'd change some code, and add some new code. Are you sure you didn't break anything? Hmm. You're probably not 100% certain, so you'd spend twenty or so minutes testing it manually. That covers just the happy path, so you'd spend another thirty minutes trying to manually test all the edge cases. Once everything works, you'd deploy it... but wait... how did you deploy it? Do you remember the exact steps that you did for the first deploy? Do you see where I'm going with this? Fortunately we know a better way to develop software.

Rather than diving right in, let's focus less on the application requirements and more on it's skeleton and CI/CD pipeline. Along the way, we'll also create resources needed to deploy our application. We'll start implementing ideas only after the circle is completed.

Create a Service

Let's start with our tests. Move to your project directory, and create a new folder called "services":

$ cd talk-booking
$ mkdir services

Within "services", create a new folder called "talk_booking":

$ cd services
$ mkdir talk_booking

Initialize a project with Poetry inside "talk_booking":

$ cd talk_booking
$ poetry init

Package name [talk_booking]:
Version [0.1.0]:
Description []:
Author [Your name <[email protected]>, n to skip]:
License []:
Compatible Python versions [^3.9]:

Would you like to define your main dependencies interactively? (yes/no) [yes] no
Would you like to define your development dependencies interactively? (yes/no) [yes] no
Do you confirm generation? (yes/no) [yes]

Next, install the following development tools:

$ poetry add --dev pytest pytest-cov black isort flake8 bandit safety

These tools will be used to help us produce high quality software.

Details:

  • pytest - used for writing and running automated tests
  • pytest-cov - used to generate code coverage reports
  • Black - takes care of code formatting
  • isort - optimizes imports
  • Flake8 - checks for PEP8 compliance
  • Bandit - checks for code security vulnerabilities
  • Safety - checks for vulnerable packages

You can learn more about these tools in Python Code Quality, Testing in Python, and Modern Test-Driven Development in Python

Next, add the pyproject.toml and poetry.lock files to git, create a commit, and push your code:

$ git add pyproject.toml poetry.lock
$ git commit -m 'Create talk_booking service'
$ git push -u origin master

Add Tests

With the service initialized and development dependencies installed, let's add tests which will run inside our CI/CD pipeline.

Create a new folder called "tests". Add an __init__.py to the new folder to turn it into a package.

Inside "tests", add "unit", "integration", and "e2e" folders. Add an __init__.py to each if them for the same reason as before. Your "talk_booking" project structure should now look like this:

talk_booking
├── poetry.lock
├── pyproject.toml
└── tests
    ├── __init__.py
    ├── e2e
    │   └── __init__.py
    ├── integration
    │   └── __init__.py
    └── unit
        └── __init__.py

As the names suggest, unit tests will be located inside "unit", integration tests inside "integration", and end-to-end tests inside "e2e". That way we can easily select which tests to run inside a particular CI/CD pipeline job.

For example, to run just the unit and integration tests, you can run:

$ poetry run pytest tests/unit/ tests/integration/

Flask Application

Since your application needs to be accessible from a web browser, we need a web framework. Let's go with Flask

First, install Flask:

$ poetry add Flask

We'll start with a simple health check endpoint, starting with a test. First, add a new folder to "tests/integration" called "test_web_app". Add an __init__.py file to it to create a package. Next, create a new file tests/integration/test_web_app/test_app.py:

import pytest

from web_app.app import app


@pytest.fixture
def client():
    app.config["TESTING"] = True

    with app.test_client() as client:
        yield client


def test_health_check(client):
    """
    GIVEN
    WHEN health check endpoint is called with GET method
    THEN response with status 200 and body OK is returned
    """
    response = client.get("/health-check/")
    assert response.status_code == 200
    assert response.data.decode() == "OK"

client is a pytest fixture. A fixture is function that's executed by the pytest runner. By default, fixtures run before each test, but this can be changed via the scope argument.

Its returned value is passed to the test function's client argument. The name of fixture and argument must be the same.

We added the GIVEN, WHEN, THEN notation to the test:

  1. GIVEN tells you about the initial conditions/context.
  2. WHEN tells you what's occurring that needs to be tested.
  3. THEN tells you what's the expected outcome.

Such docstrings improve the readability of your tests.

Want to learn more about fixtures and the GIVEN, WHEN, THEN notation? Check out the Modern Test-Driven Development in Python article.

Ensure the test fails:

$ poetry run pytest tests/integration

Failure:

    from web_app.app import app
E   ModuleNotFoundError: No module named 'web_app'

After that, create a new folder called "web_app" inside "services/talk_booking". Add an __init__.py to, again, make it a package along with an app.py file:

from flask import Flask

app = Flask(__name__)


@app.route("/health-check/")
def health_check():
    return "OK"

if __name__ == "__main__":  # pragma: no cover
    app.run(debug=True)

That's it. Simple as that. Just return "OK".

# pragma: no cover excludes the line from the coverage report. Any line with a comment of "pragma: no cover" is excluded. If that line introduces a clause, for example, an if clause, or a function or class definition, then the entire clause is also excluded.

The test should now pass:

$ poetry run pytest tests/integration

At the end, add all changes to git, create a commit, and push:

$ git add -A
$ git commit -m 'Application skeleton is set up'
$ git push -u origin master

You now have an running application that's tested.

Want to test it out? Run FLASK_APP=web_app/app poetry run flask run. Then, navigate to http://127.0.0.1:5000/health-check/ in your browser. You should see "OK".

Small, Incremental Changes

Thus far, you've built an application that returns "OK" when you call /health-check/. It may not seem like much but you actually have a working application. You have a code base with tests that can run inside a CI/CD pipeline. You have an application that you can deploy and monitor. That's perfect for now.

So, what comes next? Before we answer that, let's think about what you want to achieve first. You want to build an application that will solve your problem. You don't want to create new ones. You want your application to run as smooth as possible. You probably also want to build an application that will be able to follow changes in your business model.

It's easier said than done. Nevertheless, we know approaches and tools that can help us achieve all of these: Test-driven Development (TDD) and Continuous Integration and Delivery (CI/CD).

In order to practice these, we need to focus on making small, incremental changes to the code base. Such changes are easier to control and manage.

A perfect example of this is numerical integration. Say you have a function f(x) and want to calculate the integral from point a to point b. If you take a big step you'll create a huge error:

Big changes

On the other hand, the error will be much, much smaller if your steps are small:

Small changes

It's similar with the size of changes during software development. If changes are small, you can adapt quickly. On the other hand, if changes are big, you may not realize just how far off you are from reality until the changes are completed and live in production.

To answer the initial question of what comes next, we'll set up a CI/CD pipeline for our current application. We'll set up CI jobs for checking code quality, running our automated tests, and deploying our application. Again, we'll do this incrementally, step-by-step. Once done, only then will we start building out the application.

What Have You Done?

In this chapter, you added a new service called talk-booking, installed Flask via Poetry, and added a health check endpoint covered by a test. At this point, your project should look like this:

├── .gitignore
└── services
    └── talk_booking
        ├── poetry.lock
        ├── pyproject.toml
        ├── tests
        │   ├── __init__.py
        │   ├── e2e
        │   │   └── __init__.py
        │   ├── integration
        │   │   ├── test_web_app
        │   │   │    ├── __init__.py
        │   │   │    └── test_app.py
        │   └── unit
        │       └── __init__.py
        └── web_app
            ├── __init__.py
            └── app.py



« Git Repository Tests and Code Quality »

✓ Mark as Completed