Tips and Tricks

Docker tagging best practices

Docker best practice:

Version Docker images to know which version of your code is running and to simplify rollbacks. Avoid the latest tag.

Examples:

docker build -t web-prod-a072c4e-0.1.4 .

Docker - include a HEALTHCHECK instruction

Docker best practice:

Use HEALTHCHECK to verify that the process running inside the container is healthy.

For example, call the health check endpoint of your web app:

HEALTHCHECK CMD curl --fail http://localhost:8000 || exit 1

Docker - array vs string based CMD

Docker best practice:

Use array over string syntax in your Dockerfiles to handle signals properly:

# array (exec)
CMD ["gunicorn", "-w", "4", "-k", "uvicorn.workers.UvicornWorker", "main:app"]

# string (shell)
CMD "gunicorn -w 4 -k uvicorn.workers.UvicornWorker main:app"

Using the string form causes Docker to run your process using bash, which doesn't handle signals properly. Since most shells don't process signals to child processes, if you use the shell format, CTRL-C (which generates a SIGTERM) may not stop a child process.

Docker - run only one process per container

Docker best practice:

Run only one process per container to make it easier to reuse and scale each of the individual services:

1. Scaling - With each service being in a separate container, you can scale one of your web servers horizontally as needed to handle more traffic.
2. Reusability - Perhaps you have another service that needs a containerized database. You can simply reuse the same database container without bringing two unnecessary services along with it.
3. Logging - Coupling containers makes logging much more complex.
4. Portability and Predictability - It's much easier to make security patches or debug an issue when there's less surface area to work with.

Docker - Cache Python Packages to the Docker Host

Docker best practice:

Cache Python packages to the Docker host by mounting a volume or using BuildKit.

Example Dockerfile:

# Mount volume option
-v $HOME/.cache/pip-docker/:/root/.cache/pip


# BuildKit
# syntax = docker/dockerfile:1.2

...

COPY requirements.txt .

RUN --mount=type=cache,target=/root/.cache/pip \
        pip install -r requirements.txt

...

Docker ADD vs COPY

Docker best practice:

Prefer COPY over ADD when copying files from a location to a Docker image.

Use ADD to:

1. download external files
2. extract an archive to the destination

👇

# copy local files on the host to the destination
COPY /source/path  /destination/path
ADD /source/path  /destination/path

# download external file and copy to the destination
ADD http://external.file/url  /destination/path

# copy and extract local compresses files
ADD source.file.tar.gz /destination/path

Docker - use unprivileged containers

Docker best practice:

Always run a container with a non-root user. Running as root inside the container is running as root in the Docker host. If an attacker gains access to your container, they have access to all the root privileges and can perform several attacks against the Docker host.

👇

RUN addgroup --system app && adduser --system --group app

USER app

Dockerfile - Multiple RUN commands v. single chained RUN command

Docker best practice:

In your Dockerfile, combine commands to minimize the number of layers and therefore reduce the image size.

# 2 commands
RUN apt-get update
RUN apt-get install -y netcat


# single command
RUN apt-get update && apt-get install -y netcat

Results:

# docker history to see layers

$ docker images
REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
dockerfile   latest    180f98132d02   51 seconds ago   259MB

$ docker history 180f98132d02

IMAGE          CREATED              CREATED BY                                      SIZE      COMMENT
180f98132d02   58 seconds ago       COPY . . # buildkit                             6.71kB    buildkit.dockerfile.v0
<missing>      58 seconds ago       RUN /bin/sh -c pip install -r requirements.t…   35.5MB    buildkit.dockerfile.v0
<missing>      About a minute ago   COPY requirements.txt . # buildkit              58B       buildkit.dockerfile.v0
<missing>      About a minute ago   WORKDIR /app
...

Which Docker base image should you use?

Docker best practice:

Use smaller base images for your application. *-slim is usually a good choice.

• faster building
• faster pushing
• faster pulling

REPOSITORY   TAG                 IMAGE ID       CREATED      SIZE
python       3.9.6-alpine3.14    f773016f760e   3 days ago   45.1MB
python       3.9.6-slim          907fc13ca8e7   3 days ago   115MB
python       3.9.6-slim-buster   907fc13ca8e7   3 days ago   115MB
python       3.9.6               cba42c28d9b8   3 days ago   886MB
python       3.9.6-buster        cba42c28d9b8   3 days ago   886MB
5:17

Pay close attention to the order of your Dockerfile commands to leverage layer caching

Docker best practice:

Order Dockerfile commands appropriately to better leverage caching.

Example:

# sample.py is copied before requirements.txt
# dependencies will be installed for every change to sample.py

FROM python:3.9-slim

WORKDIR /app

COPY sample.py .

COPY requirements.txt .

RUN pip install -r /requirements.txt


# sample.py is copied after requirements.txt
# dependencies will be installed only for changes to requirements.txt
# when there are no changes, Docker cache will be used

FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install -r /requirements.txt

COPY sample.py .

Docker multi-stage builds

Docker best practice:

Use multistage builds to reduce the size of the production image.

# temp stage
FROM python:3.9-slim as builder

WORKDIR /app

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

RUN apt-get update && \
    apt-get install -y --no-install-recommends gcc

COPY requirements.txt .
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /app/wheels -r requirements.txt


# final stage
FROM python:3.9-slim

WORKDIR /app

COPY --from=builder /app/wheels /wheels
COPY --from=builder /app/requirements.txt .

RUN pip install --no-cache /wheels/*

Serving files with Python's HTTP server

Python tip:

When you need to just serve your static files inside a folder you can do that with Python's HTTP server:

$ cat index.html
<html>
  <h1>Website Prototype</h1>
  <h2>List of Users:</h2>
  <ul>
    <li>Patrick</li>
    <li>Jan</li>
  </ul>
</html>

$ python3 -m http.server
Serving HTTP on :: port 8000 (http://[::]:8000/) ...

Python docstrings examples

Python Clean Code Tip:

Use docstrings to document usage of your modules, classes, and functions.

"""
The temperature module: Manipulate your temperature easily

Easily calculate daily average temperature
"""

from typing import List


class HighTemperature:
    """Class representing very high temperatures"""

    def __init__(self, value: float):
        """
        :param value: value of temperature
        """

        self.value = value


def daily_average(temperatures: List[float]) -> float:
    """
    Get average daily temperature

    Calculate average temperature from multiple measurements

    :param temperatures: list of temperatures
    :return: average temperature
    """

    return sum(temperatures) / len(temperatures)

Do not store secrets in plaintext in code

Python Clean Code Tip:

Avoid storing things like secret keys, passwords, connection strings, and API keys inside your code. Instead, use a secrets management solution like AWS Secrets Manager or Vault.

# bad


class ProductionConfig:
    DEBUG = False
    TESTING = False
    APP_ENVIRONMENT = "production"
    SQLALCHEMY_DATABASE_URI = (
        "postgresql://my_user:strong_password@my_server:5432/my_db"
    )


# better

import boto3


class ProductionConfig:
    DEBUG = False
    TESTING = False
    APP_ENVIRONMENT = "production"
    _SQLALCHEMY_DATABASE_URI = None

    @property
    def SQLALCHEMY_DATABASE_URI(self):
        if self._SQLALCHEMY_DATABASE_URI is None:
            self._SQLALCHEMY_DATABASE_URI = boto3.client(
                "secretsmanager"
            ).get_secret_value(SecretId=f"db-connection-string-{self.APP_ENVIRONMENT}")[
                "SecretString"
            ]

        return self._SQLALCHEMY_DATABASE_URI

If a secrets management tool is overkill for your project, store secrets in environment variables. Never store them in plaintext in your code.

Python - use real objects over primitive types

Python Clean Code Tip:

Favor real objects over primitive types such as dictionaries.

Why?

1. It's easier to type user.name rather than user['name']
2. You'll get help from your IDE
3. You can actually check your code before it runs with mypy
4. It makes your code more clear

# bad
user = {"first_name": "John", "last_name": "Doe"}
full_name = f"{user['first_name']} {user['last_name']}"
print(full_name)
# => John Doe


# better
class User:
    def __init__(self, first_name, last_name):
        self.first_name = first_name
        self.last_name = last_name

    def full_name(self):
        return f"{self.first_name} {self.last_name}"


user = User(first_name="John", last_name="Doe")
print(user.full_name())
# => John Doe

Python - find minimum value using special comparator

Python Clean Code Tip:

Use min to find an element with minimal value inside an iterable. You can provide a custom function as a key argument to serve as a key for the min comparison.

temperatures = [22.3, 28.7, 15.3, 18.2]

# without min
min_temperature = 10000

for temperature in temperatures:
    if temperature < min_temperature:
        min_temperature = temperature

print(min_temperature)
# => 15.3


# with min
min_temperature = min(temperatures)
print(min_temperature)
# => 15.3


# using key
users = [
    {"username": "johndoe", "height": 1.81},
    {"username": "marrydoe", "height": 1.69},
    {"username": "joedoe", "height": 2.03},
]
shortest_user = min(users, key=lambda user: user["height"])
print(shortest_user)
# {'username': 'marrydoe', 'height': 1.69}

Be consistent with the order of your parameters

Python Clean Code Tip:

Be consistent with order of parameters for similar functions/methods. Don't confuse your readers.

# bad
def give_first_dose_of_vaccine(person, vaccine):
    print(f"Give first dose of {vaccine} to {person}.")


def give_second_dose_of_vaccine(vaccine, person):
    print(f"Give second dose of {vaccine} to {person}.")


give_first_dose_of_vaccine("john", "pfizer")
# Give first dose of pfizer to john.
give_second_dose_of_vaccine("jane", "pfizer")
# Give second dose of jane to pfizer.


# good
def give_first_dose_of_vaccine(person, vaccine):
    print(f"Give first dose of {vaccine} to {person}.")


def give_second_dose_of_vaccine(person, vaccine):
    print(f"Give second dose of {vaccine} to {person}.")


give_first_dose_of_vaccine("john", "pfizer")
# Give first dose of pfizer to john.
give_second_dose_of_vaccine("jane", "pfizer")
# Give second dose of pfizer to jane.

Python - High-precision calculations with Decimal

Python Clean Code Tip:

Avoid using floats when you need precise results. Use Decimal instead.

e.g. prices

👇

from dataclasses import dataclass


# bad
from decimal import Decimal


@dataclass
class Product:
    price: float


print(Product(price=0.1 + 0.2))
# => Product(price=0.30000000000000004)


# good
@dataclass
class Product:
    price: Decimal


print(Product(price=Decimal("0.1") + Decimal("0.2")))
# => Product(price=Decimal('0.3'))

Python - OOP tip: set attributes in the constructor

Python Clean Code Tip:

Avoid setting attributes of your objects outside of the constructor. Instead, implement methods that map to real-world concepts.

Why?

To ensure attributes exist and are easily discoverable.

👇

from dataclasses import dataclass
from enum import Enum
from uuid import UUID


class OrderStatus(str, Enum):
    PLACED = "PLACED"
    CANCELED = "CANCELED"
    FULFILLED = "FULFILLED"


# bad
@dataclass
class Order:
    status: OrderStatus


class CancelOrder:
    def __init__(self, order_repository):
        self.order_repository = order_repository

    def execute(self, order_id: UUID):
        order = self.order_repository.get_by_id(order_id)
        order.status = OrderStatus.CANCELED
        self.order_repository.save(order)


# better
class Order:
    def __init__(self, status: OrderStatus):
        self._status = status

    def cancel(self):
        self._status = OrderStatus.CANCELED


class CancelOrder:
    def __init__(self, order_repository):
        self.order_repository = order_repository

    def execute(self, order_id: UUID):
        order = self.order_repository.get_by_id(order_id)
        order.cancel()
        self.order_repository.save(order)

Python - OOP tip: avoid using too many attributes on a single object

Python Clean Code Tip:

Avoid using too many attributes on a single object. Try to cluster them to improve cohesion, reduce coupling, and improve readability

👇

import datetime
from dataclasses import dataclass


# bad
@dataclass
class ExcelSheet:
    file_name: str
    file_encoding: str
    document_owner: str
    document_read_password: str
    document_write_password: str
    creation_time: datetime.datetime
    update_time: datetime.datetime


# good
@dataclass
class FileProperties:
    name: str
    encoding: str


@dataclass
class SecurityProperties:
    owner: str
    read_password: str
    write_password: str


@dataclass
class DocumentDating:
    creation_time: datetime.datetime
    update_time: datetime.datetime


@dataclass
class ExcelSheet:
    file_properties: FileProperties
    security_properties: SecurityProperties
    document_dating: DocumentDating

Do not use bare except

Python Clean Code Tip:

Avoid empty except blocks -> try-except-pass.

They lead to hard-to-find bugs.

👇

# bad
import logging


def send_email():
    print("Sending email")
    raise ConnectionError("Oops")


try:
    send_email()
except:  # AVOID THIS
    pass


# better
logger = logging.getLogger(__name__)
try:
    send_email()
except ConnectionError as exc:
    logger.error(f"Cannot send email {exc}")

Python - use all uppercase for constants

Python Clean Code Tip:

Use upper case names for constants

👇

from typing import Final

MAX_NUMBER_OF_RETRIES: Final = 666


class Driver:
    MAX_HEIGHT: Final = 190

Python type annotation specificity

Python tip:

Specify the most general type for inputs and the most specific type for outputs.

For example:

from typing import List


def sum_of_elements(elements: List[int]) -> int:
    sum_el = 0

    for element in elements:
        sum_el += element

    return sum_el


print(sum_of_elements((9, 9)))

"""
$ mypy example.py

example.py:13: error: Argument 1 to "sum_of_elements" has 
incompatible type "Tuple[int, int]"; expected "List[int]"
Found 1 error in 1 file (checked 1 source file)
"""

from typing import Iterable


def sum_of_elements(elements: Iterable[int]) -> int:
    sum_el = 0

    for element in elements:
        sum_el += element

    return sum_el


print(sum_of_elements((9, 9)))

"""
$ mypy example.py

Success: no issues found in 1 source file
"""

Python: Check if an iterable contains a specific element

Python Clean Code Tip:

Use in to check whether an iterable contains a specific element.

👇

lucky_numbers = [1, 23, 13, 1234]
BEST_NUMBER = 13


# without in
best_number_is_lucky_number = False

for number in lucky_numbers:
    if number == BEST_NUMBER:
        best_number_is_lucky_number = True

print(best_number_is_lucky_number)
# => True


# with in
best_number_is_lucky_number = BEST_NUMBER in lucky_numbers
print(best_number_is_lucky_number)
# => True

Python type hints - descriptive variable names

Python Clean Code Tip:

Avoid using the variable/parameter type inside your variable/parameter name. Use type hints instead.

# BAD: user_list

# GOOD: users: list[User]

Full example👇

from dataclasses import dataclass


@dataclass
class User:
    username: str


# bad
def print_users(user_list):
    for user in user_list:
        print(user.username)


print_users([User(username="johndoe")])
# => johndoe


# good
def print_users(users: list[User]):
    for user in users:
        print(user.username)


print_users([User(username="johndoe")])
# => johndoe

Python - avoid HTTP status code magic numbers with http.HTTPStatus()

Python Clean Code Tip:

Use HTTPStatus from http (it's inside the standard library) to avoid "magic" numbers for statuses inside your code.

Example:

from http import HTTPStatus

from fastapi import FastAPI

app = FastAPI()


@app.get("/old", status_code=200)
async def old():
    return {"message": "Hello World"}


@app.get("/", status_code=HTTPStatus.OK)
async def home():
    return {"message": "Hello World"}

Python - splitting a module into multiple files

Python Clean Code Tip:

When your module becomes too big you can restructure it to a package while keeping all the imports from the module as they were.

👇

# BEFORE

# models.py
class Order:
    pass


class Shipment:
    pass


# └── models.py


# AFTER

# change to package
# models/__init__.py
from .order import Order
from .shipment import Shipment

__all__ = ["Order", "Shipment"]


# models/order.py
class Order:
    pass


# models/shipment.py
class Shipment:
    pass


# └── models
#     ├── __init__.py
#     ├── order.py
#     └── shipment.py


# imports from module/package can stay the same
from models import Order, Shipment

Design by contract in Python - preconditions

Python Clean Code Tip:

Use preconditions to ensure the integrity of your objects.

For example:

class Date:
    def __init__(self, day, month, year):
        self.day = day
        self.month = month
        self.year = year


startDate = Date(3, 11, 2020)
# OK

startDate = Date(31, 13, 2020)
# this one should fail since there are only 12 months


class Date:
    LAST_MONTH = 12
    LAST_DAY = 31

    def __init__(self, day, month, year):
        if month > self.LAST_MONTH:
            raise Exception(f"Month cannot be greater than {self.LAST_MONTH}")
        if day > self.LAST_DAY:
            raise Exception(f"Day cannot be greater than {self.LAST_DAY}")
        self.day = day
        self.month = month
        self.year = year


startDate = Date(3, 11, 2020)
# OK

startDate = Date(31, 13, 2020)
# this one fails


# DISCLAIMER: production ready validation should be more complex since not all months have 31 days

Operator Overloading in Python

Python Clean Code Tip:

Use operator overloading to enable usage of operators such as +, -, /, *, ... on your instances.

👇

from dataclasses import dataclass


# without operator overloading
@dataclass
class TestDrivenIOCoin:
    value: float

    def add(self, other):
        if not isinstance(other, TestDrivenIOCoin):
            return NotImplemented

        return TestDrivenIOCoin(value=self.value + other.value)


my_coins = TestDrivenIOCoin(value=120).add(TestDrivenIOCoin(value=357.01))
print(my_coins)
# TestDrivenIOCoin(value=477.01)


# with operator overloading
@dataclass
class TestDrivenIOCoin:
    value: float

    def __add__(self, other):
        if not isinstance(other, TestDrivenIOCoin):
            return NotImplemented

        return TestDrivenIOCoin(value=self.value + other.value)


my_coins = TestDrivenIOCoin(value=120) + TestDrivenIOCoin(value=357.01)
print(my_coins)
# TestDrivenIOCoin(value=477.01)

Chaining comparison operators in Python

Python Clean Code Tip:

Use chained comparison when you need to check whether some variable is between MIN and MAX values.

👇

from dataclasses import dataclass


@dataclass
class SurfBoard:
    width: float
    length: float


MINIMAL_LENGTH = 201.3
MAXIMAL_LENGTH = 278.5


# without chained comparison
def board_is_pwa_compliant(surf_board: SurfBoard):
    return surf_board.length > MINIMAL_LENGTH and surf_board.length < MAXIMAL_LENGTH


surf_board = SurfBoard(width=75.3, length=202.7)
print(board_is_pwa_compliant(surf_board))
# True


# with chained comparison
def board_is_pwa_compliant(surf_board: SurfBoard):
    return MINIMAL_LENGTH < surf_board.length < MAXIMAL_LENGTH


print(board_is_pwa_compliant(surf_board))
# True


# don't abuse it like this: a <= b < c > d

__all__ in Python

Python Clean Code Tip:

Use __all__ to define exported members of your package.

Hint: IDEs will do a much better job at importing and autocomplete.

from .my_module import my_function

__all__ = ["my_function"]

Python - built-in sum function vs. for loop

Python Clean Code Tip:

Use sum to sum the values of all elements inside an iterable instead of a for loop.

Why?

1. Don't re-invent the wheel!
2. sum is much faster

👇

transactions = [10.0, -5.21, 101.32, 1.11, -0.38]

# without sum
balance = 0


for transaction in transactions:
    balance += transaction


# with sum
balance = sum(transactions)

Python - Reduce Boilerplate Code with Dataclasses

Python Clean Code Tip:

Use dataclasses when only storing attributes inside your class instances to reduce the amount of boilerplate code.

For example:

# without dataclass
class Address:
    def __init__(self, street, city, zip_code):
        self.street = street
        self.city = city
        self.zip_code = zip_code

    def __repr__(self):
        return (
            f"Address(street={self.street}, city={self.city}, zip_code={self.zip_code})"
        )

    def __hash__(self) -> int:
        return hash((self.street, self.city, self.zip_code))

    def __eq__(self, other) -> bool:
        if not isinstance(other, Address):
            return NotImplemented
        return (self.street, self.city, self.zip_code) == (
            other.street,
            other.city,
            other.zip_code,
        )


# with dataclass
from dataclasses import dataclass

@dataclass(unsafe_hash=True)
class Address:
    street: str
    city: str
    zip_code: str

Check for code quality issues inside your CI/CD pipelines

Python Clean Code Tip:

Check the quality of your code inside your CI pipeline.

Use:

1. flake8 - style guide enforcer
2. black - code formatting
3. isort - optimize imports
4. bandit - check for security vulnerabilities
5. safety - check for security vulnerabilities of dependencies

Github Actions Example 👇

name: Check code quality
on: [push]

jobs:
  code-quality:
    strategy:
      fail-fast: false
    matrix:
      python-version: [3.9]
      poetry-version: [1.1.8]
      os: [ubuntu-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v2
      - uses:
        with:   actions/setup-python@v2
           python-version:   ${{ matrix. python-version }}
      - name: Run image
        uses: abatilo/[email protected]
        with:
          poetry-version: ${{ matrix. poetry-version }}
      - name: Install dependencies
        run: poetry install
      - name: Run black
        run: poetry run black . --check
      - name: Run isort
        run: poetry run isort . --check-only --profile black
      - name: Run flake8
        run: poetry run flake8 .
      - name: Run bandit
        run: poetry run bandit .
      - name: Run saftey
        run: poetry run safety check

It's a good idea to couple this with pre-commit hooks:

• pre-commit - format code with black and isort
• CI pipeline - run black and isort with check flags to ensure that code has been properly formatted

In other words, you shouldn't actually format any code in the CI pipeline. You just want to verify that formatting happened via pre-commit.

Don't use flags in functions

Python Clean Code Tip:

Don't use flags in functions.

Flags are variables passed to functions, which the function uses to determine its behavior. This pattern should be avoided since functions should only perform a single task. If you find yourself doing this, split your function into smaller functions.

👇

text = "This is a cool blog post"

# This is bad
def transform(text, uppercase):
    if uppercase:
        return text.upper()
    else:
        return text.lower()


# This is good
def uppercase(text):
    return text.upper()


def lowercase(text):
    return text.lower()

Python Clean Code: Keep your function arguments at a minimum

Python Clean Code Tip:

Keep your arguments at a minimum.

Ideally, your functions should only have one to two arguments. If you need to provide more arguments to the function, you can create a config object which you pass to the function or split it into multiple functions.

Example:

# This is bad
def render_blog_post(title, author, created_timestamp, updated_timestamp, content):
    # ...

render_blog_post("Clean code", "Nik Tomazic", 1622148362, 1622148362, "...")


# This is good
class BlogPost:
    def __init__(self, title, author, created_timestamp, updated_timestamp, content):
        self.title = title
        self.author = author
        self.created_timestamp = created_timestamp
        self.updated_timestamp = updated_timestamp
        self.content = content

blog_post1 = BlogPost("Clean code", "Nik Tomazic", 1622148362, 1622148362, "...")

def render_blog_post(blog_post):
    # ...

render_blog_post(blog_post1)

Functions should only perform a single task

Python Clean Code Tip:

Functions should only perform a single task

Hint: If your function contains the keyword 'and' you can probably split it into two functions.

# This is bad
def fetch_and_display_personnel():
    data = # ...

    for person in data:
        print(person)


# This is good
def fetch_personnel():
    return # ...

def display_personnel(data):
    for person in data:
        print(person)

Clean code tip - Don't add unnecessary context

Python Clean Code Tip:

Don't add redundant context.

Do not add unnecessary data to variable names, especially if you're working with classes.

# This is bad
class Person:
    def __init__(self, person_first_name, person_last_name, person_age):
        self.person_first_name = person_first_name
        self.person_last_name = person_last_name
        self.person_age = person_age


# This is good
class Person:
    def __init__(self, first_name, last_name, age):
        self.first_name = first_name
        self.last_name = last_name
        self.age = age

We're already inside the Person class, so there's no need to add a person_ prefix to every class variable.

Clean code tip - Don't use magic numbers

Python Clean Code Tip:

Don't use "magic numbers".

Magic numbers are strange numbers that appear in code, which do not have a clear meaning.

👇

import random

# This is bad
def roll():
    return random.randint(0, 36)  # what is 36 supposed to represent?

# This is good
ROULETTE_POCKET_COUNT = 36

def roll():
    return random.randint(0, ROULETTE_POCKET_COUNT)

Instead of using magic numbers, extract them into a meaningful variable.

Clean code tip - Avoid using ambiguous abbreviations

Python clean code tip:

Avoid using ambiguous abbreviations

Don't try to come up with your own abbreviations. It's better for a variable to have a longer name than a confusing name.

👇

# This is bad
fna = 'Bob'
cre_tmstp = 1621535852

# This is good
first_name = 'Bob'
creation_timestamp = 1621535852

Queryset.explain() in Django

Django tip:

If you want to know how the database would execute a given query, you can use explain().

Knowing this can be helpful when you're trying to improve the performance of slow queries.

>>> print(Payment.objects.filter(created_at__gt=datetime.date(2021, 1, 1)).explain())

Seq Scan on payments_payment  (cost=0.00..14.25 rows=113 width=212)
    Filter: (created_at > '2021-01-01 00:00:00+00'::timestamp with time zone)

Django templates - lorem ipsum

Django tip:

You can generate lorem ipsum inside a Django template with the lorem tag.

https://docs.djangoproject.com/en/3.2/ref/templates/builtins/#lorem

You can provide any (or none) of the following arguments:

1. count - number of paragraphs or words
2. method - words/HTML paragraphs/plain-text paragraphs
3. random - doesn't use the common paragraph ("Lorem ipsum dolor sit amet...")

Example:

{% lorem 6 p random &}

# generates 6 paragraphs of text that doesn't
# start with "Lorem ips um dolor sit amet"

Django - update_or_create

Django's update_or_create() method either-

• updates an existing object with the given kwargs along with a defaults dictionary of pairs for updating the object
• creates a new object if it doesn't exist

It returns a tuple containing an object and a boolean specifying whether a new object was created.

Visitor.objects.create(name="Harry", surname="Potter", age=16)

visitor, created = Visitor.objects.update_or_create(
    name="Harry", surname="Potter", defaults={"age": 21}
)

print(visitor.age)
# => '21'

print(created)
# => False

Django - get_or_create

Django's get_or_create() method either-

• gets an existing object with the given kwargs
• creates a new object if it doesn't exist

It returns a tuple containing an object and a boolean specifying whether a new object was created.

Visitor.objects.create(name="Harry", surname="Potter", age=16)

visitor, created = Visitor.objects.get_or_create(
    name="Harry", surname="Potter", age=16
)
print(created)
# => False

visitor, created = Visitor.objects.get_or_create(
    name="Hermione", surname="Granger", age=16
)
print(created)
# => True

Django QuerySet - only() vs defer() vs exclude()

If you have some fields in your Django model that contain a lot of data and you don't need those fields for a particular query, you can tell Django not to retrieve them with defer():

Event.objects.defer("description")

While defer() works at the attribute level, exclude() works on the row level.

In other words, exclude() arguments are used after the WHERE clause -- i.e., SELECT * FROM users WHERE name = 'jan' -- while defer() changes * to the provided fields -- i.e., SELECT name, email FROM users.

Opposite to defer() is only(). If you have fewer fields that you want to retrieve than those you don't, you can use only() to retrieve only the fields provided as arguments:

Event.objects.only("title")

Check if a file is a symlink in Python

Python tip:

You can use pathlib's is_symlink() to check whether a path is a symlink.

👇

import pathlib

path = pathlib.Path("/usr/bin/python")

print(path.is_symlink())
# => True

Using Django's get_object_or_404 shortcut

Django tip:

You can use get_object_or_404 to raise the Http404 exception when the object doesn't exist instead of handling DoesNotExist and raising Http404 by yourself.

👇

from django.http import Http404
from django.shortcuts import get_object_or_404


def my_view(request):
    obj = get_object_or_404(MyModel, pk=1)


# the above is equivalent to
def my_view(request):
    try:
        obj = MyModel.objects.get(pk=1)
    except MyModel.DoesNotExist:
        raise Http404("No MyModel matches the given query.")

Find the union of two Django querysets

Django tip:

You can use | to create a union of multiple queries.

👇

by_username = User.objects.filter(username="John")
by_name = User.objects.filter(full_name="John")
users = by_username | by_name

Mock AWS Services

Pytest tip:

Use moto to mock AWS services such as S3 and DynamoDB:

https://docs.getmoto.org/

👇

import bot03
import pytest
from moto import mock_dynamodb2


@pytest.fixture
def dynamodb_table():
    with mock_dynamodb2():
        dynamodb = bot03.resource("dynamodb")

        table = dynamodb.create_table(
            TableName="test",
            KeySchema=[
                {"AttributeName": "PK", "KeyType": "HASH"},
                {"AttributeName": "SK", "KeyType": "Range"},
            ],
            AttributeDefinitions=[
                {"AttributeName": "PK", "AttributeType": "S"},
                {"AttributeName": "SK", "AttributeType": "S"},
                {"AttributeName": "GSIPK", "AttributeType": "S"},
                {"AttributeName": "GSISK", "AttributeType": "S"},
            ],
            GlobalSecondarylndexes=[
                {
                    "IndexName": "GS1",
                    "KeySchema": [
                        {"AttributeName": "GS1PK", "KeyType": "HASH"},
                        {"AttributeName": "GS1SK", "KeyType": "Range"},
                    ],
                    "Projection": {"ProjectionType": "ALL"},
                },
            ],
        )

        table.delete()

Django REST Framework - Combining and Excluding Permission Classes

Did you know?

You can combine permissions in Django REST Framework using &, |, and ~.

👇

class MyModelViewSet(viewsets.ModelViewSet):
    permission_classes = IsAuthenticated & (IsAdminUser | IsFaculty | ReadOnly)


class MyModelViewSet(viewsets.ModelViewSet):
    permission_classes = ~IsStudent & IsAuthenticated

For more, check out the Combining and Excluding Permission Classes section from Custom Permission Classes in Django REST Framework.

Asynchronous Background Tasks in FastAPI

FastAPI tip:

You can use FastAPI's BackGround Tasks to run simple tasks in the background.

👇

from fastapi import BackgroundTasks


def send_email(email, message):
    pass


@app.get("/")
async def ping(background_tasks: BackgroundTasks):
    background_tasks.add_task(send_email, "[email protected]", "Hi!")
    return {"message": "pong!"}

Use Celery for CPU intensive tasks and when you need a task queue.

Python - slice a generator object

Python tip:

You can use itertools.islice to use only part of a generator.

👇

from itertools import cycle, islice

chord_sequence = cycle(["G", "D", "e", "C"])

song_chords = [chord for chord in islice(chord_sequence, 16)]

print(song_chords)
"""
['G', 'D', 'e', 'C', 'G', 'D', 'e', 'C', 'G', 'D', 'e', 'C', 'G', 'D', 'e', 'C']
"""

Flask - async and await

Flask tip:

With Flask >= 2.0 you can create asynchronous route handlers using async/await.

Example:

import asyncio


async def async_get_data():
    await asyncio.sleep(1)
    return "Done!"


@app.route("/data")
async def get_data():
    data = await async_get_data()
    return data

Want to learn more? Check out Async in Flask 2.0.

Calculate the execution time of Flask views

Did you know?

You can use a decorator to time the execution of Flask views.

For example👇

from functools import wraps
from timeit import default_timer


def timer(f):
    @wraps(f)
    def wrapper(*args, **kwargs):
        start_time = default_timer()
        response = f(*args, **kwargs)
        total_elapsed_time = default_timer() - start_time
        response += f"<h3>Elapsedtime: {total_elapsed_time}</h3>"
        return response

    return wrapper


@app.route("/")
@timer
def hello_world():
    return "Hello World!"

Positional-only arguments in Python

Did you know?

You can force a user to call a function with positional arguments only using /.

Example:

def full_name(user, /):
    return f"{user['first_name']} {user['last_name']}"


print(full_name({"first_name": "Jan", "last_name": "Giamcomelli"}))
# => Jan Giamcomelli

print(full_name(user={"first_name": "Jan", "last_name": "Giamcomelli"}))
# => TypeError: full_name() got some positional-only arguments passed as keyword arguments: 'user'

Why?

Makes refactoring easier. You can change the name of your parameters without worrying about it breaking any code that uses the function.

all() in Python

Python tip:

You can use all to check whether all values inside an iterable are truthy.

👇

numbers = [10, 99, 321, 3]
print(all(number > 0 for number in numbers))
# => True

print(all([]))
# => True

numbers = [-1, 2, 5, 10, 0]
print(all(number > 0 for number in numbers))
# => False

any() in Python

Python tip:

You can use any to check whether any element inside an iterable has a truthy value.

An example👇

platforms = ["Facebook", "Twitter", "Instagram"]

print(any(platform == "Twitter" for platform in platforms))
# => True

print(any([]))
# => False

print(any([2, 3, 4]))
# => True

Parse datetime strings in Python with dateutil

Did you know?

You can use dateutil.parser to parse all sorts of different date and datetime string formats.

https://pypi.org/project/python-dateutil/

An example👇

from dateutil.parser import parse

print(parse("2012-01-19 17:21:00"))
# => 2012-01-19 17:21:00

print(parse("1st Jan, 2019"))
# => 2019-01-01 00:00:00

print(parse("23.11.2020"))
# => 2020-11-23 00:00:00

Unfortunately, this is quite slow:

from dateutil.parser import parse
%timeit parse(datetime_txt).date()
54.3 µs ± 450 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

import datetime
%timeit datetime.datetime.strptime(datetime_txt, "%H:%M %d-%m-%Y").date()
7.44 µs ± 240 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

Python @classmethod

Python tip:

You can use @classmethod to create class methods.

For example, you can create a class method that loads events from a JSON string message👇

import datetime
import json


class UserRegistered:
    def __init__(self, username, event_time):
        self.username = username
        self.event_time = event_time

    @classmethod
    def from_event_message(cls, message):
        message = json.loads(message)

        return cls(
            username=message["username"],
            event_time=datetime.datetime.fromisoformat(message["event_time"]),
        )


message = '{"username": "johndoe", "event_time": "2021-04-26T20:00:00"}'
event = UserRegistered.from_event_message(message)
print(event.username, event.event_time)
# => johndoe 2021-04-26 20:00:00

5 awesome tools for writing Python tests

5 awesome tools for writing Python tests:

1. pytest - go-to testing framework for testing Python code
2. unittest.mock - built-in mocking library
3. coverage.py - measuring code coverage (use pytest-cov plugin when using pytest)
4. mutmut - improve the quality of your test suite with mutation testing
5. hypothesis - property-based testing library

You can learn about all of these tools here.

How do you parameterize a test in pytest?

Pytest tip:

You can parametrize your tests with @pytest.mark.parametrize.

Example:

import pytest


def is_palindrome(text):
    return text == "".join(reversed(text))


@pytest.mark.parametrize(
    "text, is_pal",
    [
        ("kayak", True),
        ("racecar", True),
        ("bmw", False),
        ("songs", False),
    ],
)
def test_is_palindrome(text, is_pal):
    assert is_palindrome(text) == is_pal

How can I implement a custom error handler in Flask?

Did you know?

You can register exception handlers to a Flask app based on an exception class or response status code.

An example👇

from flask import Flask, jsonify, abort

app = Flask(__name__)


class ValidationException(Exception):
    code = 500
    message = "Unknown error"


@app.errorhandler(ValidationException)
def handle_validation_exception(exc):
    return (
        jsonify({"msssage": exc.message, "exception": exc.__class__.__name__}),
        exc.code,
    )


@app.errorhandler(500)
def handle_internal_server_error(exc):
    return jsonify({"msssage": "Oops!", "exception": "Internal server error"}), 500


@app.route("/")
def hello():
    raise ValidationException()

Custom field validators in Django

Did you know?

In Django, you can add custom validators to your model fields

For example, you can validate that price is always greater than 0👇

from django.core.exceptions import ValidationError
from django.db import models
from django.utils.translation import gettext_lazy as _


def validate_greater_than_zero(value):
    if value <= 0:
        raise ValidationError(
            _("%(value)s is not greater than zero."),
            params={"value": value},
        )


class Book(models.Model):
    title = models.CharField(max_length=120)
    price = models.DecimalField(validators=[validate_greater_than_zero])

pydantic typecasting

Did you know?

Pydantic uses typecasting.

That means that it tries to convert provided values to their expected types if it's possible.

Examples👇

import datetime

from pydantic import BaseModel


class Location(BaseModel):
    city: str
    country: str


class Measurement(BaseModel):
    temperature: float
    measured_at: datetime.datetime
    location: Location


print(
    Measurement(
        temperature=10,
        measured_at="2021-04-16T21:15:03.000012",
        location={"city": "London", "country": "UK"},
    )
)

"""
temperature=10.0 measured_at=datetime.datetime(2021, 4, 16, 21, 15, 3, 12) location=Location(city='London', country='UK')
"""

print(
    Measurement(
        temperature=10.0,
        measured_at="2021-04-16T21:15:03",
        location=Location(city="London", country="UK"),
    )
)

"""
temperature=10.0 measured_at=datetime.datetime(2021, 4, 16, 21, 15, 3) location=Location(city='London', country='UK')
"""

print(
    Measurement(
        temperature=10,
        measured_at="16186000528",
        location=[("city", "London"), ("country", "UK")],
    )
)

"""
temperature=10.0 measured_at=datetime.datetime(2021, 4, 16, 19, 15, 28, tzinfo=datetime.timezone.utc) location=Location(city='London', country='UK')
"""

Public, protected, and private methods ​in Python

Did you know?

Python doesn't have public, protected, or private methods.

Instead, use the following naming conventions:

1. method_name -> public
2. _method_name -> protected
3. __method_name -> private

The same goes for attributes.

An example👇

class User:
    def __init__(self, first_name, last_name, age):
        self.first_name = first_name
        self.last_name = last_name
        self.__age = age

    def age(self):
        return "You shouldn't ask that. It's a private information."

    def _is_adult(self):
        return self.__age >= 21

    def can_drink_alcohol(self):
        return self._is_adult()

Python - checking whether a list contains duplicates

Did you know?

As long as the elements are hashable, you can check whether a list contains duplicated values by converting it to a set and comparing the lengths of the list and set.

Example:

names = ["Jan", "John", "Marry", "Daisy", "Jan"]

names_has_duplicates = len(set(names)) < len(names)
print(names_has_duplicates)
# => True

Black compatibility with flake8

Python tip:

When using black and flake8 together set the max line length for flake8 to 88 to match black's default value.

Minimal flake8 black compatible config:

[flake8]
max-line-length = 88
extend-ignore = E203

Flask Sentry Example

Flask tip:

Add Sentry to your Flask app to track unhandled exceptions.

An example👇

import sentry_sdk
from flask import Flask
from sentry_sdk.integrations.flask import FlaskIntegration

sentry_sdk.init(
    dsn="your-sentry-dsn",
    integrations=[FlaskIntegration()],
    traces_sample_rate=0.3,
    environment="production",
)

app = Flask(__name__)

Python 3.10 - parenthesized context managers

Did you know?

Python 3.10 comes with support for continuation across multiple lines when using context managers.

An example👇

import unittest
from unittest import mock

from my_module import my_function


# Python < 3.10


class Test(unittest.TestCase):
    def test(self):
        with mock.patch("my_module.secrets.token_urlsafe") as a, mock.patch("my_module.string.capwords") as b, mock.patch("my_module.collections.defaultdict") as c:
            my_function()
            a.assert_called()
            b.assert_called()
            c.assert_called()

    def test_same(self):
        with mock.patch("my_module.secrets.token_urlsafe") as a:
            with mock.patch("my_module.string.capwords") as b:
                with mock.patch("my_module.collections.defaultdict") as c:
                    my_function()
                    a.assert_called()
                    b.assert_called()
                    c.assert_called()


# Python >= 3.10

class Test(unittest.TestCase):
    def test(self):
        with (
            mock.patch("my_module.secrets.token_urlsafe") as a,
            mock.patch("my_module.string.capwords") as b,
            mock.patch("my_module.collections.defaultdictl") as c,
        ):
            my_function()
            a.assert_called()
            b.assert_called()
            c.assert_called()

Change a model field name in Django REST Framework

Django tip:

You can rename the model field inside the serializer by using the source for the selected field.

For example, the is_active field from the model can be returned as active👇

# models. py
from django.contrib.auth.models import User
from django.db import models


class UserProfile(models.Model):
    user = models.OneToOneField(to=User, on_delete=models.CASCADE)
    bio = models.TextField()
    birth_date = models.DateField()

    def __str__(self):
        return f"{self.user.username} profile"


# serializers.py
from rest_framework import serializers


class UserSerializer(serializers.ModelSerializer):
    active = serializers.BooleanField(source="is_active")

    class Meta:
        model = User
        fields = ["id", "username", "email", "is_staff", "active"]

Difference between re.search and re.match in Python?

re.match() searches for matches from the beginning of a string while re.search() searches for matches anywhere in the string.

Example:

import re

claim = 'People love Python.'

print(re.search(r'Python', claim).group())
# => Python

print(re.match(r'Python', claim))
# => None

print(re.search(r'People', claim).group())
# => People

print(re.match(r'People', claim).group())
# => People

Extract contents of a PDF file with pikepdf in Python

Python tip:

You can use pikepdf to extract a subset pf pages from an original PDF and create a new file containing only the extracted pages.

For example,👇 save pages 2 and 3 from the original PDF

import pathlib

from pikepdf import Pdf

start = 2
stop = 3
filepath = pathlib.Path("file.pdf")

pdf = Pdf.open(filepath)
new_pdf = Pdf.new()

new_pdf.pages.extend(pdf.pages[start - 1 : stop])

new_pdf.save("new.pdf", compress_streams=False)

Using pathlib's read_bytes method in Python

Python tip:

You can use read_bytes() (which handles the opening and closing of the file) from pathlib's Path to read bytes from a file instead of using with open().

Example:

# pathlib
import pathlib
file_bytes = pathlib.Path("file.pdf").read_bytes()


# with open
with open("file.pdf", "rb") as file:
    file_bytes = file.read()

Skip tests based on a condition in pytest

Pytest tip:

You can skip a test based on a given condition.

For example, you can skip a test when running on Windows

import sys

import pytest


@pytest.mark.skipif(sys.platform == "win32", reason="Not running on Window")
def test_windows():
    assert True

Python - provide a function as an argument to another function to improve its testability

Python tip:

You can provide a function as an argument to another function to improve its testability.

For example, you can provide input as a function in a real application and mock the function in test

def get_user_expense(get_input):
    raw_input = get_input(
        "Enter your expense (date;description;amount) (e.g. 2021-03-21;Gasoline;75.43): "
    )
    date, description, amount = raw_input.split(";")
    return date, description, amount


# test
def test_get_user_expense():
    assert get_user_expense(lambda *args: "2021-03-21;Gasoline;75.43") == (
        "2021-03-21",
        "Gasoline",
        "75.43",
    )


# real application
def main():
    date, description, amount = get_user_expense(input)
    print(date, description, amount)

Poetry - generate requirements.txt without hashes

Poetry tip:

When using Docker, you can export dependencies to a requirements.txt file without hashes to decrease time to resolve dependencies.

👇

poetry export --without-hashes --format=requirements.txt > requirements.txt

Caching poetry install for CI

Poetry tip:

When running poetry inside a CI pipeline, set virtualenvs.in-project config to true.

That way the virtual environment will be created inside the current folder -> you can cache it.

Gitlab example👇

stages:
  - test

cache:
  key: virtualenv
  paths:
    - .venv/

tests:
  stage: test
  image: python3.8-slim
  before_script:
    - poetry config virtualenvs.in-project true
    - poetry install
  script:
    - poetry run python -m pytest tests

Black compatibility with isort

Python tip:

When using black and isort on the same project, use isort's profile black to avoid code formatting conflicts

👇

$ isort . --profile black

Python dictionary clear() method

Python tip:

You can use .clear to clear a dictionary.

Is it the same as foo = {} ?

No.

1. foo = {} will create a new instance but other references will still point to the old items.
2. Because of #1, foo = {} is probably faster than foo.clear()
3. foo.clear() is slower but safer. Be careful with scope with foo = {}.

An example👇

user = {"name": "Jan", "username": "giaco"}

user.clear()
print(user). # => {}

Testing code examples in docstrings with pytest

Pytest tip:

You can test code examples inside your docstrings like so:

$ pytest --doctest-modules http://yourmodule.py

👇

from typing import List


def daily_average(temperatures: List[float]) -> float:
    """
    Get average daily temperature

    Calculate average temperature from multiple measurements

    >>> daily_average( [10.0, 12.0, 14.0])
    12.0

    : param temperatures: list of temperatures
    : return: Average temperature
    """

    return sum(temperatures) / len(temperatures)


# python -m pytest --doctest-modules temperature. py

What’s the difference between '==' and 'is' in Python?

What's the difference between == and is?

Let's find out👇

== is an equality operator. It checks whether two objects are equal by their values. To compare, objects must have an __eq__ method implemented.

my_list = [1, 2, 3]
another_list = [element for element in my_list]
print(my_list == another_list)
# => True


class User:
    def __init__(self, name):
        self.name = name

    def __eq__(self, other):
        return other.name == self.name


user1 = User(name="Jan")
user2 = User(name="Jan")
print(user1 == user2)
# => True

is is an identity operator. It checks whether you're referencing the same object on both sides of the comparison. It doesn't care about the values.

my_list = [1, 2, 3]
another_list = [element for element in my_list]
print(my_list is another_list)
# => False


class User:
    def __init__(self, name):
        self.name = name


user1 = User(name="Jan")
user2 = User(name="Jan")
print(user1 is user2)
# => False
print(user1 is user1)
# => True

When should you use one over the other?

Use== when you care about values and use is when you care about actual objects.

Python - assert that a mock function was called with certain arguments

Python tip:

You can assert that the method on a MagicMock object was called with certain arguments.

https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called_with

For example, you can mock an email service and assert that the send method was called with specific values:

from unittest.mock import MagicMock


def place_order(user, items, email_service):
    print(f"Place order for user {user}")
    print(f"With items: {items}")
    email_service.send(user, "Order placed.")


def test_place_order():
    email_service = MagicMock()

    place_order("[email protected]", ["Computer", "USB drive", "Vodka"], email_service)
    email_service.send.assert_called_with("[email protected]", "Order placed.")

Python - unittest.mock.create_autospec() example

Python tip:

You can use mock.create_autospec to create a mock object with the same interface as the provided class, function, or method.

For example, you can use it to mock Response from the requests package👇

from unittest import mock

import requests
from requests import Response


def get_my_ip():
    response = requests.get(" http://ipinfo.to/json")
    return response.json()["ip"]


def test_get_my_ip(monkeypatch):
    my_ip = "123.123.123.123"
    response = mock.create_autospec(Response)
    response.json.return_value = {"ip": my_ip}
    monkeypatch.setattr(requests, "get", lambda *args, **kwargs: response)
    assert get_my_ip() == my_ip

Managing session data in Flask

Flask tip:

Setting a value to Flask's session is as simple as:

session['key'] = 'value'

https://testdriven.io/blog/flask-sessions/

For example:

from flask import Flask, redirect, request, session, url_for

# Create the Flask application
app = Flask(__name__)

app.secret_key = "BAD SECRET KEY"


@app.route("/set_email", methods=["GET", "POST"])
def set_email():
    if request.method == "POST":
        # Save the form data to the session object
        session["email"] = request.form["email_address"]
        print(session["email"])
        return redirect(url_for("set_email"))
    return """
        <form method="post">
        <label for="email">Enter your email address:</label>
        <input name="email_address" required / >
        <button type="submit">Submit</button
        </form>
        """

Getting started with Python type hints

Python tip:

Use type hints to annotate expected types for variables, function parameters, and function returns.

https://testdriven.io/blog/python-type-checking/

An example👇

from typing import Collection


def daily_average(temperatures: Collection[float]) -> float:
    return sum(temperatures) / len(temperatures)


print(daily_average.__annotations__)
# => {'temperatures': typing.Collection[float], 'return': <class 'float'>}

Teaching kids programming with Python Turtle

Python tip:

Turtle graphics is a popular way to introduce programming to kids.

You can use it to draw different shapes. The cursor moves on the screen.

An example👇

from turtle import *

color("red", "yellow")
begin_fill()

while True:
    forward(200)
    left(170)
    if abs(pos()) < 1:
        break

end_fill()
done()

Python Testing - Given When Then syntax

TDD tip:

You can add GIVEN, WHEN, THEN as a docstring to your tests to better express your intent.

An example👇

def test_all(self):
    """
    GIVEN a PhoneBook with a records property
    WHEN the all method is called
    THEN all numbers should be returned in ascending order
    """

    phone_book = PhoneBook(
        records=[
            ("John Doe", "03 234 567 890"),
            ("Marry Doe", "01 234 567 890"),
            ("Donald Doe", "02 234 567 890"),
        ]
    )

    previous = ""

    for record in phone_book.all():
        assert record[0] > previous
        previous = record[0]

Django deployment checklist

Django tip:

Check your production settings.py file for security vulnerabilities with the check command:

$ ./manage.py check --deploy

https://docs.djangoproject.com/en/3.2/ref/django-admin/#check

Example:

$ ./manage.py check --deploy
System check identified some issues:

WARNINGS:
have not set CSRF COOKIE SECURE to True. Using a secure-only CSRF cookie
makes it more difficult for network traffic sniffers to steal the CSRF token.
? (security.W018) You should not have DEBUG set to True in deployment.
? (security.W022) You have not set the SECURE_REFERRER_POLICY setting.
Without this, your site will not send a Referrer-Policy header.
You should consider enabling this header to protect user privacy.

Test and publish your Python packages to PyPI with poetry and GitHub Actions

Python tip:

Use poetry and Github Actions to:

1. run tests for your packages - on every push
2. check code quality - on every push
3. publish the packages to PyPI - on every release

https://testdriven.io/blog/python-project-workflow/

An example👇

Test and check for code quality:

name: Push
on: [push]

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        python-version: [3.9]
        poetry-version: [1.1.2]
        os: [ubuntu-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Run image
        uses: abatilo/[email protected]
        with:
          poetry-version: ${{ matrix.poetry-version }}
      - name: Install dependencies
        run: poetry install
      - name: Run tests
        run: poetry run pytest --cov=./ --cov-report=xml
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v1
  code-quality:
    strategy:
      fail-fast: false
      matrix:
        python-version: [3.9]
        poetry-version: [1.1.2]
        os: [ubuntu-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Run image
        uses: abatilo/[email protected]
        with:
          poetry-version: ${{ matrix.poetry-version }}
      - name: Install dependencies
        run: poetry install
      - name: Run black
        run: poetry run black . --check
      - name: Run isort
        run: poetry run isort . --check-only
      - name: Run flake8
        run: poetry run flake8 .
      - name: Run bandit
        run: poetry run bandit .
      - name: Run saftey
        run: poetry run safety check

Release:

name: Release
on:
  release:
    types:
      - created

jobs:
  publish:
    strategy:
      fail-fast: false
      matrix:
        python-version: [3.9]
        poetry-version: [1.1.2]
        os: [ubuntu-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Run image
        uses: abatilo/[email protected]
        with:
          poetry-version: ${{ matrix.poetry-version }}
      - name: Publish
        env:
          PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
        run: |
          poetry config pypi-token.pypi $PYPI_TOKEN
          poetry publish --build

Finding memory leaks in Python with tracemalloc

Python tip:

You can use tracemalloc to display files allocating the most memory.

An example👇

import tracemalloc

import app

tracemalloc.start()

app.run()

snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics("lineno")

for stat in top_stats[:10]:
    print(stat)

Unpacking iterables in Python

Python tip:

You can unpack any iterable to variables as long as the number of variables is the same as the number of elements in the iterable.

Examples👇

a, b, c = (1, 2, 3)
print(a, b, c)
# => 1 2 3

a, b, c, d = (1, 2, 3, 4)
print(a, b, c, d)
# => 1 2 3 4

a, b = {1, 2}
print(a, b)
# => 1 2

a, b = {'a': 1, 'b': 2}
print(a, b)
# => a b

a, b, c = (i for i in range(1, 4))
print(a, b, c)
# => 1 2 3

a, b, c = (1, 2)
print(a, b, c)
# => ValueError: not enough values to unpack (expected 3, got 2)

a, b, c = (1, 2, 3, 4)
print(a, b, c)
# => ValueError: too many values to unpack (expected 3)

Python - quick tkinter example

Python tip:

You can use tkinter to build simple desktop applications

An example👇

import tkinter as tk


class Application(tk.Frame):
    def __init__(self, master=None):
        self.number_of_clicks = tk.IntVar(value=0)

        super().__init__(master)
        self.master = master
        self.pack()
        self.create_widgets()

    def create_widgets(self):
        self.number_of_clicks_value = tk.Label(self, textvariable=self.number_of_clicks)
        self.number_of_clicks_value.pack(side="top")

        self.click_me = tk.Button(self, text="Click me", command=self.increment)
        self.click_me.pack(side="top")

        self.quit = tk.Button(self, text="QUIT", fg="red", command=self.master.destroy)
        self.quit.pack(side="bottom")

    def increment(self):
        self.number_of_clicks.set(self.number_of_clicks.get() + 1)


root = tk.Tk()
app = Application(master=root)
app.mainloop()

Call a function after some interval in Python with Timer

Python tip:

You can use Timer to run some function only after a certain amount of time has passed

An example👇

from threading import Timer


def truth():
    print("Python rocks!")


t = Timer(15, truth)
t.start() # truth will be called after a 15 second interval

Mutable default parameter values in Python

Python tip:

Avoid setting mutable default parameters for functions. Instead, use None as the default and assign the mutable value inside the function.

Why?

Python evaluates a function's default parameters once, not each time the function is called. So, if you allow a mutable default parameter and mutate it, it will be mutated for all future calls.

An example👇

# wrong
def add_jan(users=[]):
    users.append("Jan")
    return users


print(add_jan())  # => ['Jan']
print(add_jan())  # => ['Jan', 'Jan']


# right
def add_jan(users=None):
    users = users or []
    users.append("Jan")
    return users


print(add_jan())  # => ['Jan']
print(add_jan())  # => ['Jan']

Textwrap - text wrapping in Python

Python tip:

You can wrap long strings into multiple lines where each line is shorter than the limit by using textwrap.wrap.

An example👇

import textwrap

text = (
    "The single-responsibility principle (SRP) is a computer programming principle "
    "that states that every class in a computer program should have responsibility over "
    "a single part of that program's functionality, which it should encapsulate. "
    "All of the module's, class' or function's services should be narrowly aligned with that responsibility"
)
lines = textwrap.wrap(text, 70)

for line in lines:
    print(line)

"""
The single-responsibility principle (SRP) is a computer programming
principle that states that every class in a computer program should
have responsibility over a single part of that program's
functionality, which it should encapsulate. All of the module's,
class' or function's services should be narrowly aligned with that
responsibility
"""

How do I extract a tar file in Python?

Python tip:

You can extract a tar archive to a destination folder using tarfile.

An example👇

import tarfile

with tarfile.open("example.tar.gz") as tar:
    tar.extractall("destination_folder")

Arithmetic, Geometric, and Harmonic Means in Python

Python tip:

Use statistics.mean to calculate the arithmetic mean of an iterable.

Use statistics.geometric_mean to calculate the geometric mean of an iterable (Python >= 3.8).

Use statistics.harmonic_mean to calculate the harmonic mean of an iterable.

An example👇

from statistics import geometric_mean, harmonic_mean, mean

print(mean([1, 3, 7, 2, 5]))
# => 3.6

print(geometric_mean([1, 3, 7, 2, 5]))
# => 2.913693458576192

print(harmonic_mean([1, 3, 7, 2, 5]))
# => 2.297592997811816

Python - execute shell commands in subprocess

Python tip:

Use the subprocess module to spawn new processes.

https://docs.python.org/3/library/subprocess.html#subprocess.Popen

Use Popen.communicate to capture result and errors.

An example👇

import subprocess

proc = subprocess.Popen(
    ["ls", "-la", "/home/johndoe/Images"],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
)
output, err = proc.communicate()

if err:
    raise Exception(err.decode())

print(output.decode())
"""
drwxrwxr-x   2 johndoe  johndoe    4096 Sep 28 08:17 .
drwxr-xr-x  13 johndoe  johndoe    4096 Sep 28 08:12 ..
-rw-rw-r--   1 johndoe  johndoe  115887 Sep 19 09:15 1_initial_user.png
-rw-rw-r--   1 johndoe  johndoe  115055 Sep 19 09:15 2_change_ban.png
-rw-rw-r--   1 johndoe  johndoe  114587 Sep 19 09:15 4_change_database.png
-rw-rw-r--   1 johndoe  johndoe  188718 Sep 19 09:15 5_srp.png
-rw-rw-r--   1 johndoe  johndoe  188718 Sep 19 09:15 change_charge.png
"""

Limit execution time of a function call in Python

Python tip:

You can limit the execution time of a function by using the signal library.

An example👇

import signal
import time


def handle_timeout(signum, frame):
    raise TimeoutError


def my_task():
    for i in range(6):
        print(f"I am working something long running, step {i}")
        time.sleep(1)


signal.signal(signal.SIGALRM, handle_timeout)
signal.alarm(5)  # 5 seconds

try:
    my_task()
except TimeoutError:
    print("It took too long to finish the job")
finally:
    signal.alarm(0)


"""
I am working something long running, step 0
I am working something long running, step 1
I am working something long running, step 2
I am working something long running, step 3
I am working something long running, step 4
It took too long to finish the job
"""

Generate a password in Python

Python tip:

To generate a random password you can use the secrets module.

For example, you can generate a random password of length 21 containing letters, numbers, and special characters👇

import secrets
import string

alphabet = string.ascii_letters + string.digits + '!"#$%&/()=?*+|@{}[]'
password = "".join(secrets.choice(alphabet) for i in range(21))

print(password)
# => *cgsV!2LN|8f7)EzVR]eX