Tips and Tricks

Parse datetime strings in Python with dateutil

Did you know?

You can use dateutil.parser to parse all sorts of different date and datetime string formats.

https://pypi.org/project/python-dateutil/

An example👇

from dateutil.parser import parse

print(parse("2012-01-19 17:21:00"))
# => 2012-01-19 17:21:00

print(parse("1st Jan, 2019"))
# => 2019-01-01 00:00:00

print(parse("23.11.2020"))
# => 2020-11-23 00:00:00

Unfortunately, this is quite slow:

from dateutil.parser import parse
%timeit parse(datetime_txt).date()
54.3 µs ± 450 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

import datetime
%timeit datetime.datetime.strptime(datetime_txt, "%H:%M %d-%m-%Y").date()
7.44 µs ± 240 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

Python @classmethod

Python tip:

You can use @classmethod to create class methods.

For example, you can create a class method that loads events from a JSON string message👇

import datetime
import json


class UserRegistered:
    def __init__(self, username, event_time):
        self.username = username
        self.event_time = event_time

    @classmethod
    def from_event_message(cls, message):
        message = json.loads(message)

        return cls(
            username=message["username"],
            event_time=datetime.datetime.fromisoformat(message["event_time"]),
        )


message = '{"username": "johndoe", "event_time": "2021-04-26T20:00:00"}'
event = UserRegistered.from_event_message(message)
print(event.username, event.event_time)
# => johndoe 2021-04-26 20:00:00

5 awesome tools for writing Python tests

5 awesome tools for writing Python tests:

1. pytest - go-to testing framework for testing Python code
2. unittest.mock - built-in mocking library
3. coverage.py - measuring code coverage (use pytest-cov plugin when using pytest)
4. mutmut - improve the quality of your test suite with mutation testing
5. hypothesis - property-based testing library

You can learn about all of these tools here.

How do you parameterize a test in pytest?

Pytest tip:

You can parametrize your tests with @pytest.mark.parametrize.

Example:

import pytest


def is_palindrome(text):
    return text == "".join(reversed(text))


@pytest.mark.parametrize(
    "text, is_pal",
    [
        ("kayak", True),
        ("racecar", True),
        ("bmw", False),
        ("songs", False),
    ],
)
def test_is_palindrome(text, is_pal):
    assert is_palindrome(text) == is_pal

How can I implement a custom error handler in Flask?

Did you know?

You can register exception handlers to a Flask app based on an exception class or response status code.

An example👇

from flask import Flask, jsonify, abort

app = Flask(__name__)


class ValidationException(Exception):
    code = 500
    message = "Unknown error"


@app.errorhandler(ValidationException)
def handle_validation_exception(exc):
    return (
        jsonify({"msssage": exc.message, "exception": exc.__class__.__name__}),
        exc.code,
    )


@app.errorhandler(500)
def handle_internal_server_error(exc):
    return jsonify({"msssage": "Oops!", "exception": "Internal server error"}), 500


@app.route("/")
def hello():
    raise ValidationException()

Custom field validators in Django

Did you know?

In Django, you can add custom validators to your model fields

For example, you can validate that price is always greater than 0👇

from django.core.exceptions import ValidationError
from django.db import models
from django.utils.translation import gettext_lazy as _


def validate_greater_than_zero(value):
    if value <= 0:
        raise ValidationError(
            _("%(value)s is not greater than zero."),
            params={"value": value},
        )


class Book(models.Model):
    title = models.CharField(max_length=120)
    price = models.DecimalField(validators=[validate_greater_than_zero])

pydantic typecasting

Did you know?

Pydantic uses typecasting.

That means that it tries to convert provided values to their expected types if it's possible.

Examples👇

import datetime

from pydantic import BaseModel


class Location(BaseModel):
    city: str
    country: str


class Measurement(BaseModel):
    temperature: float
    measured_at: datetime.datetime
    location: Location


print(
    Measurement(
        temperature=10,
        measured_at="2021-04-16T21:15:03.000012",
        location={"city": "London", "country": "UK"},
    )
)

"""
temperature=10.0 measured_at=datetime.datetime(2021, 4, 16, 21, 15, 3, 12) location=Location(city='London', country='UK')
"""

print(
    Measurement(
        temperature=10.0,
        measured_at="2021-04-16T21:15:03",
        location=Location(city="London", country="UK"),
    )
)

"""
temperature=10.0 measured_at=datetime.datetime(2021, 4, 16, 21, 15, 3) location=Location(city='London', country='UK')
"""

print(
    Measurement(
        temperature=10,
        measured_at="16186000528",
        location=[("city", "London"), ("country", "UK")],
    )
)

"""
temperature=10.0 measured_at=datetime.datetime(2021, 4, 16, 19, 15, 28, tzinfo=datetime.timezone.utc) location=Location(city='London', country='UK')
"""

Public, protected, and private methods ​in Python

Did you know?

Python doesn't have public, protected, or private methods.

Instead, use the following naming conventions:

1. method_name -> public
2. _method_name -> protected
3. __method_name -> private

The same goes for attributes.

An example👇

class User:
    def __init__(self, first_name, last_name, age):
        self.first_name = first_name
        self.last_name = last_name
        self.__age = age

    def age(self):
        return "You shouldn't ask that. It's a private information."

    def _is_adult(self):
        return self.__age >= 21

    def can_drink_alcohol(self):
        return self._is_adult()

Python - checking whether a list contains duplicates

Did you know?

As long as the elements are hashable, you can check whether a list contains duplicated values by converting it to a set and comparing the lengths of the list and set.

Example:

names = ["Jan", "John", "Marry", "Daisy", "Jan"]

names_has_duplicates = len(set(names)) < len(names)
print(names_has_duplicates)
# => True

Black compatibility with flake8

Python tip:

When using black and flake8 together set the max line length for flake8 to 88 to match black's default value.

Minimal flake8 black compatible config:

[flake8]
max-line-length = 88
extend-ignore = E203

Flask Sentry Example

Flask tip:

Add Sentry to your Flask app to track unhandled exceptions.

An example👇

import sentry_sdk
from flask import Flask
from sentry_sdk.integrations.flask import FlaskIntegration

sentry_sdk.init(
    dsn="your-sentry-dsn",
    integrations=[FlaskIntegration()],
    traces_sample_rate=0.3,
    environment="production",
)

app = Flask(__name__)

Python 3.10 - parenthesized context managers

Did you know?

Python 3.10 comes with support for continuation across multiple lines when using context managers.

An example👇

import unittest
from unittest import mock

from my_module import my_function


# Python < 3.10


class Test(unittest.TestCase):
    def test(self):
        with mock.patch("my_module.secrets.token_urlsafe") as a, mock.patch("my_module.string.capwords") as b, mock.patch("my_module.collections.defaultdict") as c:
            my_function()
            a.assert_called()
            b.assert_called()
            c.assert_called()

    def test_same(self):
        with mock.patch("my_module.secrets.token_urlsafe") as a:
            with mock.patch("my_module.string.capwords") as b:
                with mock.patch("my_module.collections.defaultdict") as c:
                    my_function()
                    a.assert_called()
                    b.assert_called()
                    c.assert_called()


# Python >= 3.10

class Test(unittest.TestCase):
    def test(self):
        with (
            mock.patch("my_module.secrets.token_urlsafe") as a,
            mock.patch("my_module.string.capwords") as b,
            mock.patch("my_module.collections.defaultdictl") as c,
        ):
            my_function()
            a.assert_called()
            b.assert_called()
            c.assert_called()

Change a model field name in Django REST Framework

Django tip:

You can rename the model field inside the serializer by using the source for the selected field.

For example, the is_active field from the model can be returned as active👇

# models. py
from django.contrib.auth.models import User
from django.db import models


class UserProfile(models.Model):
    user = models.OneToOneField(to=User, on_delete=models.CASCADE)
    bio = models.TextField()
    birth_date = models.DateField()

    def __str__(self):
        return f"{self.user.username} profile"


# serializers.py
from rest_framework import serializers


class UserSerializer(serializers.ModelSerializer):
    active = serializers.BooleanField(source="is_active")

    class Meta:
        model = User
        fields = ["id", "username", "email", "is_staff", "active"]

Difference between re.search and re.match in Python?

re.match() searches for matches from the beginning of a string while re.search() searches for matches anywhere in the string.

Example:

import re

claim = 'People love Python.'

print(re.search(r'Python', claim).group())
# => Python

print(re.match(r'Python', claim))
# => None

print(re.search(r'People', claim).group())
# => People

print(re.match(r'People', claim).group())
# => People

Extract contents of a PDF file with pikepdf in Python

Python tip:

You can use pikepdf to extract a subset pf pages from an original PDF and create a new file containing only the extracted pages.

For example,👇 save pages 2 and 3 from the original PDF

import pathlib

from pikepdf import Pdf

start = 2
stop = 3
filepath = pathlib.Path("file.pdf")

pdf = Pdf.open(filepath)
new_pdf = Pdf.new()

new_pdf.pages.extend(pdf.pages[start - 1 : stop])

new_pdf.save("new.pdf", compress_streams=False)

Using pathlib's read_bytes method in Python

Python tip:

You can use read_bytes() (which handles the opening and closing of the file) from pathlib's Path to read bytes from a file instead of using with open().

Example:

# pathlib
import pathlib
file_bytes = pathlib.Path("file.pdf").read_bytes()


# with open
with open("file.pdf", "rb") as file:
    file_bytes = file.read()

Skip tests based on a condition in pytest

Pytest tip:

You can skip a test based on a given condition.

For example, you can skip a test when running on Windows

import sys

import pytest


@pytest.mark.skipif(sys.platform == "win32", reason="Not running on Window")
def test_windows():
    assert True

Python - provide a function as an argument to another function to improve its testability

Python tip:

You can provide a function as an argument to another function to improve its testability.

For example, you can provide input as a function in a real application and mock the function in test

def get_user_expense(get_input):
    raw_input = get_input(
        "Enter your expense (date;description;amount) (e.g. 2021-03-21;Gasoline;75.43): "
    )
    date, description, amount = raw_input.split(";")
    return date, description, amount


# test
def test_get_user_expense():
    assert get_user_expense(lambda *args: "2021-03-21;Gasoline;75.43") == (
        "2021-03-21",
        "Gasoline",
        "75.43",
    )


# real application
def main():
    date, description, amount = get_user_expense(input)
    print(date, description, amount)

Poetry - generate requirements.txt without hashes

Poetry tip:

When using Docker, you can export dependencies to a requirements.txt file without hashes to decrease time to resolve dependencies.

👇

poetry export --without-hashes --format=requirements.txt > requirements.txt

Caching poetry install for CI

Poetry tip:

When running poetry inside a CI pipeline, set virtualenvs.in-project config to true.

That way the virtual environment will be created inside the current folder -> you can cache it.

Gitlab example👇

stages:
  - test

cache:
  key: virtualenv
  paths:
    - .venv/

tests:
  stage: test
  image: python3.8-slim
  before_script:
    - poetry config virtualenvs.in-project true
    - poetry install
  script:
    - poetry run python -m pytest tests

Black compatibility with isort

Python tip:

When using black and isort on the same project, use isort's profile black to avoid code formatting conflicts

👇

$ isort . --profile black

Testing code examples in docstrings with pytest

Pytest tip:

You can test code examples inside your docstrings like so:

$ pytest --doctest-modules http://yourmodule.py

👇

from typing import List


def daily_average(temperatures: List[float]) -> float:
    """
    Get average daily temperature

    Calculate average temperature from multiple measurements

    >>> daily_average( [10.0, 12.0, 14.0])
    12.0

    : param temperatures: list of temperatures
    : return: Average temperature
    """

    return sum(temperatures) / len(temperatures)


# python -m pytest --doctest-modules temperature. py

Python dictionary clear() method

Python tip:

You can use .clear to clear a dictionary.

Is it the same as foo = {} ?

No.

1. foo = {} will create a new instance but other references will still point to the old items.
2. Because of #1, foo = {} is probably faster than foo.clear()
3. foo.clear() is slower but safer. Be careful with scope with foo = {}.

An example👇

user = {"name": "Jan", "username": "giaco"}

user.clear()
print(user). # => {}

What’s the difference between '==' and 'is' in Python?

What's the difference between == and is?

Let's find out👇

== is an equality operator. It checks whether two objects are equal by their values. To compare, objects must have an __eq__ method implemented.

my_list = [1, 2, 3]
another_list = [element for element in my_list]
print(my_list == another_list)
# => True


class User:
    def __init__(self, name):
        self.name = name

    def __eq__(self, other):
        return other.name == self.name


user1 = User(name="Jan")
user2 = User(name="Jan")
print(user1 == user2)
# => True

is is an identity operator. It checks whether you're referencing the same object on both sides of the comparison. It doesn't care about the values.

my_list = [1, 2, 3]
another_list = [element for element in my_list]
print(my_list is another_list)
# => False


class User:
    def __init__(self, name):
        self.name = name


user1 = User(name="Jan")
user2 = User(name="Jan")
print(user1 is user2)
# => False
print(user1 is user1)
# => True

When should you use one over the other?

Use== when you care about values and use is when you care about actual objects.

Python - assert that a mock function was called with certain arguments

Python tip:

You can assert that the method on a MagicMock object was called with certain arguments.

https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called_with

For example, you can mock an email service and assert that the send method was called with specific values:

from unittest.mock import MagicMock


def place_order(user, items, email_service):
    print(f"Place order for user {user}")
    print(f"With items: {items}")
    email_service.send(user, "Order placed.")


def test_place_order():
    email_service = MagicMock()

    place_order("[email protected]", ["Computer", "USB drive", "Vodka"], email_service)
    email_service.send.assert_called_with("[email protected]", "Order placed.")

Python - unittest.mock.create_autospec() example

Python tip:

You can use mock.create_autospec to create a mock object with the same interface as the provided class, function, or method.

For example, you can use it to mock Response from the requests package👇

from unittest import mock

import requests
from requests import Response


def get_my_ip():
    response = requests.get(" http://ipinfo.to/json")
    return response.json()["ip"]


def test_get_my_ip(monkeypatch):
    my_ip = "123.123.123.123"
    response = mock.create_autospec(Response)
    response.json.return_value = {"ip": my_ip}
    monkeypatch.setattr(requests, "get", lambda *args, **kwargs: response)
    assert get_my_ip() == my_ip

Getting started with Python type hints

Python tip:

Use type hints to annotate expected types for variables, function parameters, and function returns.

https://testdriven.io/blog/python-type-checking/

An example👇

from typing import Collection


def daily_average(temperatures: Collection[float]) -> float:
    return sum(temperatures) / len(temperatures)


print(daily_average.__annotations__)
# => {'temperatures': typing.Collection[float], 'return': <class 'float'>}

Managing session data in Flask

Flask tip:

Setting a value to Flask's session is as simple as:

session['key'] = 'value'

https://testdriven.io/blog/flask-sessions/

For example:

from flask import Flask, redirect, request, session, url_for

# Create the Flask application
app = Flask(__name__)

app.secret_key = "BAD SECRET KEY"


@app.route("/set_email", methods=["GET", "POST"])
def set_email():
    if request.method == "POST":
        # Save the form data to the session object
        session["email"] = request.form["email_address"]
        print(session["email"])
        return redirect(url_for("set_email"))
    return """
        <form method="post">
        <label for="email">Enter your email address:</label>
        <input name="email_address" required / >
        <button type="submit">Submit</button
        </form>
        """

Python Testing - Given When Then syntax

TDD tip:

You can add GIVEN, WHEN, THEN as a docstring to your tests to better express your intent.

An example👇

def test_all(self):
    """
    GIVEN a PhoneBook with a records property
    WHEN the all method is called
    THEN all numbers should be returned in ascending order
    """

    phone_book = PhoneBook(
        records=[
            ("John Doe", "03 234 567 890"),
            ("Marry Doe", "01 234 567 890"),
            ("Donald Doe", "02 234 567 890"),
        ]
    )

    previous = ""

    for record in phone_book.all():
        assert record[0] > previous
        previous = record[0]

Teaching kids programming with Python Turtle

Python tip:

Turtle graphics is a popular way to introduce programming to kids.

You can use it to draw different shapes. The cursor moves on the screen.

An example👇

from turtle import *

color("red", "yellow")
begin_fill()

while True:
    forward(200)
    left(170)
    if abs(pos()) < 1:
        break

end_fill()
done()

Django deployment checklist

Django tip:

Check your production settings.py file for security vulnerabilities with the check command:

$ ./manage.py check --deploy

https://docs.djangoproject.com/en/3.2/ref/django-admin/#check

Example:

$ ./manage.py check --deploy
System check identified some issues:

WARNINGS:
have not set CSRF COOKIE SECURE to True. Using a secure-only CSRF cookie
makes it more difficult for network traffic sniffers to steal the CSRF token.
? (security.W018) You should not have DEBUG set to True in deployment.
? (security.W022) You have not set the SECURE_REFERRER_POLICY setting.
Without this, your site will not send a Referrer-Policy header.
You should consider enabling this header to protect user privacy.

Test and publish your Python packages to PyPI with poetry and GitHub Actions

Python tip:

Use poetry and Github Actions to:

1. run tests for your packages - on every push
2. check code quality - on every push
3. publish the packages to PyPI - on every release

https://testdriven.io/blog/python-project-workflow/

An example👇

Test and check for code quality:

name: Push
on: [push]

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        python-version: [3.9]
        poetry-version: [1.1.2]
        os: [ubuntu-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/[email protected]
      - uses: actions/[email protected]
        with:
          python-version: ${{ matrix.python-version }}
      - name: Run image
        uses: abatilo/[email protected]
        with:
          poetry-version: ${{ matrix.poetry-version }}
      - name: Install dependencies
        run: poetry install
      - name: Run tests
        run: poetry run pytest --cov=./ --cov-report=xml
      - name: Upload coverage to Codecov
        uses: codecov/[email protected]
  code-quality:
    strategy:
      fail-fast: false
      matrix:
        python-version: [3.9]
        poetry-version: [1.1.2]
        os: [ubuntu-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/[email protected]
      - uses: actions/[email protected]
        with:
          python-version: ${{ matrix.python-version }}
      - name: Run image
        uses: abatilo/[email protected]
        with:
          poetry-version: ${{ matrix.poetry-version }}
      - name: Install dependencies
        run: poetry install
      - name: Run black
        run: poetry run black . --check
      - name: Run isort
        run: poetry run isort . --check-only
      - name: Run flake8
        run: poetry run flake8 .
      - name: Run bandit
        run: poetry run bandit .
      - name: Run saftey
        run: poetry run safety check

Release:

name: Release
on:
  release:
    types:
      - created

jobs:
  publish:
    strategy:
      fail-fast: false
      matrix:
        python-version: [3.9]
        poetry-version: [1.1.2]
        os: [ubuntu-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/[email protected]
      - uses: actions/[email protected]
        with:
          python-version: ${{ matrix.python-version }}
      - name: Run image
        uses: abatilo/[email protected]
        with:
          poetry-version: ${{ matrix.poetry-version }}
      - name: Publish
        env:
          PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
        run: |
          poetry config pypi-token.pypi $PYPI_TOKEN
          poetry publish --build

Finding memory leaks in Python with tracemalloc

Python tip:

You can use tracemalloc to display files allocating the most memory.

An example👇

import tracemalloc

import app

tracemalloc.start()

app.run()

snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics("lineno")

for stat in top_stats[:10]:
    print(stat)

Unpacking iterables in Python

Python tip:

You can unpack any iterable to variables as long as the number of variables is the same as the number of elements in the iterable.

Examples👇

a, b, c = (1, 2, 3)
print(a, b, c)
# => 1 2 3

a, b, c, d = (1, 2, 3, 4)
print(a, b, c, d)
# => 1 2 3 4

a, b = {1, 2}
print(a, b)
# => 1 2

a, b = {'a': 1, 'b': 2}
print(a, b)
# => a b

a, b, c = (i for i in range(1, 4))
print(a, b, c)
# => 1 2 3

a, b, c = (1, 2)
print(a, b, c)
# => ValueError: not enough values to unpack (expected 3, got 2)

a, b, c = (1, 2, 3, 4)
print(a, b, c)
# => ValueError: too many values to unpack (expected 3)

Python - quick tkinter example

Python tip:

You can use tkinter to build simple desktop applications

An example👇

import tkinter as tk


class Application(tk.Frame):
    def __init__(self, master=None):
        self.number_of_clicks = tk.IntVar(value=0)

        super().__init__(master)
        self.master = master
        self.pack()
        self.create_widgets()

    def create_widgets(self):
        self.number_of_clicks_value = tk.Label(self, textvariable=self.number_of_clicks)
        self.number_of_clicks_value.pack(side="top")

        self.click_me = tk.Button(self, text="Click me", command=self.increment)
        self.click_me.pack(side="top")

        self.quit = tk.Button(self, text="QUIT", fg="red", command=self.master.destroy)
        self.quit.pack(side="bottom")

    def increment(self):
        self.number_of_clicks.set(self.number_of_clicks.get() + 1)


root = tk.Tk()
app = Application(master=root)
app.mainloop()

Call a function after some interval in Python with Timer

Python tip:

You can use Timer to run some function only after a certain amount of time has passed

An example👇

from threading import Timer


def truth():
    print("Python rocks!")


t = Timer(15, truth)
t.start() # truth will be called after a 15 second interval

Mutable default parameter values in Python

Python tip:

Avoid setting mutable default parameters for functions. Instead, use None as the default and assign the mutable value inside the function.

Why?

Python evaluates a function's default parameters once, not each time the function is called. So, if you allow a mutable default parameter and mutate it, it will be mutated for all future calls.

An example👇

# wrong
def add_jan(users=[]):
    users.append("Jan")
    return users


print(add_jan())  # => ['Jan']
print(add_jan())  # => ['Jan', 'Jan']


# right
def add_jan(users=None):
    users = users or []
    users.append("Jan")
    return users


print(add_jan())  # => ['Jan']
print(add_jan())  # => ['Jan']

Textwrap - text wrapping in Python

Python tip:

You can wrap long strings into multiple lines where each line is shorter than the limit by using textwrap.wrap.

An example👇

import textwrap

text = (
    "The single-responsibility principle (SRP) is a computer programming principle "
    "that states that every class in a computer program should have responsibility over "
    "a single part of that program's functionality, which it should encapsulate. "
    "All of the module's, class' or function's services should be narrowly aligned with that responsibility"
)
lines = textwrap.wrap(text, 70)

for line in lines:
    print(line)

"""
The single-responsibility principle (SRP) is a computer programming
principle that states that every class in a computer program should
have responsibility over a single part of that program's
functionality, which it should encapsulate. All of the module's,
class' or function's services should be narrowly aligned with that
responsibility
"""

How do I extract a tar file in Python?

Python tip:

You can extract a tar archive to a destination folder using tarfile.

An example👇

import tarfile

with tarfile.open("example.tar.gz") as tar:
    tar.extractall("destination_folder")

Arithmetic, Geometric, and Harmonic Means in Python

Python tip:

Use statistics.mean to calculate the arithmetic mean of an iterable.

Use statistics.geometric_mean to calculate the geometric mean of an iterable (Python >= 3.8).

Use statistics.harmonic_mean to calculate the harmonic mean of an iterable.

An example👇

from statistics import geometric_mean, harmonic_mean, mean

print(mean([1, 3, 7, 2, 5]))
# => 3.6

print(geometric_mean([1, 3, 7, 2, 5]))
# => 2.913693458576192

print(harmonic_mean([1, 3, 7, 2, 5]))
# => 2.297592997811816

Python - execute shell commands in subprocess

Python tip:

Use the subprocess module to spawn new processes.

https://docs.python.org/3/library/subprocess.html#subprocess.Popen

Use Popen.communicate to capture result and errors.

An example👇

import subprocess

proc = subprocess.Popen(
    ["ls", "-la", "/home/johndoe/Images"],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
)
output, err = proc.communicate()

if err:
    raise Exception(err.decode())

print(output.decode())
"""
drwxrwxr-x   2 johndoe  johndoe    4096 Sep 28 08:17 .
drwxr-xr-x  13 johndoe  johndoe    4096 Sep 28 08:12 ..
-rw-rw-r--   1 johndoe  johndoe  115887 Sep 19 09:15 1_initial_user.png
-rw-rw-r--   1 johndoe  johndoe  115055 Sep 19 09:15 2_change_ban.png
-rw-rw-r--   1 johndoe  johndoe  114587 Sep 19 09:15 4_change_database.png
-rw-rw-r--   1 johndoe  johndoe  188718 Sep 19 09:15 5_srp.png
-rw-rw-r--   1 johndoe  johndoe  188718 Sep 19 09:15 change_charge.png
"""

Generate a password in Python

Python tip:

To generate a random password you can use the secrets module.

For example, you can generate a random password of length 21 containing letters, numbers, and special characters👇

import secrets
import string

alphabet = string.ascii_letters + string.digits + '!"#$%&/()=?*+|@{}[]'
password = "".join(secrets.choice(alphabet) for i in range(21))

print(password)
# => *cgsV!2LN|8f7)EzVR]eX

Limit execution time of a function call in Python

Python tip:

You can limit the execution time of a function by using the signal library.

An example👇

import signal
import time


def handle_timeout(signum, frame):
    raise TimeoutError


def my_task():
    for i in range(6):
        print(f"I am working something long running, step {i}")
        time.sleep(1)


signal.signal(signal.SIGALRM, handle_timeout)
signal.alarm(5)  # 5 seconds

try:
    my_task()
except TimeoutError:
    print("It took too long to finish the job")
finally:
    signal.alarm(0)


"""
I am working something long running, step 0
I am working something long running, step 1
I am working something long running, step 2
I am working something long running, step 3
I am working something long running, step 4
It took too long to finish the job
"""

Generate secure, URL safe, random tokens in Python

Python tip:

Use secrets.token_urlsafe() to generate security tokens.

For example, you can generate a token for a password reset👇

import secrets

url = f"https://my-app.com/password-reset/token={secrets.token_urlsafe()}"

print(url)
# => https://my-app.com/password-reset/token=KzmPWanja-z0r2BpyC7mFAZd5LXvp8SnIUbSnQyEMWw

Executing modules with runpy in Python

Python tip:

You can use runpy to run a module without importing it first. It works the same as the -m command line option.

An example👇

######## example.py ########
print("I am an example")

######## run_example.py ########
import runpy

runpy.run_module(mod_name="example")  # to run module example.py

# -> I am an example

FIFO and LIFO queue examples in Python

Python tip:

You can create a simple FIFO or LIFO queue with the queue module.

An example👇

import queue

my_queue = queue.Queue()

my_queue.put("Jan")
my_queue.put("Mike")

print(my_queue.get())
# => Jan
print(my_queue.get())
# => Mike

my_lifo_queue = queue.LifoQueue()

my_lifo_queue.put("Jan")
my_lifo_queue.put("Mike")

print(my_lifo_queue.get())
# => Mike
print(my_lifo_queue.get())
# => Jan

Using Django's choices field option

Django tip:

Use choices for character fields with a finite number of possible values.

For example, you can use it for blog's status:

from django.db import models


class Blog(models.Model):
    DRAFT = "DRF"
    PUBLISHED = "PUB"
    DELETED = "DEL"

    STATUS_CHOICES = [
        (DRAFT, "Draft"),
        (PUBLISHED, "Published"),
        (DELETED, "Deleted"),
    ]
    title = models.CharField(max_length=120)
    content = models.TextField()
    contributors = models.TextField()
    status = models.CharField(max_length=3, choices=STATUS_CHOICES, default="DRAFT")

Django CharField vs TextField

Django tip:

Use models.TextField for storing long texts inside your models instead of models.CharField since CharField has a max length of 255 characters while TextField can hold more than 255 characters.

An example:

from django.db import models


class Blog(models.Model):
    title = models.CharField(max_length=120)
    content = models.TextField()
    contributors = models.TextField()

Django Rest Framework - read only views

Django REST Framework tip:

You can use ReadOnlyModeViewSet when you don't want to allow any unsafe operations (POST, DELETE, PATCH).

An example👇

class BlogViewSet(ReadOnlyModeViewSet):
    """
    A simple ViewSet for viewing blogs.
    """

    queryset = Blog.objects.all()
    serializer_class = BlogSerializer

Python pprint – pretty-print data structures

Python tip:

You can use pprint.pprint to print a formatted representation of an object.

For example:

import json
import pprint
from urllib.request import urlopen

with urlopen("https://pypi.org/pypi/flask/json") as resp:
    project_info = json.load(resp)["info"]

pprint.pprint(project_info)

"""
{'author': 'Armin Ronacher',
 'author_email': '[email protected]',
 'bugtrack_url': None,

 ...

 'requires_python': '>=3.6',
 'summary': 'A simple framework for building complex web applications.',
 'version': '2.0.1',
 'yanked': False,
 'yanked_reason': None}
"""

Python pathlib.Path.expanduser()

Python tip:

You can use Path.expanduser from pathlib to convert ~ and ~user paths to the full path.

Example:

import pathlib

path = pathlib.Path("~/Documents/test.txt")

print(path)
# => ~/Documents/test.txt

print(path.expanduser())
# => /Users/michael/Documents/test.txt

Check if file is a symlink in Python

Python tip:

You can use pathlib.Path.is_symlink to check whether the path is a symbolic link.

An example👇

import pathlib

path = pathlib.Path("/usr/bin/python")

print(path.is_symlink())
# => True

Filename pattern matching in Python with pathlib.Path.glob()

Python tip:

You can use pathlib.Path.glob() to list all files matching a given pattern.

For example, you can list names of all txt files located inside the same directory as your Python script👇

import pathlib

parent_folder = pathlib.Path(__file__).parent.absolute()

print(parent_folder)
for file in parent_folder.glob("*.txt"):
    print(file.name)

"""
example.txt
first.txt
second.txt
"""

Test-diven devopment and pair programming

TDD Tip:

When practicing Test-driven Development with pair programming, try having one developer write the failing test while the other writes the code to get it to pass. The first developer is the responsible for any refactoring.

Why?

1. It's fun.
2. This can accelerate the learning of a less experienced developer when paired with a more experienced developer.

For more TDD benefits, check out What is Test-Driven Development? .

Sort a list of objects based on an attribute of the objects in Python

Python tip:

You can use attrgetter to sort a list of objects based on the value of the selected attribute.

An Example👇

from operator import attrgetter


class User:
    def __init__(self, username):
        self.username = username

    def __repr__(self):
        return f'User(username="{self.username}")'


users = [User("johndoe"), User("bobby"), User("marry")]

print(sorted(users, key=attrgetter("username")))
# => [User(username="bobby"), User(username="johndoe"), User(username="marry")]

Find the mime type of a file in Python

Python tip:

You can use mimetypes.guess_type to get a mime type and encoding for a file from its name.

For example👇

import mimetypes

mime_type, _ = mimetypes.guess_type("my _ file. pdf")
print(mime_type)
# => application/pdf

mime_type, encoding = mimetypes.guess_type("archive. tar.gz")
print(mime_type, encoding)
# => application/x-tar gzip

Python - logging with filters

Python tip:

You can use Filter from the logging module to add additional information to your log records.

For example, you can add the username of the system user who's running the program👇

import logging
import os


class UserRunningProgramFilter(logging.Filter):
    def filter(self, record):
        record.user = os.getenv("User")
        return True


if __name__ == "__main__":
    levels = (
        logging.DEBUG,
        logging.INFO,
        logging.WARNING,
        logging.ERROR,
        logging.CRITICAL,
    )
    logging.basicConfig(
        level=logging.DEBUG,
        format="%(asctime)-15s %(name)-5s %(levelname)-8s Run by User: %(user)-20s %(message)s",
    )
    logger = logging.getLogger("mylogger")

    f = UserRunningProgramFilter()
    logger.addFilter(f)
    logger.debug("A debug message")

# => 2021-09-22 07:56:45,473 mylogger DEBUG    Run by User: johndoe                 A debug message

Python For Else

Python tip:

You can use else with for loops when searching for something. Code inside the else will be executed if the break statement isn't hit.

items = ["right_one", "my_item", " another_item"]


def right_one(item_to_check):
    if item_to_check == "right_one":
        return True
    else:
        return False


for item in items:
    if right_one(item):
        print("Do something with the item")
        break
    else:
        print("I will do something else, no item to process")


"""
Do something with the item
"""


items = ["different_item", "my_item", "another_item"]


for item in items:
    if right_one(item):
        print("Do something with the item")
        break
    else:
        print("I will do something else, no item to process")

"""
I will do something else, no item to process
I will do something else, no item to process
I will do something else, no item to process
"""

Django - custom user model

Django Tip:

Use a custom user model when starting a project

When starting a new project, set up a custom user model. This model behaves just like the default user model, but you'll be able to customize it in the future.

https://testdriven.io/blog/django-custom-user-model/

# users/models.py

from django.contrib.auth.models import AbstractUser


class CustomUser(AbstractUser):
    pass


# users/adming.py

from django.contrib import admin
from django.contrib.auth.admin import UserAdmin

from .models import CustomUser

admin.site.register(CustomUser, UserAdmin)


# settings.py

AUTH_USER_MODEL = "users.CustomUser"

Python - prettyprint JSON with json.dumps

Python tip:

You can use the indent parameter with json.dumps to get an indented multiline string instead of a single line.

It's much easier to read.

An example👇

import json

users = [
    {"username": "johndoe", "active": True},
    {"username": "Mary", "active": False}
]


print(json.dumps(users))
# [{"username": "johndoe", "active": true}, {"username": "Mary", "active": false}]


print(json.dumps(users, indent=2))
"""
[
  {
    "username": "johndoe",
    "active": true
  },
  {
    "username": "Mary",
    "active": false
  }
]
"""

linecache.getline() in Python

Python tip:

You can get any line from a file by using the getline method from linecache:

linecache.getline(filename, lineno, module_globals=None)

It never raises an exception. Instead, it returns '' (no line with a number, no file, ...).

The trackback module uses it to retrieve source lines for formatted tracebacks.

An example👇

import linecache

"""
Content of example. txt:

Example
Tweet
too
"""


print(linecache.getline("example.txt", 2))
# => Tweet

Check if a string is a valid keyword in Python

Python tip:

You can check whether some string is a reserved Python keyword with the keyword module.

An example👇

import keyword

""""
False       await       else        import      pass
None        break       except      in          raise
True        class       finally     is          return
and         continue    for         lambda      try
as          def         from        nonlocal    while
assert      del         global      not         with
async       elif        if          or          yield
"""

print(keyword.iskeyword("raise"))
# => True

print(keyword.iskeyword("foo"))
# => False

Comparing files in Python

Python tip:

You can check if two files are equal with cmp from the filecmp module. It returns True if the files are equal and False if they're not.

For example:

import filecmp

print(filecmp.cmp("file.pdf", "file.pdf"))
# => True

print(filecmp.cmp("file.pdf", "file1.pdf"))
# => False

Automatically setting an enum member's value in Python

Python tip:

When the exact value of an enum member is not important, you can use auto() to auto-generate it:

The value starts at 1 and then increases incrementally by 1.

https://docs.python.org/3/library/enum.html#using-automatic-values

An example👇

from enum import Enum, auto


class Status(Enum):
    DRAFT = auto()
    IN_REVIEW = auto()
    PUBLISHED = auto()
    DELETED = auto()


print(list(Status))
"""
[<Status.DRAFT: 1>, <Status.IN_REVIEW: 2>, <Status.PUBLISHED: 3>, <Status.DELETED: 4>]
"""

Python deep copy via copy.deepcopy()

You can use copy.deepcopy() to create a copy of a compound object.

deepcopy creates a copy of the object and of all its nested objects. Changes to the original object won't affect the copied object since new objects are created instead of just referencing.

import copy

house = {
    "width": 12,
    "length": 8,
    "height": 3.5,
    "doors": [
        {"type": " ENTRANCE", "width": 0.9, "height": 2.2},
        {"type": "BACK DOOR", "width": 0.7, "height": 2.0},
    ],
}

same_house = copy.deepcopy(house)
print(house)
print(same_house)
print(house == same_house)

"""
{'width': 12, 'length': 8, 'height': 3.5, 'doors': [{'type': ' ENTRANCE', 'width': 0.9, 'height': 2.2}, {'type': 'BACK DOOR', 'width': 0.7, 'height': 2.0}]}
{'width': 12, 'length': 8, 'height': 3.5, 'doors': [{'type': ' ENTRANCE', 'width': 0.9, 'height': 2.2}, {'type': 'BACK DOOR', 'width': 0.7, 'height': 2.0}]}
True
"""

house["height"] = 4.0
print(house)
print(same_house)
print(house == same_house)

"""
{'width': 12, 'length': 8, 'height': 4.0, 'doors': [{'type': ' ENTRANCE', 'width': 0.9, 'height': 2.2}, {'type': 'BACK DOOR', 'width': 0.7, 'height': 2.0}]}
{'width': 12, 'length': 8, 'height': 3.5, 'doors': [{'type': ' ENTRANCE', 'width': 0.9, 'height': 2.2}, {'type': 'BACK DOOR', 'width': 0.7, 'height': 2.0}]}
False
"""

Create a CSV file from a list of dictionaries in Python

Python tip:

You can use DictWriter to create a CSV file from a list of dictionaries.

An example👇

import csv

users = [
    {"username": "johndoe", "name": "John Doe"},
    {"username": "bob", "name": "Bob Builder"},
    {"username": "daisy", "name": "Daisy Flower"},
]

fieldnames = ["username", "name"]

with open("users.csv", "w") as csvfile:
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
    writer.writeheader()
    writer.writerows(users)

"""
users.csv

username,name
johndoe,John Doe
bob,Bob Builder
daisy,Daisy Flower
"""

Python - comparing two text files with difflib.HtmlDiff()

Python tip:

You can use HtmlDiff to generate an HTML table that shows a side by side, line by line comparison of the text with inter-line and intra-line change highlights.

https://docs.python.org/3/library/difflib.html#difflib.HtmlDiff

An example👇

import difflib
from pathlib import Path

first_file_lines = Path('first.txt').read_text().splitlines()
second_file_lines = Path('second.txt').read_text().splitlines()

html_diff = difflib.HtmlDiff().make_file(first_file_lines, second_file_lines)
Path('diff.html').write_text(html_diff)

Python - find all similar strings from a provided word within a list of strings

Python tip:

You can use get_close_matches from difflib to find all similar strings from a provided word within a list of strings:

difflib.get_close_matches(word, possibilities, n=3, cutoff=0.6)

For example, find similar names👇

from difflib import get_close_matches

name = "John"
names = ["John", "Jane", "Bob", "Daisy", "Jim", "Mark", "Johnny", "Mike"]


print(get_close_matches(name, names))
# => ['John', 'Johnny']

Python - Using SequenceMatcher.ratio() to find similarity between two strings

Python tip:

You can use difflib.SequenceMatcher.ratio() to get the distance between two strings:

• T - total number of elements in both strings (len(first_string) + len(second_string))
• M - number of matches

Distance = 2.0 * M / T -> between 0.0 and 1.0 (1.0 if the sequences are identical, and 0.0 if they don't have anyhing in common)

https://docs.python.org/3/library/difflib.html#sequencematcher-objects

For example:

from difflib import SequenceMatcher

first = "Jane"
second = "John"

print(SequenceMatcher(a=first, b=second).ratio())
# => 0.5

Decimal vs float in Python

Python tip:

You can use Decimal instead of float to:

1. incorporate a notion of significant places (1.20 + 1.30 = 2.50)
2. represent decimal numbers exactly (1.1 + 2.2 = 3.3)

In short, use Decimal when precision matters.

An example 👇

from decimal import Decimal

print(1.20 + 1.30)
# => 2.5

print(Decimal("1.20") + Decimal("1.30"))
# => 2.50

print(1.1 + 2.2)
# => 3.3000000000000003

print(Decimal("1.1") + Decimal("2.2"))
# => 3.3

RGB to HSV in Python with colorsys

Python tip:

You can use colorsys to convert colors between different color systems.

https://docs.python.org/3/library/colorsys.html

RGB to HSV👇

import colorsys

# we need values between 0 and 1, you need to divide by 255
rgb = (0.00, 0.19, 0.56)  # 00308F | (0, 48, 143) -> AirForce Blue

print(colorsys.rgb_to_hsv(*rgb))
# => (0.6101190476190476, 1.0, 0.56)

Python - __post_init__ method in Dataclasses

Python tip:

You can use __post_init__ on classes decorated with dataclass to add custom logic on init.

For example, to set a value of an attribute based on a different attribute's value:

from dataclasses import dataclass, field


@dataclass
class Order:
    net: float
    vat: float
    total: float = field(init=False)

    def __post_init__(self):
        self.total = self.net + self.vat


order = Order(net=100.00, vat=22.00)
print(order)  # => Order(net=100.0, vat=22.0, total=122.0)

Semi-immutable Python objects with frozen dataclasses

Python tip:

You can freeze instances of a class decorated with dataclass by setting frozen to True.

https://docs.python.org/3/library/dataclasses.html#frozen-instances

An error will be raised if you try to assign a new value to any of attributes.

For example:

from dataclasses import dataclass


@dataclass(frozen=True)
class Post:
    title: str
    content: str


post = Post(title="Happy new year", content="2020 is finally over")

post.title = "Awesome new year"

"""
Traceback (most recent call last):
  File "example.py", line 12, in <module>
    post.title = "Awesome new year"
  File "<string>", line 4, in __setattr__
dataclasses.FrozenInstanceError: cannot assign to field 'title'
"""

Python's monthrange() method

Python tip:

You can use monthrange from the calendar module to get the weekday of the first day in a month and the number of days in a month.

An example👇

import calendar

weekday, number_of_days = calendar.monthrange(2021, 2)

print(calendar.day_name[weekday], number_of_days)

# Monday 28

Python weekday() method from the calendar module

Python tip:

Use weekday from the calendar module to get a day in a week (0 is Monday) for a specific date.

An example👇

import calendar

days = {
    0: "Monday",
    1: "Tuesday",
    2: "Wednesday",
    3: "Thursday",
    4: "Friday",
    5: "Saturday",
    6: "Saturday",
}

day_in_week = calendar.weekday(1975, 10, 23)
print(days[day_in_week])
# Thursday

Test interactive Python examples with doctest

Python tip:

You can use the doctest module to run interactive code examples inside your docstrings.

It notifies you If any of the examples don't return the expected value.

👇

def sum_ab(a, b):
    """
    Sum numbers a and b
    >>> sum_ab(1, 3)
    4
    >>> sum_ab(-41, 50)
    1
    :return: sum of 2 numbers
    """
    return a + b


# python -m doctest example.py
# **********************************************************************
# File "example.py", line 4, in example.sum_ab
# Failed example:
#     sum_ab(-41, 50)
# Expected:
#     1
#     :return: sum of 2 numbers
# Got:
#     9
# **********************************************************************
# 1 items had failures:
#    1 of   2 in example.sum_ab
# ***Test Failed*** 1 failures.

Python itertools.zip_longest() Example

Python tip:

You can use zip_longest from itertools to zip multiple iterables of different lengths.

"Missing" values from shorter iterables are replaced by fillvalue.

An example 👇

from itertools import zip_longest

students = ["Bob", "Ann", "John", "Marry", "Daisy", "Amy"]
grades = ["A", "A+", "D"]

for student, grade in zip_longest(students, grades, fillvalue="-"):
    print(student, grade)

How do I use itertools.groupby()?

Python tip:

You can use groupby from itertools to group elements.

1. Returns - an iterator of consecutive keys and groups from the iterable.
2. Keys - values returned from key function
3. Groups - iterator returns values for group

For example, to group tax IDs 👇

import re
from itertools import groupby

tax_ids = [
    "DE123456789",
    "ATU99999999",
    "BG999999999",
    "IT12345678900",
    "DE987654321",
]

grouped_tax_ids = groupby(
    tax_ids, key=lambda tax_id: re.match(r"[A-Z]{2}", tax_id).group()
)

for country, country_tax_ids in grouped_tax_ids:
    print(country, list(country_tax_ids))

# DE ['DE123456789']
# AT ['ATU99999999']
# BG ['BG999999999']
# IT ['IT12345678900']
# DE ['DE987654321']

Python's Itertools.cycle()

Python tip:

Do you need to repeat a sequence over and over again?

Use cycle from itertools 👇

from itertools import cycle

chord_sequence = cycle(["G", "D", "e", "C"])

song_chords = [next(chord_sequence) for _ in range(16)]

print(song_chords)
# ['G', 'D', 'e', 'C', 'G', 'D', 'e', 'C', 'G', 'D', 'e', 'C', 'G', 'D', 'e', 'C']

Django - Use foreign key values directly

Django tip:

Use foreign key values directly.

Make sure to use the foreign key property instead of the foreign object ID.

• DON'T: Song.objects.get(id=1).singer.id
• DO: Song.objects.get(id=1).singer_id

from django.db import connection, reset_queries

from .models import Song

reset_queries()

# makes an extra SQL query to load the singer object:
Song.objects.get(id=1).singer.id
len(connection.queries) # => 2

reset_queries()

# an extra SQL query is not required:
Song.objects.get(id=1).singer_id
len(connection.queries) # => 1

Python - itertools.count()

Python tip:

You can use count from itertools to make an iterator that returns evenly spaced values:

itertools.count(start=0, step=1)

For example, to generate all odd numbers greater or equal to 101👇

from itertools import count

odd_numbers = count(101, 2)

# first ten elements
for _ in range(10):
    print(next(odd_numbers))

"""
101
103
105
107
109
111
113
115
117
119
"""

Python - itertools.combinations()

Python tip:

You can generate all combinations of elements from an iterable of length n by using combinations from itertools.

For example, you can generate all possible match pairs 👇

from itertools import combinations

players = ["John", "Daisy", "Marry", "Bob"]

print(list(combinations(players, 2)))

"""
[
    ('John', 'Daisy'),
    ('John', 'Marry'),
    ('John', 'Bob'),
    ('Daisy', 'Marry'),
    ('Daisy', 'Bob'),
    ('Marry', 'Bob'),
]
"""

Function overloading with singledispatchmethod from functools

Python tip (>=3.8):

You can use singledispatchmethod to implement different (re)actions based on the first (non-self, non-cls) function's argument type.

For each type, you register an overloaded implementation.

For example, you can handle different events 👇

from functools import singledispatchmethod


class Event:
    pass


class UserSignedUp(Event):
    def __init__(self, email):
        self.email = email


class CustomerCanceledSubscription(Event):
    def __init__(self, email):
        self.email = email


class EventHandler:
    @singledispatchmethod
    def handle_event(event: Event):
        pass  # do nothing

    @handle_event.register
    def _(self, event: UserSignedUp):
        print(f"I will prepare resources for newly signed up user: {event.email}")

    @handle_event.register
    def _(self, event: CustomerCanceledSubscription):
        print(f"I will disable resources for customer:  {event.email}")


events = [
    UserSignedUp(email="[email protected]"),
    CustomerCanceledSubscription(email="[email protected]"),
]

handler = EventHandler()
for event in events:
    handler.handle_event(event)

# I will prepare resources for newly signed up user: [email protected]
# I will disable resources for customer:  [email protected]

Python functools - total_ordering()

Python tip:

You can create an orderable class (assuming the type is totally ordered) with __eq__, one other comparison methods (__ne__, __lt__, __le__, __gt__, __ge__), and the total_ordering decorator.

An example👇

from functools import total_ordering


@total_ordering
class Sample:
    def __init__(self, value):
        self.value = value

    def __lt__(self, other):
        return self.value < other.value


x = Sample(3)
y = Sample(4)

for expression in ["x == y", "x != y", "x < y", "x <= y", "x > y", "x >= y"]:
    result = eval(expression)
    print(f"{expression} is {result}")


"""
x == y is False
x != y is True
x < y is True
x <= y is True
x > y is False
x >= y is False
"""

Without it, you'd need to define all of the comparison methods.

Provision and manage remote Docker hosts with Docker Machine

Docker tip:

Use Docker Machine to quickly provision a remote machine for testing

https://docs.docker.com/machine/

# spin up an ec2 instance with docker engine installed
$ docker-machine create \
    --driver amazonec2 \
    --amazonec2-open-port 8000 \
    --amazonec2-region us-east-1 \
    --amazonec2-instance-type "t2.micro" \
    sandbox

# point your docker daemon at the remote docker engine
$ docker-machine env sandbox
$ eval $(docker-machine env sandbox)

# build the images and spin up the containers on the remote machine
$ docker-compose up -d --build

# ssh certs are auto generated, so you can easily ssh to the remote
$ docker-machine ssh sandbox

# you can also copy files to/ from the machine
$ docker-machine scp docker-machine scp sandbox:~/foo.txt ~/
$ docker-machine scp docker-machine scp foo.tar.gz sandbox:/home/ubuntu

Python functools - cached_property()

Python tip (>=3.8):

You can use cached_property from functools to cache the results of a class attribute for the life of the instance. It's useful if you have a property that is expensive to compute and doesn't need to change.

An example 👇

import io
import pathlib
from functools import cached_property

from pikepdf import Pdf


class PDF:
    def __init__(self, file_bytes):
        self.file_bytes = file_bytes

    @cached_property
    def number_of_pages(self):
        return len(Pdf.open(io.BytesIO(self.file_bytes)).pages)


pdf = PDF(pathlib.Path("file.pdf").read_bytes())
print(pdf.number_of_pages)

What's the difference between select_related and prefetch_related in Django?

Django tip:

Select_related vs prefetch_related

• Use select_related() on OneToOneField or ForeignKey when you need a single object
• Use prefetch_related() on ManyToManyFields or reverse relations when you need many objects

👇

class Singer(models.Model):
    first_name = models.CharField(max_length=40)
    last_name = models.CharField(max_length=40)


class Song(models.Model):
    name = models.CharField(max_length=100)
    singer = models.ForeignKey(Singer, related_name="songs")


Song.objects.select_related("singer").all()  # Forward ForeignKey relationship
Singer.objects.select_related("song").all()  # Backward ForeignKey relationship

Python - function overloading with singledispatch

Python tip:

You can use singledispatch to implement different (re)actions based on the first function's argument type

For each type, you register an overloaded implementation

For example, you can handle different events like so:

from functools import singledispatch


class Event:
    pass


class UserSignedUp(Event):
    def __init__(self, email):
        self.email = email


class CustomerCanceledSubscription(Event):
    def __init__(self, email):
        self.email = email


@singledispatch
def handle_event(event: Event):
    pass  # do nothing


@handle_event.register(UserSignedUp)
def _(event: UserSignedUp):
    print(f"I will prepare resources for newly signed up user: {event.email}")


@handle_event.register(CustomerCanceledSubscription)
def _(event: CustomerCanceledSubscription):
    print(f"I will disable resources for customer:  {event.email}")


events = [
    UserSignedUp(email="[email protected]"),
    CustomerCanceledSubscription(email="[email protected]"),
]

for event in events:
    handle_event(event)

# I will prepare resources for newly signed up user: [email protected]
# I will disable resources for customer:  [email protected]

Use the Django auth system with a Single Page App (SPA)

Django tip:

Coupling Django with a front-end framework like React, Vue, or Angular? Use session cookies for auth (even cross-domain).

Why?

1. It's easier since Django has a powerful built-in auth system
2. It's safer than using JWTs and localStorage

https://testdriven.io/blog/django-spa-auth/

Caching in Python with lru_cache

Python tip:

Improve the performance of a function with the lru_cache decorator from functools:

@functools.lru_cache(maxsize=128, typed=False)

• maxsize: max number of stored arguments
• typed: different types will be cached separately (i.e., 3 != 3.0)
• function args and kwargs must be hashable
• perfect for when you need to periodically call an expensive function with the same arguments

Example:

import timeit
from functools import lru_cache

import requests


@lru_cache(maxsize=10, typed=False)
def open_web_page(website_url):
    requests.get(website_url)


without_cache = """
import requests

def open_web_page(website_url):
    requests.get(website_url)

urls = [
    'https://testdriven.io',
    'https://testdriven.io',
    'https://google.com',
    'https://google.com',
]

for url in urls:
    open_web_page(url)
"""

with_cache = """
from functools import lru_cache

import requests

@lru_cache(maxsize=10, typed=False)
def open_web_page(website_url):
    requests.get(website_url)

urls = [
    'https://testdriven.io',
    'https://testdriven.io',
    'https://google.com',
    'https://google.com',
]
for url in urls:
    open_web_page(url)
"""


print(timeit.timeit(without_cache, number=5))
# => 7.195018381

print(timeit.timeit(with_cache, number=5))
# => 3.6599477370000004

Partial functions in Python with partial from functools

Python tip:

You can use partial from functools to freeze a certain number of arguments from a function and create a new, simplified function.

It returns a new partial object which when called will behave like a function called with the positional args and keywords.

An example 👇

from functools import partial

all_users = [
    {"email": "[email protected]"},
    {"email": "[email protected]"},
    {"email": "[email protected]"},
]


def _sort_users_by_email(users, sort_order):
    asc_dsc_to_reverse = {
        "ascending": True,
        "descending": False,
    }

    return sorted(
        users, key=lambda user: user["email"], reverse=asc_dsc_to_reverse[sort_order]
    )


_sort_users_by_email_ascending = partial(_sort_users_by_email, sort_order="ascending")
_sort_users_by_email_descending = partial(_sort_users_by_email, sort_order="descending")


print(_sort_users_by_email_ascending(all_users))
# [{'email': '[email protected]'}, {'email': '[email protected]'}, {'email': '[email protected]'}]

print(_sort_users_by_email_descending(all_users))
# [{'email': '[email protected]'}, {'email': '[email protected]'}, {'email': '[email protected]'}]

Get the unique values from a list of dictionaries

Python tip:

You can use a dictionary comprehension to create a list of dictionaries unique by value on a selected key

users = [
    {"name": "John Doe", "email": "[email protected]"},
    {"name": "Mary Doe", "email": "[email protected]"},
    {"name": "Mary A. Doe", "email": "[email protected]"},
]


print(list({user["email"]: user for user in users}.values()))

"""
[
    {'name': 'John Doe', 'email': '[email protected]'},
    {'name': 'Mary A. Doe', 'email': '[email protected]'}
]

The same key can occur only once inside a dictionary.
Every element from a list is assigned to a key with a value of email.
Each new element with the same email overwrites any existing ones.
Therefore, only one element is left with the same value on the key.

.values() returns a dict_values object containing values from the dictionary,
list() converts the dict_values object back to a list.
"""

Subtracting two lists in Python

Python tip:

To subtract two lists (element by element), you can use a list comprehension and zip.

zip iterates over both lists -> you receive elements with the same index.

The list comprehension then creates a new list containing subtracted values.

👇

first = [1, 5, 7]
second = [5, 3, 0]

print([xi - yi for xi, yi in zip(first, second)])
# => [-4, 2, 7]

Handling missing keys with Python's defaultdict

Python tip:

You can use defaultdict to provide a default value for missing keys when accessing or assigning:

defaultdict(default_factory)

default_factory is a function used to initialize the value for the missing key (None is set by default).

If you pass the list constructor as the default_factory, a defaultdict is created with the values that are list.

For example:

import json
from collections import defaultdict

products = [
    {"name": "Cold beer", "price": 3.5, "category": "Beer"},
    {"name": "Coffee", "price": 1.5, "category": "Warm drinks"},
    {"name": "Cappuccino", "price": 2.0, "category": "Warm drinks"},
    {"name": "Chicken Sandwich", "price": 3.5, "category": "Snacks"},
    {"name": "Vegan Sandwich", "price": 3.5, "category": "Snacks"},
]

products_by_group = defaultdict(list)

for product in products:
    products_by_group[product["category"]].append(product)

print(json.dumps(products_by_group, indent=2))

"""
{
  "Beer": [
    {
      "name": "Cold beer",
      "price": 3.5,
      "category": "Beer"
    }
  ],
  "Warm drinks": [
    {
      "name": "Coffee",
      "price": 1.5,
      "category": "Warm drinks"
    },
    {
      "name": "Cappuccino",
      "price": 2.0,
      "category": "Warm drinks"
    }
  ],
  "Snacks": [
    {
      "name": "Chicken Sandwich",
      "price": 3.5,
      "category": "Snacks"
    },
    {
      "name": "Vegan Sandwich",
      "price": 3.5,
      "category": "Snacks"
    }
  ]
}
"""

Python Counting - finding the most common occurence

Python tip:

You can use Counter from the collections module to find the most common element in a:

• string - character with the most occurrences
• dict - key with highest value
• list - element with the most occurrences

most_common(n) -> returns n most common element

from collections import Counter

counter = Counter("TestDriven.io tutorials")
print(counter.most_common(1))  # character with the most occurrences
# => [('t', 3)]

counter = Counter({"red": 2, "green": 10, "blue": 7})
print(counter.most_common(1))  # key with the highest value
# => [('green', 10)]

counter = Counter([0, 2, 1, 1, 4, 10, 33, 21, 12, 10, 1])
print(counter.most_common(1))  # element with the most occurrences
# => [(1, 3)]

Python type hints - typing.Literal

Python tip:

You can use Literal to indicate that value can be one of the provided literals.

Static type checkers will report an error when the value doesn't match one of the provided literals.

https://mypy.readthedocs.io/en/stable/literal_types.html#literal-types

from typing import Literal

STATUS = Literal["ACTIVE", "DISABLED"]


class User:
    def __init__(self, username: str, status: STATUS):
        self.username = username
        self.status = status


user = User("[email protected]", "CREATED")

"""
mypy example.py
example.py:12: error: Argument 2 to "User" has incompatible type "Literal['CREATED']";
expected "Union[Literal['ACTIVE'], Literal['DISABLED']]"
Found 1 error in 1 file (checked 1 source file)
"""

Python type hints - creating a type alias

Python tip:

You can create a type alias to make your code more readable.

from typing import List

Vector = List[float]


def scale(scalar: float, vector: Vector) -> Vector:
    return [scalar * num for num in vector]

Python Type Hints - typing.Union

Python tip:

Use Union from the typing module to allow any of the listed types.

from typing import Union


def sum_ab(a: Union[int, float], b: Union[int, float]) -> Union[int, float]:
    return a + b

Python Type Hints - typing.TypedDict

Python (>=3.8) tip:

You can subclass TypedDict to create a type for dictionaries with fixed keys.

Static type checking will report an error when there are extra or missing keys.

from typing import TypedDict


class Song(TypedDict):
    name: str
    year: int


song: Song = {"name": "Enter Sandman", "year": 1991, "band": "Metallica"}

"""
mypy song.py
song.py:9: error: Extra key "band" for TypedDict "Song"
Found 1 error in 1 file (checked 1 source file)
"""

Python's final qualifier

Python (>=3.8) tip:

You can use the final decorator to declare that a:

1. Method should not be overridden
2. Class should not be subclassed

You can also use the Final annotation to declare that a variable or attribute should not be reassigned, redefined, or overridden. It can be used as indicator to another developer that the given variable should be treated as a constant:

from typing import Final 

_PI: Final =  3.14

A static type check will report an error when it's violated.

https://mypy.readthedocs.io/en/stable/final_attrs.html

from typing import Final, final

# example 1


@final
class Cat:
    @final
    def meow(self):
        return "Meow"


class WildCat(Cat):
    def meow(self):
        return "Grrr"


# mypy cat.py
# error: Cannot inherit from final class "Cat"


# example 2


class Foo:
    @final
    def bar(self):
        return "baz"


def foobar():
    return "foobar"


new = Foo()
new.bar = foobar

# mypy foo.py
# error: Cannot assign to final attribute "bar"


# example 3

favorite_color: Final[str] = "red"
favorite_color = "blue"

# mypy foo.py
# error: Cannot assign to final name "favorite_color"