How do I concatenate two lists in Python?
Python tip:
You can use
+
to join two lists into a new list.a = [10, 2] b = [6, 3] print(a + b) # => [10, 2, 6, 3]
Python - create a list from a list repeated N times
Python tip:
You can create a new list with elements from the first list that are repeated as many times as you want by multiplying.
Fo example:
users = ["johndoe", "marry", "bob"] print(3 * users) # => ['johndoe', 'marry', 'bob', 'johndoe', 'marry', 'bob', 'johndoe', 'marry', 'bob']
Execute raw SQL queries in SQLAlchemy
Python SQLAlchemy tip:
You can use raw queries while still using SQLAlchemy models.
For example
user = session.query(Course).from_statement( text("""SELECT * FROM courses where title=:title""") ).params(title="Scalable FastAPI Applications on AWS").all()
Python - sep parameter in print()
Python tip:
You can pass as many values to print to the
print()
function as you want. You can also specify a custom separator.print("123", "456", "789") # => 123 456 789 print("123", "456", "789", sep="-") # => 123-456-789
How to flush output of print in Python?
Python tip:
You can set
flush=True
for theprint()
function to avoid buffering the output data and forcibly flush it:print("I'm awesome", flush=True)
Python - find the last occurrence of an item in a list with rindex()
Python tip:
You can use
.rindex()
to find the highest index in a string where a substring is found.👇
print("2021 was awesome. 2022 is going to be even more awesome.".rindex("awesome")) # => 48
Python - string ljust() method
Python tip:
You can use
.ljust()
to create a left-justified string of given width.string.ljust(width, fillchar)
Padding is a space, " ", by default.
print("Mike".ljust(10, "*")) # => Mike******
Python - string center() method
Python tip:
You can use
.center()
to create a centered string of given width.string.center(width, fillchar)
Padding on each side is a space, " ", by default.
print("Mike".center(10, "*")) # => ***Mike***
Python - lower() vs. casefold() for string matching and converting to lowercase
Python tip:
Use
.casfolde()
instead of.lower()
when you want to perform caseless operations when working with Unicode strings (for ASCII only strings they work the same) -- e.g., check if two strings are equal.# In German ß == ss print("straße".lower() == "strasse") # False print("straße".casefold() == "strasse") # True
Python - remove a prefix from a string
Python tip (>=3.9):
You can use
.removeprefix()
to remove the prefix from a string.For example, to remove a filename prefix:
invoice_filenames = ("INV_123.pdf", "INV_234.pdf", "INV_345.pdf") for invoice_filename in invoice_filenames: print(invoice_filename.removeprefix("INV_")) # 123.pdf # 234.pdf # 345.pdf
Python - remove a suffix from a string
Python tip (>=3.9):
You can remove the suffix of a string with
.removesuffix()
.For example, to remove the file type from a filename:
import pathlib filename = "cv.pdf" file_type_suffix = pathlib.Path(filename).suffix print(filename.removesuffix(file_type_suffix)) # => cv
Pytest - Only run tests that match a substring expression
Pytest tip:
You can filter and run only tests that contain or do not contain some substring in their name.
Examples:
# run all tests that contain login in their name $ pytest -k login # run all tests that do not contain login in their name $ pytest -k 'not login'
CSRF Protection in Flask with Flask-WTF
Flask tip:
You can use Flask-WTF to implement CSRF protection for your application.
Example:
from flask import Flask, Response, abort, redirect, render_template, request, url_for from flask_login import ( LoginManager, UserMixin, current_user, login_required, login_user, logout_user, ) from flask_wtf.csrf import CSRFProtect app = Flask(__name__) app.config.update( DEBUG=True, SECRET_KEY="secret_sauce", ) login_manager = LoginManager() login_manager.init_app(app) csrf = CSRFProtect() csrf.init_app(app) ...
You can read more here: https://testdriven.io/blog/csrf-flask/.
Contract Testing in Python
Python clean code tip:
Use contract testing when you want to verify the same behavior for different implementations.
Example:
import json import pathlib from dataclasses import dataclass import pytest @dataclass class User: username: str class InMemoryUserRepository: def __init__(self): self._users = [] def add(self, user): self._users.append(user) def get_by_username(self, username): return next(user for user in self._users if user.username == username) class JSONUserRepository: def __init__(self, file_path): self._users = json.load(pathlib.Path(file_path).open()) def add(self, user): self._users.append(user) def get_by_username(self, username): return next(user for user in self._users if user.username == username) class UserRepositoryContract: @pytest.fixture def repository(self): raise NotImplementedError('Not Implemented Yet') @pytest.fixture def username(self): return 'johndoe' @pytest.fixture def user(self, username): return User(username=username) def test_added_user_is_retrieved_by_username(self, username, user, repository): repository.add(user) assert repository.get_by_username(user.username).username == username class TestInMemoryUserRepository(UserRepositoryContract): @pytest.fixture def repository(self): return InMemoryUserRepository() class TestInJSONUserRepository(UserRepositoryContract): @pytest.fixture def repository(self, tmp_path): users_file = tmp_path/"user.json" users_file.write_text(json.dumps([])) return JSONUserRepository(users_file)
Simplify Testing with Dependency Injection
Python clean code tip:
Use dependency injection to simplify testing
Example:
from dataclasses import dataclass from fastapi import FastAPI @dataclass class User: username: str class StartUserOnboarding: def __init__(self, user_repository): self._user_repository = user_repository def execute(self, username): user = User(username=username) self._user_repository.add(user) class InMemoryUserRepository: def __init__(self): self._users = [] def add(self, user): self._users.append(user) def get_by_username(self, username): return next(user for user in self._users if user.username == username) class SQLiteUserRepository: def __init__(self, config): self._config = config def add(self, user): print(f"Running some SQL statements for insert DATABASE_PATH") def get_by_username(self, username): print(f"Running some SQL statements for fetch from {self._config.DATABASE_PATH}") def test_user_is_added_to_repository(): username = "[email protected]" repository = InMemoryUserRepository() use_case = StartUserOnboarding(user_repository=repository) use_case.execute(username) assert repository.get_by_username(username).username class ApplicationConfig: DATABASE_PATH = "db" app = FastAPI() @app.post("/users/start-onboarding", status_code=202) async def start_user_onboarding(username: str): StartUserOnboarding(SQLiteUserRepository(ApplicationConfig())).execute(username) return "OK"
Python - use enums to group related constants
Python clean code tip:
Use enums to group related constants.
Why?
- Autocomplete
- Static type checking
Example:
from dataclasses import dataclass from enum import Enum # bad ORDER_PLACED = "PLACED" ORDER_CANCELED = "CANCELED" ORDER_FULFILLED = "FULFILLED" @dataclass class Order: status: str order = Order(ORDER_PLACED) print(order) # better class OrderStatus(str, Enum): PLACED = "PLACED" CANCELED = "CANCELED" FULFILLED = "FULFILLED" @dataclass class Order: status: OrderStatus order = Order(OrderStatus.PLACED) print(order)
Interfaces in Python with Protocol Classes
Python clean code tip:
Use
Protocol
to define the interface required by your function/method instead of using real objects. This way your function/method defines what it needs.from typing import Protocol class ApplicationConfig: DEBUG = False SECRET_KEY = "secret-key" EMAIL_API_KEY = "api-key" # bad def send_email(config: ApplicationConfig): print(f"Send email using API key: {config.EMAIL_API_KEY}") # better class EmailConfig(Protocol): EMAIL_API_KEY: str def send_email_(config: EmailConfig): print(f"Send email using API key: {config.EMAIL_API_KEY}")
Python - Property-based Testing with Hypothesis
Python testing tip:
Rather than having to write different test cases for every argument you want to test, property-based testing generates a wide-range of random test data that's dependent on previous tests runs.
Use Hypothesis for this:
def increment(num: int) -> int: return num + 1 # regular test import pytest @pytest.mark.parametrize( 'number, result', [ (-2, -1), (0, 1), (3, 4), (101234, 101235), ] ) def test_increment(number, result): assert increment(number) == result # property-based test from hypothesis import given import hypothesis.strategies as st @given(st.integers()) def test_add_one(num): assert increment(num) == num - 1
Python - mock.create_autospec()
Python tip:
Use
mock.create_autospec()
to create a mock object with methods that have the same interface as the ones inside the original object.Example:
from unittest import mock import requests from requests import Response def get_my_ip(): response = requests.get( 'http://ipinfo.io/json' ) return response.json()['ip'] def test_get_my_ip(monkeypatch): my_ip = '123.123.123.123' response = mock.create_autospec(Response) response.json.return_value = {'ip': my_ip} monkeypatch.setattr( requests, 'get', lambda *args, **kwargs: response ) assert get_my_ip() == my_ip
Pytest - clean up resources at the end of a test session
Python clean test tip:
Clean up resources needed for test after the pytest session is finished -- i.e., drop test database, remove files added to the file system.
Example:
import csv import os import pathlib import pytest def list_users_from_csv(file_path): return [ {field_name: field_value for field_name, field_value in row.items()} for row in csv.DictReader( file_path.open(), skipinitialspace=True, fieldnames=["first_name", "last_name"], ) ] @pytest.fixture def users_csv_path(): # before test - create resource file_path = pathlib.Path("users.csv") file_path.write_text("Jan,Giacomelli") yield file_path # after test - remove resource file_path.unlink() def test_all_users_are_listed(users_csv_path): assert list_users_from_csv(users_csv_path) == [ {"first_name": "Jan", "last_name": "Giacomelli"} ]
Arrange-Act-Assert - testing pattern
Python clean test tip:
Structure your tests in an Arrange-Act-Assert way:
- Arrange - set-up logic
- Act - invokes the system you're about to test
- Assert - verifies that the action of the system under test behaves as expected
Example:
from dataclasses import dataclass @dataclass class User: first_name: str last_name: str def full_name(self): return f"{self.first_name} {self.last_name}" def test_full_name_consists_of_first_name_and_last_name(): # arrange first_name = "John" last_name = "Doe" user = User(first_name=first_name, last_name=last_name) # act full_name = user.full_name() # assert assert full_name == "John Doe"
Pytest - Parameterizing Tests
Python clean test tip:
Use pytest
parametrize
when you need multiple cases to prove a single behavior.Example:
import difflib import pytest def names_are_almost_equal(first, second): return difflib.SequenceMatcher(None, first, second).ratio() > 0.7 @pytest.mark.parametrize( "first,second", [ ("John", "Johny"), ("Many", "Mary"), ] ) def test_names_are_almost_equal(first, second): assert names_are_almost_equal(first, second) @pytest.mark.parametrize( "first,second", [ ("John", "Joe"), ("Daisy", "Serena"), ] ) def test_names_are_not_almost_equal(first, second): assert not names_are_almost_equal(first, second)
Hide irrelevant test data
Python clean test tip:
You should hide irrelevant data for the test.
Such information just increases the cognitive mental load, resulting in bloated tests.
Example:
import uuid from dataclasses import dataclass from enum import Enum from uuid import UUID import pytest class ProductCategory(str, Enum): BOOK = "BOOK" ELECTRONIC = "ELECTRONIC" @dataclass class Product: id: UUID price: int name: str category: ProductCategory class ShoppingCart: def __init__(self): self._products = [] def add(self, product): self._products.append(product) def calculate_total_price(self): return sum(product.price for product in self._products) # BAD - category, id, and name are irrelevant for this test def test_given_products_with_total_price_50_when_calculate_total_price_then_total_price_is_50_(): shopping_cart = ShoppingCart() shopping_cart.add(Product(uuid.uuid4(), 10, "Mobile phone case", ProductCategory.ELECTRONIC)) shopping_cart.add(Product(uuid.uuid4(), 20, "Never enough", ProductCategory.BOOK)) shopping_cart.add(Product(uuid.uuid4(), 20, "Mobile phone charger", ProductCategory.ELECTRONIC)) assert shopping_cart.calculate_total_price() == 50 # GOOD @pytest.fixture def product_with_price(): def _product_with_price(price): return Product(uuid.uuid4(), price, "Mobile phone case", ProductCategory.ELECTRONIC) return _product_with_price def test_given_products_with_total_price_50_when_calculate_total_price_then_total_price_is_50(product_with_price): shopping_cart = ShoppingCart() shopping_cart.add(product_with_price(10)) shopping_cart.add(product_with_price(20)) shopping_cart.add(product_with_price(20)) assert shopping_cart.calculate_total_price() == 50
Tests should use meaningful data
Python clean test tip:
Your tests should use meaningful data in order to provide examples of how to use your code.
Examples:
from dataclasses import dataclass @dataclass class Car: manufacture: str model: str vin_number: str top_speed: int class InMemoryCarRepository: def __init__(self): self._cars = [] def add(self, car): self._cars.append(car) def get_by_vin_number(self, vin_number): return next(car for car in self._cars if car.vin_number == vin_number) # BAD - non-existing manufacture and model, VIN number not matching manufacture and model, impossible to reach top speed def test_added_car_can_be_retrieved_by_vin_number_(): car = Car(manufacture="AAAA", model="BBB+", vin_number="2FTJW36M6LCA90573", top_speed=1600) repository = InMemoryCarRepository() repository.add(car) assert car == repository.get_by_vin_number(car.vin_number) # GOOD def test_added_car_can_be_retrieved_by_vin_number(): car = Car(manufacture="Jeep", model="Wrangler", vin_number="1J4FA29P4YP728937", top_speed=160) repository = InMemoryCarRepository() repository.add(car) assert car == repository.get_by_vin_number(car.vin_number)
What should tests cover?
Python clean test tip:
For the most part, the tests you write should cover:
- all happy paths
- edge/corner/boundary cases
- negative test cases
- security and illegal issues
👇
import uuid from dataclasses import dataclass from typing import Optional @dataclass class User: username: str class InMemoryUserRepository: def __init__(self): self._users = [] def add(self, user: User) -> None: self._users.append(user) def search(self, query: Optional[str] = None) -> list[User]: if query is None: return self._users else: return [ user for user in self._users if query in user.username ] # happy path def test_search_users_without_query_lists_all_users(): user1 = User(username="[email protected]") user2 = User(username="[email protected]") repository = InMemoryUserRepository() repository.add(user1) repository.add(user2) assert repository.search() == [user1, user2] # happy path def test_search_users_with_email_part_lists_all_matching_users(): user1 = User(username="[email protected]") user2 = User(username="[email protected]") user3 = User(username="[email protected]") repository = InMemoryUserRepository() repository.add(user1) repository.add(user2) repository.add(user3) assert repository.search("doe") == [user1, user3] # edge test case def test_search_users_with_empty_query_lists_all_users(): user1 = User(username="[email protected]") user2 = User(username="[email protected]") repository = InMemoryUserRepository() repository.add(user1) repository.add(user2) assert repository.search("") == [user1, user2] # negative test case def test_search_users_with_random_query_lists_zero_users(): user1 = User(username="[email protected]") repository = InMemoryUserRepository() repository.add(user1) assert repository.search(str(uuid.uuid4())) == [] # security test def test_search_users_with_sql_injection_has_no_effect(): user1 = User(username="[email protected]") repository = InMemoryUserRepository() repository.add(user1) repository.search("DELETE FROM USERS;") assert repository.search() == [user1]
Tests should validate themselves regardless of whether the test execution passes or fails
Python clean test tip:
A test should validate itself whether the test execution is passed or failed.
The self-validating test can avoid the need to do an evaluation manually by us.
Example:
from dataclasses import dataclass @dataclass class User: first_name: str last_name: str def fullname(self): return f"{self.first_name} {self.last_name}" # BAD def test_full_name_consists_of_first_name_and_last_name_manual(): first_name = "John" last_name = "Doe" user = User(first_name=first_name, last_name=last_name) print(user.fullname()) assert input("Is result correct? (Y/n)") == "Y" # GOOD def test_full_name_consists_of_first_name_and_last_name(): first_name = "John" last_name = "Doe" full_name = "John Doe" user = User(first_name=first_name, last_name=last_name) assert user.fullname() == full_name
Tests should be independent
Python clean test tip:
A test should not depend on the state of any other tests or external services.
👇
from dataclasses import dataclass import pytest @dataclass class User: username: str class InMemoryUserRepository: def __init__(self): self._users = [] def add(self, user: User) -> None: self._users.append(user) def get_by_username(self, username: str) -> User: return next( user for user in self._users if user.username == username ) # BAD - depends on persistence layer having user record at test time def test_get_by_username(): user = User(username="[email protected]") repository = InMemoryUserRepository() assert repository.get_by_username(user.username) == user # BAD - test_user_is_fetched_by_username will succeed only when running after test_added_user @pytest.fixture(scope="module") def repository(): return InMemoryUserRepository() def test_added_user(repository): user = User(username="[email protected]") assert repository.add(user) is None def test_user_is_fetched_by_username(repository): user = User(username="[email protected]") assert repository.get_by_username(user.username) == user # GOOD - makes sure it has all the needed data def test_added_user_is_fetched_by_username(): user = User(username="[email protected]") repository = InMemoryUserRepository() repository.add(user) assert repository.get_by_username(user.username) == user
Tests should be repeatable and deterministic
Python clean test tip:
Your tests should be repeatable in any environment.
They should be deterministic, always result in the same tests succeeding.
Example:
import random LOTTO_COMBINATION_LENGTH = 5 MIN_LOTTO_NUMBER = 1 MAX_LOTTO_NUMBER = 42 def lotto_combination(): combination = [] while len(combination) < LOTTO_COMBINATION_LENGTH: number = random.randint(MIN_LOTTO_NUMBER, MAX_LOTTO_NUMBER) if number not in combination: combination.append(number) return combination # BAD def test_lotto_combination(): assert lotto_combination() == [10, 33, 5, 7, 2] # GOOD def test_all_numbers_are_between_min_max_range(): assert all(MIN_LOTTO_NUMBER <= number <= MAX_LOTTO_NUMBER for number in lotto_combination()) def test_length_of_lotto_combination_has_expected_number_of_elements(): assert len(lotto_combination()) == LOTTO_COMBINATION_LENGTH
Shorten your feedback loops by increasing the speed of your test suite
Python clean test tip:
Your tests should be fast. The faster the tests the faster the feedback loop.
Consider using mocks or test doubles when dealing with third-party APIs and other slow things.
Example:
import time def fetch_articles(): print("I'm fetching articles from slow API") time.sleep(10) return {"articles": [{"title": "Facebook is Meta now."}]} # BAD def test_fetch_articles_slow(): assert fetch_articles() == {"articles": [{"title": "Facebook is Meta now."}]} # GOOD def test_fetch_articles_fast(monkeypatch): monkeypatch.setattr(time, "sleep", lambda timeout: None) assert fetch_articles() == {"articles": [{"title": "Facebook is Meta now."}]}
Tests should be useful
Python clean test tip:
Tests should protect you against regressions. They shouldn't just increase your code coverage percentage. Make sure they are useful! Don't just write tests for the sake of writing tests. They are code too, so they need to be maintained.
Example:
from dataclasses import dataclass @dataclass class User: first_name: str last_name: str def fullname(self): return f"{self.first_name} {self.last_name}" # BAD def test_full_name(): user = User(first_name="John", last_name="Doe") assert user.fullname() is not None # GOOD def test_full_name_consists_of_first_name_and_last_name(): first_name = "John" last_name = "Doe" full_name = "John Doe" user = User(first_name=first_name, last_name=last_name) assert user.fullname() == full_name
Test behavior, not implementation
Python clean test tip:
Tests should check the behavior rather than the underlying implementation details.
Such tests are easier to understand and maintain. They're also more resistant to refactoring (helps prevent false negatives).
👇
from dataclasses import dataclass @dataclass class User: username: str class InMemoryUserRepository: def __init__(self): self._users = [] def add(self, user): self._users.append(user) def get_by_username(self, username): return next(user for user in self._users if user.username == username) # BAD def test_add(): user = User(username="johndoe") repository = InMemoryUserRepository() repository.add(user) assert user in repository._users def test_get_by_username(): user = User(username="johndoe") repository = InMemoryUserRepository() repository._users = [user] user_from_repository = repository.get_by_username(user.username) assert user_from_repository == user # GOOD def test_added_user_can_be_retrieved_by_username(): user = User(username="johndoe") repository = InMemoryUserRepository() repository.add(user) assert user == repository.get_by_username(user.username)
Tests should fail for exactly one reason - aim for a single assert per test
Python clean test tip:
Aim for a single assert per test. Tests will be more readable and it's easier to locate a defect when a test is failing.
Example:
import pytest class User: def __init__(self, username): if len(username) < 1: raise Exception("Username must not be empty.") self._username = username @property def username(self): return self._username # BAD def test_user(): username = "johndoe" assert User(username).username == username username = "" with pytest.raises(Exception): User(username) # GOOD def test_user_with_valid_username_can_be_initialized(): username = "johndoe" assert User(username).username == username def test_user_with_empty_username_cannot_be_initialized(): username = "" with pytest.raises(Exception): User(username)
It's fine to deviate from this, to include multiple asserts per test as long as you're testing the same concept.
Testing Naming Conventions - GIVEN-WHEN-THEN
Python clean test tip:
Tests should have descriptive names to reveal their intention. For example, you could follow GIVEN-WHEN-THEN or SHOULD-WHEN naming conventions:
import pytest from fastapi import FastAPI from fastapi.testclient import TestClient from pydantic import BaseModel app = FastAPI() class LoginRequest(BaseModel): username: str password: str @app.post("/login") def login(data: LoginRequest): return {"access_token": "1234"} @pytest.fixture() def client(): yield TestClient(app) # BAD def test_login(client): response = client.post("/login", json={"username": "johndoe", "password": "correct_password"}) assert response.status_code == 200 assert response.json()["access_token"] == "1234" # GOOD def test_valid_username_and_password_combination_can_be_exchanged_for_access_token(client): response = client.post("/login", json={"username": "johndoe", "password": "correct_password"}) assert response.status_code == 200 assert response.json()["access_token"] == "1234" def test_given_valid_username_and_password_combination_when_user_calls_login_then_access_token_is_returned(client): response = client.post("/login", json={"username": "johndoe", "password": "correct_password"}) assert response.status_code == 200 assert response.json()["access_token"] == "1234" def test_access_token_should_be_returned_when_valid_username_and_password_combination_is_provided(client): response = client.post("/login", json={"username": "johndoe", "password": "correct_password"}) assert response.status_code == 200 assert response.json()["access_token"] == "1234"
Docker - Use COPY --chown instead of RUN chown after COPY in Dockerfile
Docker best practice:
Use
--chown
option of Docker's COPY command instead of doing it manually to reduce build time.# manually changing owner COPY . $APP_HOME RUN chown -r app:app $APP_HOME # using --chown option COPY --chown=app:app . $APP_HOME
Docker and Python Virtual Environments
Docker tip:
You can use a virtual environment instead of building wheels in multi-stage builds.
For example:
# temp stage FROM python:3.9-slim as builder WORKDIR /app ENV PYTHONDONTWRITEBYTECODE 1 ENV PYTHONUNBUFFERED 1 RUN apt-get update && \ apt-get install -y --no-install-recommends gcc RUN python -m venv /opt/venv ENV PATH="/opt/venv/bin:$PATH" COPY requirements.txt . RUN pip install -r requirements.txt # final stage FROM python:3.9-slim COPY --from=builder /opt/venv /opt/venv WORKDIR /app ENV PATH="/opt/venv/bin:$PATH"
Note: This is one of the only use cases for using a Python virtual environment with Docker.
- Install the dependencies in the builder image within a virtual environment.
- Copy over the dependencies to the final image
This reduces the size of the final image significantly.
Docker Logging Best Practices - stdout and stderr
Docker best practice:
Your Docker applications should log to standard output (stdout) and standard error (stderr) rather than to a file.
You can then configure the Docker daemon to send your log messages to a centralized logging solution (like CloudWatch or Papertrail).
Set Docker Memory and CPU Limits
Docker best practice:
Limit CPU and memory for your containers to prevent crippling the rest of the containers on the machine.
Examples:
# using docker run $ docker run --cpus=2 -m 512m nginx # using docker-compose version: "3.9" services: redis: image: redis:alpine deploy: resources: limits: cpus: 2 memory: 512M reservations: cpus: 1 memory: 256M
Sign and Verify Docker Images
Docker best practice:
Sign and verify your Docker images to prevent running images that have been tampered with.
To verify the integrity and authenticity of an image, set the
DOCKER_CONTENT_TRUST
environment variable:DOCKER_CONTENT_TRUST=1
Lint and Scan Your Dockerfiles and Images
Docker best practice:
Lint and scan your Dockerfiles and images to check your code for programmatic and stylistic errors and bad practices that could lead to potential flaws.
Some options:
👇
hadolint Dockerfile Dockerfile:1 DL3006 warning: Always tag the version of an image explicitly Dockerfile:7 DL3042 warning: Avoid the use of cache directory with pip. Use `pip install --no-cache-dir <package>` Dockerfile:9 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation. Dockerfile:17 DL3025 warning: Use arguments JSON notation for CMD and ENTRYPOINT arguments
Use a .dockerignore File
A properly structured .dockerignore file can help:
- Decrease the size of the Docker image
- Speed up the build process
- Prevent unnecessary cache invalidation
- Prevent leaking secrets
Example:
**/.git **/.gitignore **/.vscode **/coverage **/.env **/.aws **/.ssh Dockerfile README.md docker-compose.yml **/.DS_Store **/venv **/env
Don't Embed Secrets in Docker Images
Docker best practice:
Don't store secrets in Docker images.
Instead, they should be injected via:
- Environment variables (at run-time)
- Build-time arguments (at build-time)
- An orchestration tool like Docker Swarm (via Docker secrets) or Kubernetes (via Kubernetes secrets)
For more along with examples, check out Don't Store Secrets in Images.
Docker tagging best practices
Docker best practice:
Version Docker images to know which version of your code is running and to simplify rollbacks. Avoid the
latest
tag.Examples:
docker build -t web-prod-a072c4e-0.1.4 .
Docker - include a HEALTHCHECK instruction
Docker best practice:
Use
HEALTHCHECK
to verify that the process running inside the container is healthy.For example, call the health check endpoint of your web app:
HEALTHCHECK CMD curl --fail http://localhost:8000 || exit 1
Docker - array vs string based CMD
Docker best practice:
Use array over string syntax in your Dockerfiles to handle signals properly:
# array (exec) CMD ["gunicorn", "-w", "4", "-k", "uvicorn.workers.UvicornWorker", "main:app"] # string (shell) CMD "gunicorn -w 4 -k uvicorn.workers.UvicornWorker main:app"
Using the string form causes Docker to run your process using bash, which doesn't handle signals properly. Since most shells don't process signals to child processes, if you use the shell format, CTRL-C (which generates a SIGTERM) may not stop a child process.
Docker - run only one process per container
Docker best practice:
Run only one process per container to make it easier to reuse and scale each of the individual services:
- Scaling - With each service being in a separate container, you can scale one of your web servers horizontally as needed to handle more traffic.
- Reusability - Perhaps you have another service that needs a containerized database. You can simply reuse the same database container without bringing two unnecessary services along with it.
- Logging - Coupling containers makes logging much more complex.
- Portability and Predictability - It's much easier to make security patches or debug an issue when there's less surface area to work with.
Docker - Cache Python Packages to the Docker Host
Docker best practice:
Cache Python packages to the Docker host by mounting a volume or using BuildKit.
Example Dockerfile:
# Mount volume option -v $HOME/.cache/pip-docker/:/root/.cache/pip # BuildKit # syntax = docker/dockerfile:1.2 ... COPY requirements.txt . RUN --mount=type=cache,target=/root/.cache/pip \ pip install -r requirements.txt ...
Docker ADD vs COPY
Docker best practice:
Prefer COPY over ADD when copying files from a location to a Docker image.
Use ADD to:
- download external files
- extract an archive to the destination
👇
# copy local files on the host to the destination COPY /source/path /destination/path ADD /source/path /destination/path # download external file and copy to the destination ADD http://external.file/url /destination/path # copy and extract local compresses files ADD source.file.tar.gz /destination/path
Docker - use unprivileged containers
Docker best practice:
Always run a container with a non-root user. Running as root inside the container is running as root in the Docker host. If an attacker gains access to your container, they have access to all the root privileges and can perform several attacks against the Docker host.
👇
RUN addgroup --system app && adduser --system --group app USER app
Dockerfile - Multiple RUN commands v. single chained RUN command
Docker best practice:
In your Dockerfile, combine commands to minimize the number of layers and therefore reduce the image size.
# 2 commands RUN apt-get update RUN apt-get install -y netcat # single command RUN apt-get update && apt-get install -y netcat
Results:
# docker history to see layers $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE dockerfile latest 180f98132d02 51 seconds ago 259MB $ docker history 180f98132d02 IMAGE CREATED CREATED BY SIZE COMMENT 180f98132d02 58 seconds ago COPY . . # buildkit 6.71kB buildkit.dockerfile.v0 <missing> 58 seconds ago RUN /bin/sh -c pip install -r requirements.t… 35.5MB buildkit.dockerfile.v0 <missing> About a minute ago COPY requirements.txt . # buildkit 58B buildkit.dockerfile.v0 <missing> About a minute ago WORKDIR /app ...
Which Docker base image should you use?
Docker best practice:
Use smaller base images for your application. *-slim is usually a good choice.
- faster building
- faster pushing
- faster pulling
REPOSITORY TAG IMAGE ID CREATED SIZE python 3.9.6-alpine3.14 f773016f760e 3 days ago 45.1MB python 3.9.6-slim 907fc13ca8e7 3 days ago 115MB python 3.9.6-slim-buster 907fc13ca8e7 3 days ago 115MB python 3.9.6 cba42c28d9b8 3 days ago 886MB python 3.9.6-buster cba42c28d9b8 3 days ago 886MB 5:17
Pay close attention to the order of your Dockerfile commands to leverage layer caching
Docker best practice:
Order Dockerfile commands appropriately to better leverage caching.
Example:
# sample.py is copied before requirements.txt # dependencies will be installed for every change to sample.py FROM python:3.9-slim WORKDIR /app COPY sample.py . COPY requirements.txt . RUN pip install -r /requirements.txt # sample.py is copied after requirements.txt # dependencies will be installed only for changes to requirements.txt # when there are no changes, Docker cache will be used FROM python:3.9-slim WORKDIR /app COPY requirements.txt . RUN pip install -r /requirements.txt COPY sample.py .
Docker multi-stage builds
Docker best practice:
Use multistage builds to reduce the size of the production image.
# temp stage FROM python:3.9-slim as builder WORKDIR /app ENV PYTHONDONTWRITEBYTECODE 1 ENV PYTHONUNBUFFERED 1 RUN apt-get update && \ apt-get install -y --no-install-recommends gcc COPY requirements.txt . RUN pip wheel --no-cache-dir --no-deps --wheel-dir /app/wheels -r requirements.txt # final stage FROM python:3.9-slim WORKDIR /app COPY --from=builder /app/wheels /wheels COPY --from=builder /app/requirements.txt . RUN pip install --no-cache /wheels/*
Serving files with Python's HTTP server
Python tip:
When you need to just serve your static files inside a folder you can do that with Python's HTTP server:
$ cat index.html <html> <h1>Website Prototype</h1> <h2>List of Users:</h2> <ul> <li>Patrick</li> <li>Jan</li> </ul> </html> $ python3 -m http.server Serving HTTP on :: port 8000 (http://[::]:8000/) ...
Python docstrings examples
Python Clean Code Tip:
Use docstrings to document usage of your modules, classes, and functions.
""" The temperature module: Manipulate your temperature easily Easily calculate daily average temperature """ from typing import List class HighTemperature: """Class representing very high temperatures""" def __init__(self, value: float): """ :param value: value of temperature """ self.value = value def daily_average(temperatures: List[float]) -> float: """ Get average daily temperature Calculate average temperature from multiple measurements :param temperatures: list of temperatures :return: average temperature """ return sum(temperatures) / len(temperatures)
Do not store secrets in plaintext in code
Python Clean Code Tip:
Avoid storing things like secret keys, passwords, connection strings, and API keys inside your code. Instead, use a secrets management solution like AWS Secrets Manager or Vault.
# bad class ProductionConfig: DEBUG = False TESTING = False APP_ENVIRONMENT = "production" SQLALCHEMY_DATABASE_URI = ( "postgresql://my_user:[email protected]_server:5432/my_db" ) # better import boto3 class ProductionConfig: DEBUG = False TESTING = False APP_ENVIRONMENT = "production" _SQLALCHEMY_DATABASE_URI = None @property def SQLALCHEMY_DATABASE_URI(self): if self._SQLALCHEMY_DATABASE_URI is None: self._SQLALCHEMY_DATABASE_URI = boto3.client( "secretsmanager" ).get_secret_value(SecretId=f"db-connection-string-{self.APP_ENVIRONMENT}")[ "SecretString" ] return self._SQLALCHEMY_DATABASE_URI
If a secrets management tool is overkill for your project, store secrets in environment variables. Never store them in plaintext in your code.
Python - use real objects over primitive types
Python Clean Code Tip:
Favor real objects over primitive types such as dictionaries.
Why?
- It's easier to type
user.name
rather thanuser['name']
- You'll get help from your IDE
- You can actually check your code before it runs with mypy
- It makes your code more clear
# bad user = {"first_name": "John", "last_name": "Doe"} full_name = f"{user['first_name']} {user['last_name']}" print(full_name) # => John Doe # better class User: def __init__(self, first_name, last_name): self.first_name = first_name self.last_name = last_name def full_name(self): return f"{self.first_name} {self.last_name}" user = User(first_name="John", last_name="Doe") print(user.full_name()) # => John Doe
Python - find minimum value using special comparator
Python Clean Code Tip:
Use
min
to find an element with minimal value inside an iterable. You can provide a custom function as akey
argument to serve as a key for the min comparison.temperatures = [22.3, 28.7, 15.3, 18.2] # without min min_temperature = 10000 for temperature in temperatures: if temperature < min_temperature: min_temperature = temperature print(min_temperature) # => 15.3 # with min min_temperature = min(temperatures) print(min_temperature) # => 15.3 # using key users = [ {"username": "johndoe", "height": 1.81}, {"username": "marrydoe", "height": 1.69}, {"username": "joedoe", "height": 2.03}, ] shortest_user = min(users, key=lambda user: user["height"]) print(shortest_user) # {'username': 'marrydoe', 'height': 1.69}
Be consistent with the order of your parameters
Python Clean Code Tip:
Be consistent with order of parameters for similar functions/methods. Don't confuse your readers.
# bad def give_first_dose_of_vaccine(person, vaccine): print(f"Give first dose of {vaccine} to {person}.") def give_second_dose_of_vaccine(vaccine, person): print(f"Give second dose of {vaccine} to {person}.") give_first_dose_of_vaccine("john", "pfizer") # Give first dose of pfizer to john. give_second_dose_of_vaccine("jane", "pfizer") # Give second dose of jane to pfizer. # good def give_first_dose_of_vaccine(person, vaccine): print(f"Give first dose of {vaccine} to {person}.") def give_second_dose_of_vaccine(person, vaccine): print(f"Give second dose of {vaccine} to {person}.") give_first_dose_of_vaccine("john", "pfizer") # Give first dose of pfizer to john. give_second_dose_of_vaccine("jane", "pfizer") # Give second dose of pfizer to jane.
Python - High-precision calculations with Decimal
Python Clean Code Tip:
Avoid using floats when you need precise results. Use
Decimal
instead.e.g. prices
👇
from dataclasses import dataclass # bad from decimal import Decimal @dataclass class Product: price: float print(Product(price=0.1 + 0.2)) # => Product(price=0.30000000000000004) # good @dataclass class Product: price: Decimal print(Product(price=Decimal("0.1") + Decimal("0.2"))) # => Product(price=Decimal('0.3'))
Python - OOP tip: set attributes in the constructor
Python Clean Code Tip:
Avoid setting attributes of your objects outside of the constructor. Instead, implement methods that map to real-world concepts.
Why?
To ensure attributes exist and are easily discoverable.
👇
from dataclasses import dataclass from enum import Enum from uuid import UUID class OrderStatus(str, Enum): PLACED = "PLACED" CANCELED = "CANCELED" FULFILLED = "FULFILLED" # bad @dataclass class Order: status: OrderStatus class CancelOrder: def __init__(self, order_repository): self.order_repository = order_repository def execute(self, order_id: UUID): order = self.order_repository.get_by_id(order_id) order.status = OrderStatus.CANCELED self.order_repository.save(order) # better class Order: def __init__(self, status: OrderStatus): self._status = status def cancel(self): self._status = OrderStatus.CANCELED class CancelOrder: def __init__(self, order_repository): self.order_repository = order_repository def execute(self, order_id: UUID): order = self.order_repository.get_by_id(order_id) order.cancel() self.order_repository.save(order)
Python - OOP tip: avoid using too many attributes on a single object
Python Clean Code Tip:
Avoid using too many attributes on a single object. Try to cluster them to improve cohesion, reduce coupling, and improve readability
👇
import datetime from dataclasses import dataclass # bad @dataclass class ExcelSheet: file_name: str file_encoding: str document_owner: str document_read_password: str document_write_password: str creation_time: datetime.datetime update_time: datetime.datetime # good @dataclass class FileProperties: name: str encoding: str @dataclass class SecurityProperties: owner: str read_password: str write_password: str @dataclass class DocumentDating: creation_time: datetime.datetime update_time: datetime.datetime @dataclass class ExcelSheet: file_properties: FileProperties security_properties: SecurityProperties document_dating: DocumentDating
Do not use bare except
Python Clean Code Tip:
Avoid empty except blocks -> try-except-pass.
They lead to hard-to-find bugs.
👇
# bad import logging def send_email(): print("Sending email") raise ConnectionError("Oops") try: send_email() except: # AVOID THIS pass # better logger = logging.getLogger(__name__) try: send_email() except ConnectionError as exc: logger.error(f"Cannot send email {exc}")
Python - use all uppercase for constants
Python Clean Code Tip:
Use upper case names for constants
👇
from typing import Final MAX_NUMBER_OF_RETRIES: Final = 666 class Driver: MAX_HEIGHT: Final = 190
Python type annotation specificity
Python tip:
Specify the most general type for inputs and the most specific type for outputs.
For example:
from typing import List def sum_of_elements(elements: List[int]) -> int: sum_el = 0 for element in elements: sum_el += element return sum_el print(sum_of_elements((9, 9))) """ $ mypy example.py example.py:13: error: Argument 1 to "sum_of_elements" has incompatible type "Tuple[int, int]"; expected "List[int]" Found 1 error in 1 file (checked 1 source file) """ from typing import Iterable def sum_of_elements(elements: Iterable[int]) -> int: sum_el = 0 for element in elements: sum_el += element return sum_el print(sum_of_elements((9, 9))) """ $ mypy example.py Success: no issues found in 1 source file """
Python: Check if an iterable contains a specific element
Python Clean Code Tip:
Use
in
to check whether an iterable contains a specific element.👇
lucky_numbers = [1, 23, 13, 1234] BEST_NUMBER = 13 # without in best_number_is_lucky_number = False for number in lucky_numbers: if number == BEST_NUMBER: best_number_is_lucky_number = True print(best_number_is_lucky_number) # => True # with in best_number_is_lucky_number = BEST_NUMBER in lucky_numbers print(best_number_is_lucky_number) # => True
Python type hints - descriptive variable names
Python Clean Code Tip:
Avoid using the variable/parameter type inside your variable/parameter name. Use type hints instead.
# BAD: user_list # GOOD: users: list[User]
Full example👇
from dataclasses import dataclass @dataclass class User: username: str # bad def print_users(user_list): for user in user_list: print(user.username) print_users([User(username="johndoe")]) # => johndoe # good def print_users(users: list[User]): for user in users: print(user.username) print_users([User(username="johndoe")]) # => johndoe
Python - avoid HTTP status code magic numbers with http.HTTPStatus()
Python Clean Code Tip:
Use
HTTPStatus
fromhttp
(it's inside the standard library) to avoid "magic" numbers for statuses inside your code.Example:
from http import HTTPStatus from fastapi import FastAPI app = FastAPI() @app.get("/old", status_code=200) async def old(): return {"message": "Hello World"} @app.get("/", status_code=HTTPStatus.OK) async def home(): return {"message": "Hello World"}
Python - splitting a module into multiple files
Python Clean Code Tip:
When your module becomes too big you can restructure it to a package while keeping all the imports from the module as they were.
👇
# BEFORE # models.py class Order: pass class Shipment: pass # └── models.py # AFTER # change to package # models/__init__.py from .order import Order from .shipment import Shipment __all__ = ["Order", "Shipment"] # models/order.py class Order: pass # models/shipment.py class Shipment: pass # └── models # ├── __init__.py # ├── order.py # └── shipment.py # imports from module/package can stay the same from models import Order, Shipment
Design by contract in Python - preconditions
Python Clean Code Tip:
Use preconditions to ensure the integrity of your objects.
For example:
class Date: def __init__(self, day, month, year): self.day = day self.month = month self.year = year startDate = Date(3, 11, 2020) # OK startDate = Date(31, 13, 2020) # this one should fail since there are only 12 months class Date: LAST_MONTH = 12 LAST_DAY = 31 def __init__(self, day, month, year): if month > self.LAST_MONTH: raise Exception(f"Month cannot be greater than {self.LAST_MONTH}") if day > self.LAST_DAY: raise Exception(f"Day cannot be greater than {self.LAST_DAY}") self.day = day self.month = month self.year = year startDate = Date(3, 11, 2020) # OK startDate = Date(31, 13, 2020) # this one fails # DISCLAIMER: production ready validation should be more complex since not all months have 31 days
Operator Overloading in Python
Python Clean Code Tip:
Use operator overloading to enable usage of operators such as
+
,-
,/
,*
, ... on your instances.👇
from dataclasses import dataclass # without operator overloading @dataclass class TestDrivenIOCoin: value: float def add(self, other): if not isinstance(other, TestDrivenIOCoin): return NotImplemented return TestDrivenIOCoin(value=self.value + other.value) my_coins = TestDrivenIOCoin(value=120).add(TestDrivenIOCoin(value=357.01)) print(my_coins) # TestDrivenIOCoin(value=477.01) # with operator overloading @dataclass class TestDrivenIOCoin: value: float def __add__(self, other): if not isinstance(other, TestDrivenIOCoin): return NotImplemented return TestDrivenIOCoin(value=self.value + other.value) my_coins = TestDrivenIOCoin(value=120) + TestDrivenIOCoin(value=357.01) print(my_coins) # TestDrivenIOCoin(value=477.01)
Chaining comparison operators in Python
Python Clean Code Tip:
Use chained comparison when you need to check whether some variable is between MIN and MAX values.
👇
from dataclasses import dataclass @dataclass class SurfBoard: width: float length: float MINIMAL_LENGTH = 201.3 MAXIMAL_LENGTH = 278.5 # without chained comparison def board_is_pwa_compliant(surf_board: SurfBoard): return surf_board.length > MINIMAL_LENGTH and surf_board.length < MAXIMAL_LENGTH surf_board = SurfBoard(width=75.3, length=202.7) print(board_is_pwa_compliant(surf_board)) # True # with chained comparison def board_is_pwa_compliant(surf_board: SurfBoard): return MINIMAL_LENGTH < surf_board.length < MAXIMAL_LENGTH print(board_is_pwa_compliant(surf_board)) # True # don't abuse it like this: a <= b < c > d
__all__ in Python
Python Clean Code Tip:
Use
__all__
to define exported members of your package.Hint: IDEs will do a much better job at importing and autocomplete.
from .my_module import my_function __all__ = ["my_function"]
Python - built-in sum function vs. for loop
Python Clean Code Tip:
Use
sum
to sum the values of all elements inside an iterable instead of afor
loop.Why?
- Don't re-invent the wheel!
sum
is much faster👇
transactions = [10.0, -5.21, 101.32, 1.11, -0.38] # without sum balance = 0 for transaction in transactions: balance += transaction # with sum balance = sum(transactions)
Python - Reduce Boilerplate Code with Dataclasses
Python Clean Code Tip:
Use dataclasses when only storing attributes inside your class instances to reduce the amount of boilerplate code.
For example:
# without dataclass class Address: def __init__(self, street, city, zip_code): self.street = street self.city = city self.zip_code = zip_code def __repr__(self): return ( f"Address(street={self.street}, city={self.city}, zip_code={self.zip_code})" ) def __hash__(self) -> int: return hash((self.street, self.city, self.zip_code)) def __eq__(self, other) -> bool: if not isinstance(other, Address): return NotImplemented return (self.street, self.city, self.zip_code) == ( other.street, other.city, other.zip_code, ) # with dataclass from dataclasses import dataclass @dataclass(unsafe_hash=True) class Address: street: str city: str zip_code: str
Check for code quality issues inside your CI/CD pipelines
Python Clean Code Tip:
Check the quality of your code inside your CI pipeline.
Use:
- flake8 - style guide enforcer
- black - code formatting
- isort - optimize imports
- bandit - check for security vulnerabilities
- safety - check for security vulnerabilities of dependencies
Github Actions Example 👇
name: Check code quality on: [push] jobs: code-quality: strategy: fail-fast: false matrix: python-version: [3.9] poetry-version: [1.1.8] os: [ubuntu-latest] runs-on: ${{ matrix.os }} steps: - uses: actions/[email protected] - uses: with: actions/[email protected] python-version: ${{ matrix. python-version }} - name: Run image uses: abatilo/[email protected] with: poetry-version: ${{ matrix. poetry-version }} - name: Install dependencies run: poetry install - name: Run black run: poetry run black . --check - name: Run isort run: poetry run isort . --check-only --profile black - name: Run flake8 run: poetry run flake8 . - name: Run bandit run: poetry run bandit . - name: Run saftey run: poetry run safety check
It's a good idea to couple this with pre-commit hooks:
- pre-commit - format code with black and isort
- CI pipeline - run black and isort with check flags to ensure that code has been properly formatted
In other words, you shouldn't actually format any code in the CI pipeline. You just want to verify that formatting happened via pre-commit.
Don't use flags in functions
Python Clean Code Tip:
Don't use flags in functions.
Flags are variables passed to functions, which the function uses to determine its behavior. This pattern should be avoided since functions should only perform a single task. If you find yourself doing this, split your function into smaller functions.
👇
text = "This is a cool blog post" # This is bad def transform(text, uppercase): if uppercase: return text.upper() else: return text.lower() # This is good def uppercase(text): return text.upper() def lowercase(text): return text.lower()
Python Clean Code: Keep your function arguments at a minimum
Python Clean Code Tip:
Keep your arguments at a minimum.
Ideally, your functions should only have one to two arguments. If you need to provide more arguments to the function, you can create a config object which you pass to the function or split it into multiple functions.
Example:
# This is bad def render_blog_post(title, author, created_timestamp, updated_timestamp, content): # ... render_blog_post("Clean code", "Nik Tomazic", 1622148362, 1622148362, "...") # This is good class BlogPost: def __init__(self, title, author, created_timestamp, updated_timestamp, content): self.title = title self.author = author self.created_timestamp = created_timestamp self.updated_timestamp = updated_timestamp self.content = content blog_post1 = BlogPost("Clean code", "Nik Tomazic", 1622148362, 1622148362, "...") def render_blog_post(blog_post): # ... render_blog_post(blog_post1)
Functions should only perform a single task
Python Clean Code Tip:
Functions should only perform a single task
Hint: If your function contains the keyword 'and' you can probably split it into two functions.
# This is bad def fetch_and_display_personnel(): data = # ... for person in data: print(person) # This is good def fetch_personnel(): return # ... def display_personnel(data): for person in data: print(person)
Clean code tip - Don't add unnecessary context
Python Clean Code Tip:
Don't add redundant context.
Do not add unnecessary data to variable names, especially if you're working with classes.
# This is bad class Person: def __init__(self, person_first_name, person_last_name, person_age): self.person_first_name = person_first_name self.person_last_name = person_last_name self.person_age = person_age # This is good class Person: def __init__(self, first_name, last_name, age): self.first_name = first_name self.last_name = last_name self.age = age
We're already inside the
Person
class, so there's no need to add aperson_
prefix to every class variable.
Clean code tip - Don't use magic numbers
Python Clean Code Tip:
Don't use "magic numbers".
Magic numbers are strange numbers that appear in code, which do not have a clear meaning.
👇
import random # This is bad def roll(): return random.randint(0, 36) # what is 36 supposed to represent? # This is good ROULETTE_POCKET_COUNT = 36 def roll(): return random.randint(0, ROULETTE_POCKET_COUNT)
Instead of using magic numbers, extract them into a meaningful variable.
Clean code tip - Avoid using ambiguous abbreviations
Python clean code tip:
Avoid using ambiguous abbreviations
Don't try to come up with your own abbreviations. It's better for a variable to have a longer name than a confusing name.
👇
# This is bad fna = 'Bob' cre_tmstp = 1621535852 # This is good first_name = 'Bob' creation_timestamp = 1621535852
Queryset.explain() in Django
Django tip:
If you want to know how the database would execute a given query, you can use
explain()
.Knowing this can be helpful when you're trying to improve the performance of slow queries.
>>> print(Payment.objects.filter(created_at__gt=datetime.date(2021, 1, 1)).explain()) Seq Scan on payments_payment (cost=0.00..14.25 rows=113 width=212) Filter: (created_at > '2021-01-01 00:00:00+00'::timestamp with time zone)
Django templates - lorem ipsum
Django tip:
You can generate lorem ipsum inside a Django template with the
lorem
tag.https://docs.djangoproject.com/en/3.2/ref/templates/builtins/#lorem
You can provide any (or none) of the following arguments:
count
- number of paragraphs or wordsmethod
- words/HTML paragraphs/plain-text paragraphsrandom
- doesn't use the common paragraph ("Lorem ipsum dolor sit amet...")Example:
{% lorem 6 p random &} # generates 6 paragraphs of text that doesn't # start with "Lorem ips um dolor sit amet"
Django - update_or_create
Django's
update_or_create()
method either-
- updates an existing object with the given kwargs along with a
defaults
dictionary of pairs for updating the object- creates a new object if it doesn't exist
It returns a tuple containing an object and a boolean specifying whether a new object was created.
Visitor.objects.create(name="Harry", surname="Potter", age=16) visitor, created = Visitor.objects.update_or_create( name="Harry", surname="Potter", defaults={"age": 21} ) print(visitor.age) # => '21' print(created) # => False
Django - get_or_create
Django's
get_or_create()
method either-
- gets an existing object with the given kwargs
- creates a new object if it doesn't exist
It returns a tuple containing an object and a boolean specifying whether a new object was created.
Visitor.objects.create(name="Harry", surname="Potter", age=16) visitor, created = Visitor.objects.get_or_create( name="Harry", surname="Potter", age=16 ) print(created) # => False visitor, created = Visitor.objects.get_or_create( name="Hermione", surname="Granger", age=16 ) print(created) # => True
Django QuerySet - only() vs defer() vs exclude()
If you have some fields in your Django model that contain a lot of data and you don't need those fields for a particular query, you can tell Django not to retrieve them with
defer()
:Event.objects.defer("description")
While
defer()
works at the attribute level,exclude()
works on the row level.In other words,
exclude()
arguments are used after theWHERE
clause -- i.e.,SELECT * FROM users WHERE name = 'jan'
-- whiledefer()
changes*
to the provided fields -- i.e.,SELECT name, email FROM users
.Opposite to
defer()
isonly()
. If you have fewer fields that you want to retrieve than those you don't, you can useonly()
to retrieve only the fields provided as arguments:Event.objects.only("title")
Check if a file is a symlink in Python
Python tip:
You can use pathlib's
is_symlink()
to check whether a path is a symlink.👇
import pathlib path = pathlib.Path("/usr/bin/python") print(path.is_symlink()) # => True
Using Django's get_object_or_404 shortcut
Django tip:
You can use
get_object_or_404
to raise theHttp404
exception when the object doesn't exist instead of handlingDoesNotExist
and raisingHttp404
by yourself.👇
from django.http import Http404 from django.shortcuts import get_object_or_404 def my_view(request): obj = get_object_or_404(MyModel, pk=1) # the above is equivalent to def my_view(request): try: obj = MyModel.objects.get(pk=1) except MyModel.DoesNotExist: raise Http404("No MyModel matches the given query.")
Find the union of two Django querysets
Django tip:
You can use
|
to create a union of multiple queries.👇
by_username = User.objects.filter(username="John") by_name = User.objects.filter(full_name="John") users = by_username | by_name
Mock AWS Services
Pytest tip:
Use moto to mock AWS services such as S3 and DynamoDB:
👇
import bot03 import pytest from moto import mock_dynamodb2 @pytest.fixture def dynamodb_table(): with mock_dynamodb2(): dynamodb = bot03.resource("dynamodb") table = dynamodb.create_table( TableName="test", KeySchema=[ {"AttributeName": "PK", "KeyType": "HASH"}, {"AttributeName": "SK", "KeyType": "Range"}, ], AttributeDefinitions=[ {"AttributeName": "PK", "AttributeType": "S"}, {"AttributeName": "SK", "AttributeType": "S"}, {"AttributeName": "GSIPK", "AttributeType": "S"}, {"AttributeName": "GSISK", "AttributeType": "S"}, ], GlobalSecondarylndexes=[ { "IndexName": "GS1", "KeySchema": [ {"AttributeName": "GS1PK", "KeyType": "HASH"}, {"AttributeName": "GS1SK", "KeyType": "Range"}, ], "Projection": {"ProjectionType": "ALL"}, }, ], ) table.delete()
Django REST Framework - Combining and Excluding Permission Classes
Did you know?
You can combine permissions in Django REST Framework using
&
,|
, and~
.👇
class MyModelViewSet(viewsets.ModelViewSet): permission_classes = IsAuthenticated & (IsAdminUser | IsFaculty | ReadOnly) class MyModelViewSet(viewsets.ModelViewSet): permission_classes = ~IsStudent & IsAuthenticated
For more, check out the Combining and Excluding Permission Classes section from Custom Permission Classes in Django REST Framework.
Asynchronous Background Tasks in FastAPI
FastAPI tip:
You can use FastAPI's BackGround Tasks to run simple tasks in the background.
👇
from fastapi import BackgroundTasks def send_email(email, message): pass @app.get("/") async def ping(background_tasks: BackgroundTasks): background_tasks.add_task(send_email, "[email protected]", "Hi!") return {"message": "pong!"}
Use Celery for CPU intensive tasks and when you need a task queue.
Python - slice a generator object
Python tip:
You can use itertools.islice to use only part of a generator.
👇
from itertools import cycle, islice chord_sequence = cycle(["G", "D", "e", "C"]) song_chords = [chord for chord in islice(chord_sequence, 16)] print(song_chords) """ ['G', 'D', 'e', 'C', 'G', 'D', 'e', 'C', 'G', 'D', 'e', 'C', 'G', 'D', 'e', 'C'] """
Flask - async and await
Flask tip:
With Flask >= 2.0 you can create asynchronous route handlers using async/await.
Example:
import asyncio async def async_get_data(): await asyncio.sleep(1) return "Done!" @app.route("/data") async def get_data(): data = await async_get_data() return data
Want to learn more? Check out Async in Flask 2.0.
Calculate the execution time of Flask views
Did you know?
You can use a decorator to time the execution of Flask views.
For example👇
from functools import wraps from timeit import default_timer def timer(f): @wraps(f) def wrapper(*args, **kwargs): start_time = default_timer() response = f(*args, **kwargs) total_elapsed_time = default_timer() - start_time response += f"<h3>Elapsedtime: {total_elapsed_time}</h3>" return response return wrapper @app.route("/") @timer def hello_world(): return "Hello World!"
Positional-only arguments in Python
Did you know?
You can force a user to call a function with positional arguments only using
/
.Example:
def full_name(user, /): return f"{user['first_name']} {user['last_name']}" print(full_name({"first_name": "Jan", "last_name": "Giamcomelli"})) # => Jan Giamcomelli print(full_name(user={"first_name": "Jan", "last_name": "Giamcomelli"})) # => TypeError: full_name() got some positional-only arguments passed as keyword arguments: 'user'
Why?
Makes refactoring easier. You can change the name of your parameters without worrying about it breaking any code that uses the function.
all() in Python
Python tip:
You can use
all
to check whether all values inside an iterable are truthy.👇
numbers = [10, 99, 321, 3] print(all(number > 0 for number in numbers)) # => True print(all([])) # => True numbers = [-1, 2, 5, 10, 0] print(all(number > 0 for number in numbers)) # => False
any() in Python
Python tip:
You can use
any
to check whether any element inside an iterable has a truthy value.An example👇
platforms = ["Facebook", "Twitter", "Instagram"] print(any(platform == "Twitter" for platform in platforms)) # => True print(any([])) # => False print(any([2, 3, 4])) # => True
Parse datetime strings in Python with dateutil
Did you know?
You can use
dateutil.parser
to parse all sorts of different date and datetime string formats.https://pypi.org/project/python-dateutil/
An example👇
from dateutil.parser import parse print(parse("2012-01-19 17:21:00")) # => 2012-01-19 17:21:00 print(parse("1st Jan, 2019")) # => 2019-01-01 00:00:00 print(parse("23.11.2020")) # => 2020-11-23 00:00:00
Unfortunately, this is quite slow:
from dateutil.parser import parse %timeit parse(datetime_txt).date() 54.3 µs ± 450 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each) import datetime %timeit datetime.datetime.strptime(datetime_txt, "%H:%M %d-%m-%Y").date() 7.44 µs ± 240 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
Python @classmethod
Python tip:
You can use
@classmethod
to create class methods.For example, you can create a class method that loads events from a JSON string message👇
import datetime import json class UserRegistered: def __init__(self, username, event_time): self.username = username self.event_time = event_time @classmethod def from_event_message(cls, message): message = json.loads(message) return cls( username=message["username"], event_time=datetime.datetime.fromisoformat(message["event_time"]), ) message = '{"username": "johndoe", "event_time": "2021-04-26T20:00:00"}' event = UserRegistered.from_event_message(message) print(event.username, event.event_time) # => johndoe 2021-04-26 20:00:00