Python 3.10: What's New

Python 3.10 was released on October 4th, 2021. This article looks at the most interesting new additions to the Python language that will help to make your Python code even cleaner:

1. Structural pattern matching
2. Parenthesized context managers
3. Clearer error messages
4. Improved type annotations
5. Strict argument for zipping

Contents

Installing Python 3.10

If you have Docker, you can quickly spin up a Python 3.10 shell to play around with the examples in this article like so:

$ docker run -it --rm python:3.10

Not using Docker? We recommend installing Python 3.10 with pyenv:

$ pyenv install 3.10.0

You can learn more about managing Python with pyenv from the Modern Python Environments - dependency and workspace management article.

Structural Pattern Matching

Out of all the new features, structural pattern matching got the most attention and is by far the most controversial. It introduces a match/case statement to the Python language, which looks very much like a switch/case statement in other programming languages. With structural pattern matching, you can test an object against one or more patterns to determine if the structure of the comparison object matches one of the given patterns.

Quick example:

code = 404

match code:
    case 200:
        print("OK")
    case 404:
        print("Not found")
    case 500:
        print("Server error")
    case _:
        print("Code not found")

Patterns can be:

1. literals
2. captures
3. wildcards
4. values
5. groups
6. sequences
7. mappings
8. classes

The above example uses the literal pattern.

Want to remove the magic numbers from this example? Leverage the value pattern like so:

from http import HTTPStatus

code = 404

match code:
    case HTTPStatus.OK:
        print("OK")
    case HTTPStatus.NOT_FOUND:
        print("Not found")
    case HTTPStatus.INTERNAL_SERVER_ERROR:
        print("Server error")
    case _:
        print("Code not found")


# => "Not found"

You can also combine multiple patterns using the OR pattern:

from http import HTTPStatus

code = 400

match code:
    case HTTPStatus.OK:
        print("OK")
    case HTTPStatus.NOT_FOUND | HTTPStatus.BAD_REQUEST:
        print("You messed up")
    case HTTPStatus.INTERNAL_SERVER_ERROR | HTTPStatus.BAD_GATEWAY:
        print("Our bad")
    case _:
        print("Code not found")


# => "You messed up"

What makes structural pattern matching different from the switch/case syntax in other languages is that you can unpack complex data types and perform actions based on the resulting data:

point = (0, 10)

match point:
    case (0, 0):
        print("Origin")
    case (0, y):
        print(f"Y={y}")
    case (x, 0):
        print(f"X={x}")
    case (x, y):
        print(f"X={x}, Y={y}")
    case _:
        raise ValueError("Not a point")


# => Y=10

Here, the value (10) is bound from the subject ((0, 10)) to a variable inside the case. This is the capture pattern.

You can achieve nearly the same thing with the class pattern:

from dataclasses import dataclass


@dataclass
class Point:
    x: int
    y: int

point = Point(10, 10)

match point:
    case Point(x=0, y=0):
        print("Origin")
    case Point(x=0, y=y):
        print(f"On Y axis with Y={y}")
    case Point(x=x, y=0):
        print(f"On X axis with X={x}")
    case Point(x=x, y=y):
        print(f"Somewhere in a X, Y plane with X={x}, Y={y}")
    case _:
        print("Not a point")


# => Somewhere in a X, Y plane with X=10, Y=10

You use add an if clause using a guard like so:

from dataclasses import dataclass


@dataclass
class Point:
    x: int
    y: int

point = Point(7, 0)

match point:
    case Point(x, y) if x == y:
        print(f"The point is located on the diagonal Y=X at {x}.")
    case Point(x, y):
        print(f"Point is not on the diagonal.")

# => Point is not on the diagonal.

So, when the guard is False the next case is evaluated.

It's worth noting that match and case are soft keywords, so you can still use them as variable names in your existing code.

For more, refer to the official docs along with the associated PEPs:

1. PEP 622 - proposal
2. PEP 634 - specification
3. PEP 635 - motivation and rationale
4. PEP 636 - tutorial

Parenthesized Context Managers

Python now supports continuation across multiple lines when using context managers:

with (
    CtxManager1() as ctx1,
    CtxManager2() as ctx2
):

In previous versions you had to keep everything in the same line or nest with statements.

Python < 3.10:

import unittest
from unittest import mock

from my_module import my_function


class Test(unittest.TestCase):
    def test(self):
        with mock.patch("my_module.secrets.token_urlsafe") as a, mock.patch("my_module.string.capwords") as b, mock.patch("my_module.collections.defaultdict") as c:
            my_function()
            a.assert_called()
            b.assert_called()
            c.assert_called()

    def test_same(self):
        with mock.patch("my_module.secrets.token_urlsafe") as a:
            with mock.patch("my_module.string.capwords") as b:
                with mock.patch("my_module.collections.defaultdict") as c:
                    my_function()
                    a.assert_called()
                    b.assert_called()
                    c.assert_called()

Python >= 3.10:

class Test(unittest.TestCase):
    def test(self):
        with (
            mock.patch("my_module.secrets.token_urlsafe") as a,
            mock.patch("my_module.string.capwords") as b,
            mock.patch("my_module.collections.defaultdictl") as c,
        ):
            my_function()
            a.assert_called()
            b.assert_called()
            c.assert_called()

It's worth noting that Python's switch to a PEG-based parser in Python 3.9 enabled this feature. We should see new features and changes, like this, in each new Python release for the foreseeable future, which should (1) lead to an even more elegant syntax and (2) upset everyone.

More info:

1. Release notes
2. PEP-617
3. Original bug

Clearer Error Messages

Python 3.10 has improved error messages, delivering more precise information about the error and where the error actually occurs.

For example, prior to Python 3.10, if you're missing a closing } bracket --

import datetime

expected = {'Jan', 'Mike', 'Marry',

today = datetime.datetime.today()

-- you'll see the following error message:

  File "example.py", line 5
    today = datetime.datetime.today()
          ^
SyntaxError: invalid syntax

With Python 3.10, you'll see:

  File "example.py", line 3
    expected = {'Jan', 'Mike', 'Marry',
               ^
SyntaxError: '{' was never closed

As you can tell, it's much easier to spot the actual issue with the new error message. This is especially helpful for beginners, making it easier to identify the real cause of the error for-

1. Missing keywords
2. Incorrect or misspelled keywords or variable names
3. Missing colons
4. Incorrect indentation
5. Missing a closing bracket or brace

Refer to the official docs for more.

It looks like Python 3.11 will ship another improvement to error messages as well.

Improved Type Annotations

Python 3.10 comes with a number of improvements related to type annotations:

1. PEP 604: Allow writing union types as X | Y
2. PEP 613: Explicit Type Aliases
3. PEP 647: User-Defined Type Guards
4. PEP 612: Parameter Specification Variables

Union Operator

First, you'll also be able to use a new type union operator, |, so you can express either type X or Y rather than having to import Union from the typing module:

# before
from typing import Union

def sum_xy(x: Union[int, float], y: Union[int, float]) -> Union[int, float]:
    return x + y


# after
def sum_xy(x: int | float, y: int | float) -> int | float:
    return x + y

More info:

1. Official docs
2. PEP-604

Type Aliases

Another bit of sugar is the ability to explicitly define type aliases. Static type checkers, as well as other developers, sometimes have problems distinguishing between variable assignments and type aliases.

For example:

StrCache = "Cache[str]"    # a type alias
LOG_PREFIX = "LOG[DEBUG]"  # a module constant

With Python 3.10, you can use TypeAlias to explicitly define a type alias:

StrCache: TypeAlias = "Cache[str]"  # a type alias
LOG_PREFIX = "LOG[DEBUG]"           # a module constant

This will clear things up for the type checkers and hopefully for other developers as well reading your code.

More info:

1. Official docs
2. PEP-613

Type Guards

Type guards help with type narrowing, which is the process of moving a type from a less precise type (based on its definition) to a more precise type (within a program's code flow).

Take, for example, the following two flavors of is_employee:

# without type guards
def is_employee(user: User) -> bool:
    return isinstance(user, Employee)


# with type guards
from typing import TypeGuard

def is_employee(user: User) -> TypeGuard[Employee]:
    return isinstance(user, Employee)

So, with the second flavor of is_employee, when it returns True, the type checker will be able to narrow user down from User to Employee.

More info:

1. Official docs
2. PEP-647

Parameter Specification Variables

To enable real annotation of higher order functions (e.g., decorators), Python 3.10 adds typing.ParamSpec and typing.Concatenate.

For example (from PEP-612):

from typing import Awaitable, Callable, TypeVar


R = TypeVar("R")


def add_logging(f: Callable[..., R]) -> Callable[..., Awaitable[R]]:
    async def inner(*args: object, **kwargs: object) -> R:
        await log_to_database()
        return f(*args, **kwargs)
    return inner


@add_logging
def takes_int_str(x: int, y: str) -> int:
    return x + 7

await takes_int_str(1, "A")
await takes_int_str("B", 2) # fails at runtime

In this example, your code will fail at runtime -- when the decorated function is called with the incorrect arguments.

Now, you're able to enforce parameter types by using typing.ParamSpec:

from typing import Awaitable, Callable, ParamSpec, TypeVar


P = ParamSpec("P")
R = TypeVar("R")


def add_logging(f: Callable[P, R]) -> Callable[P, Awaitable[R]]:
    async def inner(*args: P.args, **kwargs: P.kwargs) -> R:
        await log_to_database()
        return f(*args, **kwargs)
    return inner


@add_logging
def takes_int_str(x: int, y: str) -> int:
    return x + 7


await takes_int_str(1, "A") # Accepted
await takes_int_str("B", 2) # Correctly rejected by the type checker

Type checkers, like mypy, will catch the error when the code is analyzed.

More info:

1. Official docs
2. PEP-612

Postponed Evaluation of Annotations

Rather than evaluating annotations at function definition time, postponed evaluation of annotations proposes to save them as strings in the built-in __annotations__ dictionary. Tools that leverage annotations at runtime will then need to evaluate the annotations explicitly via typing.get_type_hints() rather than relying on them already being evaluated.

This was originally set to be part of Python 3.10, but tools like pydantic struggled to leverage typing.get_type_hints() to obtain the types due to Python’s scoping rules. So, the Python Steering Council decided to postpone the change. We may see it in Python 3.11.

More info:

1. PEP 563, PEP 649, and the Future of Python Type Annotations
2. IMPORTANT: PEP 563, PEP 649 and the future of pydantic
3. The Future of FastAPI and Pydantic is Bright
4. PEP-563

Strict Argument for Zipping

What happens when you try to zip two iterables that don't have the same length?

names = ["Jan", "Mike", "Marry", "Daisy"]
grades = ["B+", "A", "A+"]

for name, grade in zip(names, grades):
    print(name, grade)

"""
Jan B+
Mike A
Marry A+
"""

Iteration stops when the end of the shorter iterable is reached.

Python 3.10 introduces a new strict keyword parameter to ensure at runtime all iterables have the same length:

names = ["Jan", "Mike", "Marry", "Daisy"]
grades = ["B+", "A", "A+"]

for name, grade in zip(names, grades, strict=True):
    print(name, grade)

# ValueError: zip() argument 2 is shorter than argument 1

Refer to PEP-618 to learn more.

Additional Updates and Optimizations

The distutils package is deprecated and will be completely removed in Python 3.12. Its functionality exists within setuptools and packaging.

Finally, Python 3.10 introduces several optimizations leading to improved performance. The str(), bytes() and bytearray() constructors are 30 to 40% faster for small objects. The runpy module now imports fewer modules, so python -m module-name is 1.4x faster on average.

Conclusion

As you can see, Python 3.10 brought many new features. Some old, rarely used features have been depreciated or removed altogether as well. This article just touched on the new features and changes to the language. Be sure to check out the official release notes for all the changes: What’s New In Python 3.10.

Happy Pythoning!