.. meta::
:description lang=en: Collect useful snippets of Python typing
:keywords: Python3, Static Typing, Python Type hints, Type hints Cheat Sheet
======
Typing
======
PEP `484 `_, which provides a
specification about what a type system should look like in Python3, introduced
the concept of type hints. Moreover, to better understand the type hints design
philosophy, it is crucial to read PEP `483 `_
that would be helpful to aid a pythoneer to understand reasons why Python
introduce a type system. The main goal of this cheat sheet is to show some
common usage about type hints in Python3.
.. contents:: Table of Contents
:backlinks: none
Without type check
-------------------
.. code-block:: python
def fib(n):
a, b = 0, 1
for _ in range(n):
yield a
b, a = a + b, b
print([n for n in fib(3.6)])
output:
.. code-block:: bash
# errors will not be detected until runtime
$ python fib.py
Traceback (most recent call last):
File "fib.py", line 8, in
print([n for n in fib(3.5)])
File "fib.py", line 8, in
print([n for n in fib(3.5)])
File "fib.py", line 3, in fib
for _ in range(n):
TypeError: 'float' object cannot be interpreted as an integer
With type check
----------------
.. code-block:: python
# give a type hint
from typing import Generator
def fib(n: int) -> Generator:
a: int = 0
b: int = 1
for _ in range(n):
yield a
b, a = a + b, b
print([n for n in fib(3.6)])
output:
.. code-block:: bash
# errors will be detected before running
$ mypy --strict fib.py
fib.py:12: error: Argument 1 to "fib" has incompatible type "float"; expected "int"
Basic types
-----------
.. code-block:: python
import io
import re
from collections import deque, namedtuple
from typing import (
Dict,
List,
Tuple,
Set,
Deque,
NamedTuple,
IO,
Pattern,
Match,
Text,
Optional,
Sequence,
Iterable,
Mapping,
MutableMapping,
Any,
)
# without initializing
x: int
# any type
y: Any
y = 1
y = "1"
# built-in
var_int: int = 1
var_str: str = "Hello Typing"
var_byte: bytes = b"Hello Typing"
var_bool: bool = True
var_float: float = 1.
var_unicode: Text = u'\u2713'
# could be none
var_could_be_none: Optional[int] = None
var_could_be_none = 1
# collections
var_set: Set[int] = {i for i in range(3)}
var_dict: Dict[str, str] = {"foo": "Foo"}
var_list: List[int] = [i for i in range(3)]
var_static_length_Tuple: Tuple[int, int, int] = (1, 2, 3)
var_dynamic_length_Tuple: Tuple[int, ...] = (i for i in range(10, 3))
var_deque: Deque = deque([1, 2, 3])
var_nametuple: NamedTuple = namedtuple('P', ['x', 'y'])
# io
var_io_str: IO[str] = io.StringIO("Hello String")
var_io_byte: IO[bytes] = io.BytesIO(b"Hello Bytes")
var_io_file_str: IO[str] = open(__file__)
var_io_file_byte: IO[bytes] = open(__file__, 'rb')
# re
p: Pattern = re.compile("(https?)://([^/\r\n]+)(/[^\r\n]*)?")
m: Optional[Match] = p.match("https://www.python.org/")
# duck types: list-like
var_seq_list: Sequence[int] = [1, 2, 3]
var_seq_tuple: Sequence[int] = (1, 2, 3)
var_iter_list: Iterable[int] = [1, 2, 3]
var_iter_tuple: Iterable[int] = (1, 2, 3)
# duck types: dict-like
var_map_dict: Mapping[str, str] = {"foo": "Foo"}
var_mutable_dict: MutableMapping[str, str] = {"bar": "Bar"}
Functions
----------
.. code-block:: python
from typing import Generator, Callable
# function
def gcd(a: int, b: int) -> int:
while b:
a, b = b, a % b
return a
# callback
def fun(cb: Callable[[int, int], int]) -> int:
return cb(55, 66)
# lambda
f: Callable[[int], int] = lambda x: x * 2
Classes
--------
.. code-block:: python
from typing import ClassVar, Dict, List
class Foo:
x: int = 1 # instance variable. default = 1
y: ClassVar[str] = "class var" # class variable
def __init__(self) -> None:
self.i: List[int] = [0]
def foo(self, a: int, b: str) -> Dict[int, str]:
return {a: b}
foo = Foo()
foo.x = 123
print(foo.x)
print(foo.i)
print(Foo.y)
print(foo.foo(1, "abc"))
Generator
----------
.. code-block:: python
from typing import Generator
# Generator[YieldType, SendType, ReturnType]
def fib(n: int) -> Generator[int, None, None]:
a: int = 0
b: int = 1
while n > 0:
yield a
b, a = a + b, b
n -= 1
g: Generator = fib(10)
i: Iterator[int] = (x for x in range(3))
Asynchronous Generator
-----------------------
.. code-block:: python
import asyncio
from typing import AsyncGenerator, AsyncIterator
async def fib(n: int) -> AsyncGenerator:
a: int = 0
b: int = 1
while n > 0:
await asyncio.sleep(0.1)
yield a
b, a = a + b, b
n -= 1
async def main() -> None:
async for f in fib(10):
print(f)
ag: AsyncIterator = (f async for f in fib(10))
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Context Manager
---------------
.. code-block:: python
from typing import ContextManager, Generator, IO
from contextlib import contextmanager
@contextmanager
def open_file(name: str) -> Generator:
f = open(name)
yield f
f.close()
cm: ContextManager[IO] = open_file(__file__)
with cm as f:
print(f.read())
Asynchronous Context Manager
-----------------------------
.. code-block:: python
import asyncio
from typing import AsyncContextManager, AsyncGenerator, IO
from contextlib import asynccontextmanager
# need python 3.7 or above
@asynccontextmanager
async def open_file(name: str) -> AsyncGenerator:
await asyncio.sleep(0.1)
f = open(name)
yield f
await asyncio.sleep(0.1)
f.close()
async def main() -> None:
acm: AsyncContextManager[IO] = open_file(__file__)
async with acm as f:
print(f.read())
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Avoid ``None`` access
----------------------
.. code-block:: python
import re
from typing import Pattern, Dict, Optional
# like c++
# std::regex url("(https?)://([^/\r\n]+)(/[^\r\n]*)?");
# std::regex color("^#?([a-f0-9]{6}|[a-f0-9]{3})$");
url: Pattern = re.compile("(https?)://([^/\r\n]+)(/[^\r\n]*)?")
color: Pattern = re.compile("^#?([a-f0-9]{6}|[a-f0-9]{3})$")
x: Dict[str, Pattern] = {"url": url, "color": color}
y: Optional[Pattern] = x.get("baz", None)
print(y.match("https://www.python.org/"))
output:
.. code-block:: bash
$ mypy --strict foo.py
foo.py:15: error: Item "None" of "Optional[Pattern[Any]]" has no attribute "match"
Positional-only arguments
--------------------------
.. code-block:: python
# define arguments with names beginning with __
def fib(__n: int) -> int: # positional only arg
a, b = 0, 1
for _ in range(__n):
b, a = a + b, b
return a
def gcd(*, a: int, b: int) -> int: # keyword only arg
while b:
a, b = b, a % b
return a
print(fib(__n=10)) # error
print(gcd(10, 5)) # error
output:
.. code-block:: bash
mypy --strict foo.py
foo.py:1: note: "fib" defined here
foo.py:14: error: Unexpected keyword argument "__n" for "fib"
foo.py:15: error: Too many positional arguments for "gcd"
Multiple return values
-----------------------
.. code-block:: python
from typing import Tuple, Iterable, Union
def foo(x: int, y: int) -> Tuple[int, int]:
return x, y
# or
def bar(x: int, y: str) -> Iterable[Union[int, str]]:
# XXX: not recommend declaring in this way
return x, y
a: int
b: int
a, b = foo(1, 2) # ok
c, d = bar(3, "bar") # ok
Union[Any, None] == Optional[Any]
----------------------------------
.. code-block:: python
from typing import List, Union
def first(l: List[Union[int, None]]) -> Union[int, None]:
return None if len(l) == 0 else l[0]
first([None])
# equal to
from typing import List, Optional
def first(l: List[Optional[int]]) -> Optional[int]:
return None if len(l) == 0 else l[0]
first([None])
Be careful of ``Optional``
---------------------------
.. code-block:: python
from typing import cast, Optional
def fib(n):
a, b = 0, 1
for _ in range(n):
b, a = a + b, b
return a
def cal(n: Optional[int]) -> None:
print(fib(n))
cal(None)
output:
.. code-block:: bash
# mypy will not detect errors
$ mypy foo.py
Explicitly declare
.. code-block:: python
from typing import Optional
def fib(n: int) -> int: # declare n to be int
a, b = 0, 1
for _ in range(n):
b, a = a + b, b
return a
def cal(n: Optional[int]) -> None:
print(fib(n))
output:
.. code-block:: bash
# mypy can detect errors even we do not check None
$ mypy --strict foo.py
foo.py:11: error: Argument 1 to "fib" has incompatible type "Optional[int]"; expected "int"
Be careful of casting
----------------------
.. code-block:: python
from typing import cast, Optional
def gcd(a: int, b: int) -> int:
while b:
a, b = b, a % b
return a
def cal(a: Optional[int], b: Optional[int]) -> None:
# XXX: Avoid casting
ca, cb = cast(int, a), cast(int, b)
print(gcd(ca, cb))
cal(None, None)
output:
.. code-block:: bash
# mypy will not detect type errors
$ mypy --strict foo.py
Forward references
-------------------
Based on PEP 484, if we want to reference a type before it has been declared, we
have to use **string literal** to imply that there is a type of that name later on
in the file.
.. code-block:: python
from typing import Optional
class Tree:
def __init__(
self, data: int,
left: Optional["Tree"], # Forward references.
right: Optional["Tree"]
) -> None:
self.data = data
self.left = left
self.right = right
.. note::
There are some issues that mypy does not complain about Forward References.
Get further information from `Issue#948`_.
.. _Issue\#948: https://github.com/python/mypy/issues/948
.. code-block:: python
class A:
def __init__(self, a: A) -> None: # should fail
self.a = a
output:
.. code-block:: bash
$ mypy --strict type.py
$ echo $?
0
$ python type.py # get runtime fail
Traceback (most recent call last):
File "type.py", line 1, in
class A:
File "type.py", line 2, in A
def __init__(self, a: A) -> None: # should fail
NameError: name 'A' is not defined
Postponed Evaluation of Annotations
-----------------------------------
**New in Python 3.7**
- PEP 563_ - Postponed Evaluation of Annotations
.. _563: https://www.python.org/dev/peps/pep-0563/
Before Python 3.7
.. code-block:: python
>>> class A:
... def __init__(self, a: A) -> None:
... self._a = a
...
Traceback (most recent call last):
File "", line 1, in
File "", line 2, in A
NameError: name 'A' is not defined
After Python 3.7 (include 3.7)
.. code-block:: python
>>> from __future__ import annotations
>>> class A:
... def __init__(self, a: A) -> None:
... self._a = a
...
.. note::
Annotation can only be used within the scope which names have already
existed. Therefore, **forward reference** does not support the case which
names are not available in the current scope. **Postponed evaluation
of annotations** will become the default behavior in Python 4.0.
Type alias
----------
Like ``typedef`` or ``using`` in c/c++
.. code-block:: cpp
#include
#include
#include
#include
typedef std::string Url;
template using Vector = std::vector;
int main(int argc, char *argv[])
{
Url url = "https://python.org";
std::regex p("(https?)://([^/\r\n]+)(/[^\r\n]*)?");
bool m = std::regex_match(url, p);
Vector v = {1, 2};
std::cout << m << std::endl;
for (auto it : v) std::cout << it << std::endl;
return 0;
}
Type aliases are defined by simple variable assignments
.. code-block:: python
import re
from typing import Pattern, List
# Like typedef, using in c/c++
# PEP 484 recommend capitalizing alias names
Url = str
url: Url = "https://www.python.org/"
p: Pattern = re.compile("(https?)://([^/\r\n]+)(/[^\r\n]*)?")
m = p.match(url)
Vector = List[int]
v: Vector = [1., 2.]
Define a ``NewType``
---------------------
Unlike alias, ``NewType`` returns a separate type but is identical to the original type at runtime.
.. code-block:: python
from sqlalchemy import Column, String, Integer
from sqlalchemy.ext.declarative import declarative_base
from typing import NewType, Any
# check mypy #2477
Base: Any = declarative_base()
# create a new type
Id = NewType('Id', int) # not equal alias, it's a 'new type'
class User(Base):
__tablename__ = 'User'
id = Column(Integer, primary_key=True)
age = Column(Integer, nullable=False)
name = Column(String, nullable=False)
def __init__(self, id: Id, age: int, name: str) -> None:
self.id = id
self.age = age
self.name = name
# create users
user1 = User(Id(1), 62, "Guido van Rossum") # ok
user2 = User(2, 48, "David M. Beazley") # error
output:
.. code-block:: bash
$ python foo.py
$ mypy --ignore-missing-imports foo.py
foo.py:24: error: Argument 1 to "User" has incompatible type "int"; expected "Id"
Further reading:
- `Issue\#1284`_
.. _`Issue\#1284`: https://github.com/python/mypy/issues/1284
Using ``TypeVar`` as template
------------------------------
Like c++ ``template ``
.. code-block:: cpp
#include
template
T add(T x, T y) {
return x + y;
}
int main(int argc, char *argv[])
{
std::cout << add(1, 2) << std::endl;
std::cout << add(1., 2.) << std::endl;
return 0;
}
Python using ``TypeVar``
.. code-block:: python
from typing import TypeVar
T = TypeVar("T")
def add(x: T, y: T) -> T:
return x + y
add(1, 2)
add(1., 2.)
Using ``TypeVar`` and ``Generic`` as class template
----------------------------------------------------
Like c++ ``template class``
.. code-block:: cpp
#include
template
class Foo {
public:
Foo(T foo) {
foo_ = foo;
}
T Get() {
return foo_;
}
private:
T foo_;
};
int main(int argc, char *argv[])
{
Foo f(123);
std::cout << f.Get() << std::endl;
return 0;
}
Define a generic class in Python
.. code-block:: python
from typing import Generic, TypeVar
T = TypeVar("T")
class Foo(Generic[T]):
def __init__(self, foo: T) -> None:
self.foo = foo
def get(self) -> T:
return self.foo
f: Foo[str] = Foo("Foo")
v: int = f.get()
output:
.. code-block:: bash
$ mypy --strict foo.py
foo.py:13: error: Incompatible types in assignment (expression has type "str", variable has type "int")
Scoping rules for ``TypeVar``
------------------------------
- ``TypeVar`` used in different generic function will be inferred to be different types.
.. code-block:: python
from typing import TypeVar
T = TypeVar("T")
def foo(x: T) -> T:
return x
def bar(y: T) -> T:
return y
a: int = foo(1) # ok: T is inferred to be int
b: int = bar("2") # error: T is inferred to be str
output:
.. code-block:: bash
$ mypy --strict foo.py
foo.py:12: error: Incompatible types in assignment (expression has type "str", variable has type "int")
- ``TypeVar`` used in a generic class will be inferred to be same types.
.. code-block:: python
from typing import TypeVar, Generic
T = TypeVar("T")
class Foo(Generic[T]):
def foo(self, x: T) -> T:
return x
def bar(self, y: T) -> T:
return y
f: Foo[int] = Foo()
a: int = f.foo(1) # ok: T is inferred to be int
b: str = f.bar("2") # error: T is expected to be int
output:
.. code-block:: bash
$ mypy --strict foo.py
foo.py:15: error: Incompatible types in assignment (expression has type "int", variable has type "str")
foo.py:15: error: Argument 1 to "bar" of "Foo" has incompatible type "str"; expected "int"
- ``TypeVar`` used in a method but did not match any parameters which declare in ``Generic`` can be inferred to be different types.
.. code-block:: python
from typing import TypeVar, Generic
T = TypeVar("T")
S = TypeVar("S")
class Foo(Generic[T]): # S does not match params
def foo(self, x: T, y: S) -> S:
return y
def bar(self, z: S) -> S:
return z
f: Foo[int] = Foo()
a: str = f.foo(1, "foo") # S is inferred to be str
b: int = f.bar(12345678) # S is inferred to be int
output:
.. code-block:: bash
$ mypy --strict foo.py
- ``TypeVar`` should not appear in body of method/function if it is unbound type.
.. code-block:: python
from typing import TypeVar, Generic
T = TypeVar("T")
S = TypeVar("S")
def foo(x: T) -> None:
a: T = x # ok
b: S = 123 # error: invalid type
output:
.. code-block:: bash
$ mypy --strict foo.py
foo.py:8: error: Invalid type "foo.S"
Restricting to a fixed set of possible types
----------------------------------------------
``T = TypeVar('T', ClassA, ...)`` means we create a **type variable with a value restriction**.
.. code-block:: python
from typing import TypeVar
# restrict T = int or T = float
T = TypeVar("T", int, float)
def add(x: T, y: T) -> T:
return x + y
add(1, 2)
add(1., 2.)
add("1", 2)
add("hello", "world")
output:
.. code-block:: bash
# mypy can detect wrong type
$ mypy --strict foo.py
foo.py:10: error: Value of type variable "T" of "add" cannot be "object"
foo.py:11: error: Value of type variable "T" of "add" cannot be "str"
``TypeVar`` with an upper bound
--------------------------------
``T = TypeVar('T', bound=BaseClass)`` means we create a **type variable with an upper bound**.
The concept is similar to **polymorphism** in c++.
.. code-block:: cpp
#include
class Shape {
public:
Shape(double width, double height) {
width_ = width;
height_ = height;
};
virtual double Area() = 0;
protected:
double width_;
double height_;
};
class Rectangle: public Shape {
public:
Rectangle(double width, double height)
:Shape(width, height)
{};
double Area() {
return width_ * height_;
};
};
class Triangle: public Shape {
public:
Triangle(double width, double height)
:Shape(width, height)
{};
double Area() {
return width_ * height_ / 2;
};
};
double Area(Shape &s) {
return s.Area();
}
int main(int argc, char *argv[])
{
Rectangle r(1., 2.);
Triangle t(3., 4.);
std::cout << Area(r) << std::endl;
std::cout << Area(t) << std::endl;
return 0;
}
Like c++, create a base class and ``TypeVar`` which bounds to the base class.
Then, static type checker will take every subclass as type of base class.
.. code-block:: python
from typing import TypeVar
class Shape:
def __init__(self, width: float, height: float) -> None:
self.width = width
self.height = height
def area(self) -> float:
return 0
class Rectangle(Shape):
def area(self) -> float:
width: float = self.width
height: float = self.height
return width * height
class Triangle(Shape):
def area(self) -> float:
width: float = self.width
height: float = self.height
return width * height / 2
S = TypeVar("S", bound=Shape)
def area(s: S) -> float:
return s.area()
r: Rectangle = Rectangle(1, 2)
t: Triangle = Triangle(3, 4)
i: int = 5566
print(area(r))
print(area(t))
print(area(i))
output:
.. code-block:: bash
$ mypy --strict foo.py
foo.py:40: error: Value of type variable "S" of "area" cannot be "int"
@overload
----------
Sometimes, we use ``Union`` to infer that the return of a function has multiple
different types. However, type checker cannot distinguish which type do we want.
Therefore, following snippet shows that type checker cannot determine which type
is correct.
.. code-block:: python
from typing import List, Union
class Array(object):
def __init__(self, arr: List[int]) -> None:
self.arr = arr
def __getitem__(self, i: Union[int, str]) -> Union[int, str]:
if isinstance(i, int):
return self.arr[i]
if isinstance(i, str):
return str(self.arr[int(i)])
arr = Array([1, 2, 3, 4, 5])
x:int = arr[1]
y:str = arr["2"]
output:
.. code-block:: bash
$ mypy --strict foo.py
foo.py:16: error: Incompatible types in assignment (expression has type "Union[int, str]", variable has type "int")
foo.py:17: error: Incompatible types in assignment (expression has type "Union[int, str]", variable has type "str")
Although we can use ``cast`` to solve the problem, it cannot avoid typo and ``cast`` is not safe.
.. code-block:: python
from typing import List, Union, cast
class Array(object):
def __init__(self, arr: List[int]) -> None:
self.arr = arr
def __getitem__(self, i: Union[int, str]) -> Union[int, str]:
if isinstance(i, int):
return self.arr[i]
if isinstance(i, str):
return str(self.arr[int(i)])
arr = Array([1, 2, 3, 4, 5])
x: int = cast(int, arr[1])
y: str = cast(str, arr[2]) # typo. we want to assign arr["2"]
output:
.. code-block:: bash
$ mypy --strict foo.py
$ echo $?
0
Using ``@overload`` can solve the problem. We can declare the return type explicitly.
.. code-block:: python
from typing import Generic, List, Union, overload
class Array(object):
def __init__(self, arr: List[int]) -> None:
self.arr = arr
@overload
def __getitem__(self, i: str) -> str:
...
@overload
def __getitem__(self, i: int) -> int:
...
def __getitem__(self, i: Union[int, str]) -> Union[int, str]:
if isinstance(i, int):
return self.arr[i]
if isinstance(i, str):
return str(self.arr[int(i)])
arr = Array([1, 2, 3, 4, 5])
x: int = arr[1]
y: str = arr["2"]
output:
.. code-block:: bash
$ mypy --strict foo.py
$ echo $?
0
.. warning::
Based on PEP 484, the ``@overload`` decorator just **for type checker only**, it does not implement
the real overloading like c++/java. Thus, we have to implement one exactly non-``@overload``
function. At the runtime, calling the ``@overload`` function will raise ``NotImplementedError``.
.. code-block:: python
from typing import List, Union, overload
class Array(object):
def __init__(self, arr: List[int]) -> None:
self.arr = arr
@overload
def __getitem__(self, i: Union[int, str]) -> Union[int, str]:
if isinstance(i, int):
return self.arr[i]
if isinstance(i, str):
return str(self.arr[int(i)])
arr = Array([1, 2, 3, 4, 5])
try:
x: int = arr[1]
except NotImplementedError as e:
print("NotImplementedError")
output:
.. code-block:: bash
$ python foo.py
NotImplementedError
Stub Files
----------
Stub files just like header files which we usually use to define our interfaces in c/c++.
In python, we can define our interfaces in the same module directory or ``export MYPYPATH=${stubs}``
First, we need to create a stub file (interface file) for module.
.. code-block:: bash
$ mkdir fib
$ touch fib/__init__.py fib/__init__.pyi
Then, define the interface of the function in ``__init__.pyi`` and implement the module.
.. code-block:: python
# fib/__init__.pyi
def fib(n: int) -> int: ...
# fib/__init__.py
def fib(n):
a, b = 0, 1
for _ in range(n):
b, a = a + b, b
return a
Then, write a test.py for testing ``fib`` module.
.. code-block:: python
# touch test.py
import sys
from pathlib import Path
p = Path(__file__).parent / "fib"
sys.path.append(str(p))
from fib import fib
print(fib(10.0))
output:
.. code-block:: bash
$ mypy --strict test.py
test.py:10: error: Argument 1 to "fib" has incompatible type "float"; expected "int"