How to add type annotations to Python dataclasses
During the COVID19 outbreak, a lof of companies are either giving away free books, tutorials, or access their training material. One such company is Pluralsight. For the whole month of April 2020, they have given everyone free access to their complete library! The content is just awesome, some of the best I've seen. I have been working through their Python - Beyond the Basics when I came across their section on Properties and Class Methods.
Sample program
Their sample program was pretty cool:
import iso6346
class ShippingContainer:
HEIGHT_FT = 8.5
WIDTH_FT = 8.0
next_serial = 1337
@staticmethod
def _make_bic_code(owner_code, serial):
return iso6346.create(owner_code=owner_code,
serial=str(serial).zfill(6))
@classmethod
def _get_next_serial(cls):
result = cls.next_serial
cls.next_serial += 1
return result
@classmethod
def create_empty(cls, owner_code, length_ft, *args, **kwargs):
return cls(owner_code, length_ft, contents=None, *args, **kwargs)
@classmethod
def create_with_items(cls, owner_code, length_ft, items, *args, **kwargs):
return cls(owner_code, length_ft, contents=list(items), *args, **kwargs)
def __init__(self, owner_code, length_ft, contents):
self.contents = contents
self.length_ft = length_ft
self.bic = self._make_bic_code(
owner_code=owner_code,
serial=ShippingContainer._get_next_serial())
@property
def volume_ft3(self):
return ShippingContainer.HEIGHT_FT * ShippingContainer.WIDTH_FT * self.length_ft
class RefrigeratedShippingContainer(ShippingContainer):
MAX_CELSIUS = 4.0
@staticmethod
def _make_bic_code(owner_code, serial):
return iso6346.create(owner_code=owner_code,
serial=str(serial).zfill(6),
category='R')
@staticmethod
def _c_to_f(celsius):
return celsius * 9/5 + 32
@staticmethod
def _f_to_c(fahrenheit):
return (fahrenheit - 32) * 5/9
def __init__(self, owner_code, length_ft, contents, celsius):
super().__init__(owner_code, length_ft, contents)
self.celsius = celsius
@property
def celsius(self):
return self._celsius
@celsius.setter
def celsius(self, value):
if value > RefrigeratedShippingContainer.MAX_CELSIUS:
raise ValueError("Temperature too hot!")
self._celsius = value
@property
def fahrenheit(self):
return RefrigeratedShippingContainer._c_to_f(self.celsius)
@fahrenheit.setter
def fahrenheit(self, value):
self.celsius = RefrigeratedShippingContainer._f_to_c(value)
@property
def volume_ft3(self):
return super().volume_ft3 - RefrigeratedShippingContainer.FRIDGE_VOLUME_FT3
class HeatedRefrigeratedShippingContainer(RefrigeratedShippingContainer):
MIN_CELSIUS = -20.0
@RefrigeratedShippingContainer.celsius.setter
def celsius(self, value):
if value < HeatedRefrigeratedShippingContainer.MIN_CELSIUS:
raise ValueError("Temperature too cold!")
RefrigeratedShippingContainer.celsius.fset(self, value)
As you can see, there is a lot going on in that code.
I don't know how you guys learn, but I just can't sit and watch a video; I have to be actually coding along to make it stick.
That bit of code seemed simple enough, so I decided to kick it up a notch by using dataclasses and type annotations!
Hell week
Hence my journey into hell week commenced!
To make it an even better learning experience, I decided to only read the documentation on type annotations. I lost track of how many hours I spent trying to get it, not only to work, but to get mypy to NOT complain about any part of it.
After a few days of pulling out my hair I relaxed that restriction, but to my surprise, there really wasn't that much documentation on how to do what I wanted to do! Mostly every example out there dealt with type hinting normal classes, not dataclasses!
In the end I discovered that mypy doesn't support type annotations of settable property variables! Oh well, I could not get a clean mypy run in the end, but I definitely learned a lot about type hinting the rest of the code.
What I ended up with
Instead of going step by step over every bit of the code, I'm just going to present it in its entirety and let you soak it in. I'll talk about the parts that gave me the most problems and point out the parts were I learned things.
from dataclasses import InitVar, dataclass, field
from typing import Any, ClassVar, Dict, List, Optional, Tuple, Union
import iso6346
@dataclass
class ShippingContainer:
owner_code: str
length_ft: float
contents: Optional[Union[List[Any], str]]
HEIGHT_FT: ClassVar[float] = 8.5
WIDTH_FT: ClassVar[float] = 8.0
next_serial: ClassVar[int] = 1337
args: Optional[Tuple[Any]] = field(init=False, repr=False)
kwargs: Optional[Dict[str, Any]] = field(init=False, repr=False)
def __post_init__(self):
self.bic = self._make_bic_code(
owner_code=self.owner_code, serial=ShippingContainer._get_next_serial(),
)
@staticmethod
def _make_bic_code(owner_code, serial):
return iso6346.create(owner_code=owner_code, serial=str(serial).zfill(6))
@classmethod
def _get_next_serial(cls):
result = cls.next_serial
cls.next_serial += 1
return result
@classmethod
def create_empty(cls, owner_code, length_ft, *args, **kwargs):
return cls(owner_code, length_ft, contents=None, *args, **kwargs)
@classmethod
def create_with_items(cls, owner_code, length_ft, items, *args, **kwargs):
return cls(owner_code, length_ft, contents=list(items), *args, **kwargs)
def _calc_volume(self):
return ShippingContainer.HEIGHT_FT * ShippingContainer.WIDTH_FT * self.length_ft
@property
def volume_ft3(self):
return self._calc_volume()
@dataclass
class RefrigeratedShippingContainer(ShippingContainer):
celsius: float
_celsius: float = field(init=False, repr=False)
MAX_CELSIUS: ClassVar[float] = 4.0
FRIDGE_VOLUME_FT3: ClassVar[float] = 100
@staticmethod
def _make_bic_code(owner_code: str, serial: int) -> str:
return iso6346.create(
owner_code=owner_code, serial=str(serial).zfill(6), category="R",
)
@staticmethod
def _c_to_f(celsius) -> float:
return celsius * 9 / 5 + 32
@staticmethod
def _f_to_c(fahrenheit) -> float:
return (fahrenheit - 32) * 5 / 9
@property
def celsius(self) -> float:
return self._celsius
def _set_celsius(self, value: float) -> None:
try:
if value > RefrigeratedShippingContainer.MAX_CELSIUS:
raise ValueError("Temperature too hot!")
except TypeError:
raise TypeError(
"__init__() missing 1 required positional argumentL 'celsius'"
)
self._celsius = float(value)
@celsius.setter
def celsius(self, value: float) -> None:
self._set_celsius(value)
@property
def fahrenheit(self) -> float:
return RefrigeratedShippingContainer._c_to_f(self.celsius)
@fahrenheit.setter
def fahrenheit(self, value) -> None:
self.celsius = RefrigeratedShippingContainer._f_to_c(value)
def _calc_volume(self):
return super()._calc_volume() - RefrigeratedShippingContainer.FRIDGE_VOLUME_FT3
@dataclass
class HeatedRefrigeratedShippingContainer(RefrigeratedShippingContainer):
MIN_CELSIUS: ClassVar[float] = -20.0
def _set_celsius(self, value: float) -> None:
try:
if value < HeatedRefrigeratedShippingContainer.MIN_CELSIUS:
raise ValueError("Temperature too cold!")
super()._set_celsius(value)
except TypeError:
raise TypeError(
f"TypeError: __init__() missing 1 required keyword argument: {'celsius'}"
)
Learning points
Let's start at the top.
The class definitions
@dataclass
class ShippingContainer:
owner_code: str
length_ft: float
contents: Optional[Union[List[Any], str]]
HEIGHT_FT: ClassVar[float] = 8.5
WIDTH_FT: ClassVar[float] = 8.0
next_serial: ClassVar[int] = 1337
args: Optional[Tuple[Any]] = field(init=False, repr=False)
kwargs: Optional[Dict[str, Any]] = field(init=False, repr=False)
This class definition is filled with lots of good nuggets.
- Class variables must be annotated with ClassVar
- contents is not enforced because of the use of the classmethod create_empty, so it was annotated with Optional
- args and kwargs were declared because PyCharm was complaining about them not being there, even though the code will run fine without them
- args are tuples, not a list!
- kwargs are just dicts not a list of them
__post_init__
In this method we generate the custom bic code that's made up from the owner_code, the serial number, and a category code.
def __post_init__(self):
self.bic = self._make_bic_code(
owner_code=self.owner_code, serial=ShippingContainer._get_next_serial(),
)
Volume property
The @classmethods remained the same, so we're not going to discuss those. Things start to get interesting when it comes to the volume_ft3 property though.
def _calc_volume(self):
return ShippingContainer.HEIGHT_FT * ShippingContainer.WIDTH_FT * self.length_ft
@property
def volume_ft3(self):
return self._calc_volume()
As you can see, the logic for the volume calculation was extracted from the property and moved into its own method.
This will come in handy later on when this method gets overwritten in another class that inherits from this one.
Inheriting the main class
Now comes the fun parts! When you normally inherit a class, you usually call super() to initialize some of the variables. Not so with dataclasses! This is one point where I was pulling my hair out to figure out how to declare these and it turned out to be simpler than I thought. Just don't!
@dataclass
class RefrigeratedShippingContainer(ShippingContainer):
celsius: float
_celsius: float = field(init=False, repr=False)
MAX_CELSIUS: ClassVar[float] = 4.0
FRIDGE_VOLUME_FT3: ClassVar[float] = 100
The part that gave me the biggest headache was declaring the required celcius variable because celsius is also a property value!
We also don't want the user to be able to manually set the celsius value, because it has to be within a set range, so the "private" variable _celsius was created.
_celsius: float = field(init=False, repr=False)
Pay attention to the usage of field. This tells the class constructor not only to not expect a value to be passed during initialization, but also to not display this variable through repr!
Property init values
Here is how you handle init values that are also properties of the class. A more detailed explanation can be found in this wonderful write-up: Reconciling Dataclasses and Properties in Python
@property
def celsius(self) -> float:
return self._celsius
def _set_celsius(self, value: float) -> None:
try:
if value > RefrigeratedShippingContainer.MAX_CELSIUS:
raise ValueError("Temperature too hot!")
except TypeError:
raise TypeError(
"__init__() missing 1 required positional argumentL 'celsius'"
)
self._celsius = float(value)
As with the property value in the main class, the logic was also extracted from the property and moved into its own method. I also had to add a try/except clause to it because for some reason, if the celcius value was not passed during initialization, it would try and initialize it with a None value but would give some obscure error message...
Overwriting methods
Next we'll skip to overwriting the _calc_volume method.
def _calc_volume(self):
return super()._calc_volume() - RefrigeratedShippingContainer.FRIDGE_VOLUME_FT3
Notice how we call the parent class with super() but that we also use the class name while retrieving the class variable FRIDGE_VOLUME_FT3!
Conclusion
There's a lot going on here, and it might take you a while to grok it all, but keep at it and I can assure you that the next time that you need to annotate some dataclasses, this information will come in really handy!