Understanding the Python data model: dunder methods and protocols
Python's data model is the quiet machinery that makes objects behave in predictable, composable ways. When you call len(my_list), iterate over a custom container, or open a file with a with statement, you are using protocols defined by the language itself. Understanding these protocols unlocks the ability to write code that integrates seamlessly with the standard library and with code written by other developers across Sydney, Melbourne, and the rest of the Australian tech scene.
Dunder methods, sometimes called magic methods, are how a class opts into those protocols. Their names begin and end with double underscores, and they are not meant to be called directly. Instead, the interpreter calls them in response to built-in operations. Mastering them means writing classes that feel native to Python rather than foreign bodies grafted onto it.
The object lifecycle: construction and destruction
Every Python object passes through three lifecycle stages: creation, initialisation, and eventual garbage collection. The data model exposes hooks for each. __new__ is the constructor that allocates the object — it is a classmethod called before __init__, and it is rarely overridden except for subclasses of immutable types like str, int, or tuple. __init__ is the initialiser that most Python developers are familiar with; it receives the already-allocated instance and sets its attributes.
A subtle point that surprises newcomers is that __init__ cannot return a value. If you need to influence what MyClass(...) actually produces, you must override __new__ instead. For most everyday classes in Australia-based engineering teams — whether they are building fintech APIs in Barangaroo or logistics dashboards in Docklands — __init__ is sufficient.
The third hook, __del__, fires when the garbage collector is about to reclaim an object. Its timing is not guaranteed, so it should not be relied upon for critical cleanup such as closing database connections or flushing network buffers. Use a context manager for that, which the data model provides through __enter__ and __exit__.
String representation: showing objects to humans and machines
Two dunder methods control how an object appears when printed, logged, or inspected. __repr__ should return an unambiguous string that, where possible, looks like valid Python code to recreate the object. This is what the REPL displays and what debugging tools show. __str__ returns a human-readable summary intended for end users, and it falls back to __repr__ if not defined.
The rule of thumb, popularised by Alex Martelli and often quoted in meetups from Brisbane to Perth, is: implement __repr__ for yourself and other developers, implement __str__ for your users. A datetime object, for instance, has __repr__ that yields datetime.datetime(2024, 5, 12, 9, 0) and __str__ that produces something like 2024-05-12 09:00:00.
There is also __format__, which integrates with format specifications and f-strings. Overriding __format__ lets you write f"{obj:short}" or f"{obj:csv}" and have your class respond intelligently. Before diving into the most-used protocols, here is a compact view of what each one enables:
| Protocol | Methods to implement | Built-in operations enabled |
|---|---|---|
| Sequence | __len__, __getitem__ |
len(), indexing, slicing, iteration, in |
| Iterable | __iter__ |
for, iter(), list/set comprehensions |
| Container | __contains__ |
in, not in |
| Numeric | __add__, __sub__, __mul__ |
+, -, * operators |
| Comparison | __eq__, __lt__, __hash__ |
==, <, >, use as dict keys, set membership |
| Context | __enter__, __exit__ |
with statement |
Iteration, containment, and the sequence protocol
Containers in Python are defined by behaviour, not by inheritance. The sequence protocol requires __len__ and __getitem__, and once a class provides those, it gains indexing, slicing, iteration through iter(), and membership testing through in. This is an elegant application of duck typing that the language enforces through protocols rather than explicit interfaces.
For full iteration control, implement __iter__ to return an iterator object, and __next__ on that iterator to yield successive values. The membership operator x in container calls __contains__ if defined, otherwise falls back to a linear scan using __getitem__. A practical example: a class representing a roster of AFL players at a Melbourne-based sports analytics startup might implement __len__ to return squad size, __getitem__ for index access, and __iter__ to yield players in jersey order. Such an object then works seamlessly with for player in squad, len(squad), and sorted(squad). For a worked example of how iteration protocols feed into a familiar algorithm, the walkthrough of KNN shows a clean implementation that iterates over training data.
Numeric operations and comparison
Operator overloading in Python is governed entirely by dunder methods. To make + work on a custom class, define __add__. For in-place addition (+=), define __iadd__. Reflected operations, where the right operand handles the call, use __radd__ and friends. This is how the integer 5 knows what to do when added to a numpy.ndarray — numpy defines __radd__ on the array, and the interpreter dispatches correctly.
Comparison methods deserve special attention because they interact with hashing. If you override __eq__, Python sets __hash__ to None, which makes your object unhashable and therefore unusable as a dictionary key or set member. To preserve hashability, explicitly define __hash__ alongside __eq__. A common idiom is to compute the hash from immutable fields and compare them in __eq__. For example, an Invoice class at a Perth-based accounting SaaS might hash on (vendor_id, invoice_number) to ensure idempotency. Rich comparison methods (__lt__, __le__, __gt__, __ge__) can be wired together with functools.total_ordering, which fills in the remaining methods from one defined comparison and __eq__. For assignment-style problems where comparison operators decide the result, the Hungarian algorithm review demonstrates how __lt__ and __eq__ keep the data model tidy.
Context managers: the with statement protocol
The with statement is one of Python's most reliable features, and it is powered by __enter__ and __exit__. When a with block begins, __enter__ is called and its return value is bound to the as target. When the block exits — whether normally or via an exception — __exit__ is called with the exception type, value, and traceback. Returning a truthy value from __exit__ suppresses the exception.
This protocol matters for resource management. Opening a file, acquiring a database connection from a Sydney-region Postgres cluster, or locking a thread should always happen inside a context manager to guarantee cleanup. For classes that need only lightweight context behaviour, contextlib.contextmanager provides a decorator that turns a generator function into a context manager, avoiding the need to write a full class. For more complex cases, the class-based approach is more explicit.
The Australian Privacy Principles and the Notifiable Data Breaches scheme under the Privacy Act 1988 mean that local developers often need guaranteed cleanup of file handles that may contain personal data, and context managers are the standard way to enforce that discipline across a codebase.
Practical patterns and where protocols shine
A useful mental model: every dunder method you implement is a vote of participation in a protocol. Implement __len__ and you join the size protocol. Implement __iter__ and you join the iteration protocol. Implement __getitem__ alone and Python will accept your class as iterable by synthesising iteration on demand. This composition is what makes the data model feel like Lego: small, interlocking pieces.
Practical habits worth adopting:
- Define
__repr__on every public class — even if it is justf"{self.__class__.__name__}({self.x}, {self.y})". - Use
__slots__to lock down attributes when memory matters; this is relevant when running batch processing on commodity VMs in regional Australian data centres where RAM is at a premium. - Prefer
dataclasses.dataclasswitheq=True, frozen=Truefor value-like objects; it generates__init__,__repr__, and__eq__for you. - Treat
__del__as a last resort; prefer explicit close methods or context managers.
Other protocol hooks worth knowing:
__call__lets instances be invoked like functions, useful for callable configuration objects.__getattr__and__getattribute__intercept attribute access, enabling proxies and ORMs.__set_name__, available since PEP 487, lets descriptors know the attribute name they are bound to.__class_getitem__enables subscription on class objects, poweringList[int]-style syntax in generic code.
Reading and practising these patterns together builds real fluency. The problem-solving collection has graded exercises that reward idiomatic Python over hand-rolled ceremony, and they make a natural next step after working through the protocols above.