Using Python's contextlib for custom context managers

Python's with statement turns resource management into a predictable scope. The object is acquired before the block begins, made available inside it, and cleaned up when the block ends—even if the block raises an exception. This pattern keeps setup beside the work that needs the resource and removes the scattered try/finally blocks that otherwise obscure control flow.

The contextlib module in the standard library supplies building blocks for this lifecycle. Its contextmanager decorator lets a normal generator function act as a custom context manager, while ExitStack coordinates several resources. You can therefore express ownership and teardown without inheriting from a framework class or adding a third-party dependency.

That matters beyond tidy code. Sydney and Melbourne's fintech companies and data-analytics teams use Python for services, reporting and automation, where an abandoned database connection or temporary file can affect customers, memory usage and operational jobs. Canberra's government technology teams face similar concerns while moving data between constrained public-sector systems.

Resource cleanup also has an everyday parallel: many cafés in Melbourne and Sydney welcome customers bringing reusable cups, and returning the cup completes the visit. Software has a similar obligation. Under the Privacy Act 1988, covered organisations must handle personal information according to the Australian Privacy Principles; the Notifiable Data Breaches scheme adds duties for eligible breaches. Closing a handle promptly does not establish compliance, but it avoids avoidable exposure.

The contract behind Python's with statement

A custom manager follows a small protocol. __enter__ performs setup and returns the object used in the block; __exit__ receives information about any exception and performs teardown. For a class-based manager, the two methods must work together: setup failures occur before the body runs, while teardown still runs after the body succeeds or fails.

The exception arguments passed to __exit__ are the exception's type, value and traceback. Returning True from that method suppresses the exception, whereas returning False or None lets it continue. Most resource wrappers should propagate failures, because hiding an error can leave data incomplete or a resource in an uncertain state.

The same ownership model appears in core algorithm explanations: a correct routine leaves state in the condition its caller expects. A resource manager follows a similar contract, establishing a usable state on entry and restoring a safe state on exit.

Writing a generator-based manager

The most compact custom manager uses @contextmanager. Decorate a generator function, accept whatever configuration it needs, and place yield at the point where the resource becomes available. Code before yield performs acquisition; the try/finally around it guarantees release. The with target receives the yielded value.

import sqlite3
from collections.abc import Iterator
from contextlib import contextmanager


@contextmanager
def transaction(connection: sqlite3.Connection) -> Iterator[sqlite3.Connection]:
    try:
        yield connection
        connection.commit()
    except Exception:
        connection.rollback()
        raise
    finally:
        connection.close()

This wrapper assumes ownership of the supplied connection, so the caller does not close it separately. The decorated function returns a single-use manager object, so create a new instance for each with block. Entering the same instance twice is a common source of confusing errors.

An exception raised inside the with body is thrown back into the generator at its yield. The wrapper can then commit successful work, roll back failed work or translate the error before re-raising it. A bare raise preserves the original traceback, while a generator that yields more than once cannot satisfy the context-manager protocol.

Making failures visible and cleanup dependable

An exception raised inside the with body is passed back into a generator-based manager at its yield. If the generator does not catch it, the exception continues outwards unchanged. Catching Exception, handling a meaningful condition and then using a bare raise preserves both the cause and the traceback for debugging.

Acquisition and release are separate responsibilities. Put only the steps needed to obtain the resource before yield, then guard everything afterwards with try/finally. Once acquisition succeeds, cleanup must have a path even when the body raises, returns early or is interrupted by KeyboardInterrupt.

Teardown errors need careful treatment too. An exception raised while closing a resource can replace the active exception, although Python keeps the earlier failure as context. Keep cleanup operations small, make them idempotent where possible and avoid broad except: pass blocks that erase both the original problem and the cleanup problem.

When a reusable class makes more sense

A class-based manager is preferable when the resource has meaningful state, several operations or a lifecycle worth exposing. Implement __enter__ to prepare the object and __exit__ to clean it up. The class can also provide other methods without hiding them behind a generator.

import time


class Stopwatch:
    def __enter__(self):
        self.started_at = time.perf_counter()
        return self

    def __exit__(self, exc_type, exc, traceback):
        self.elapsed = time.perf_counter() - self.started_at
        return False

The stopwatch is reusable sequentially: each entry records a fresh start time, and each exit stores the elapsed duration. Returning False from __exit__ keeps exceptions visible. Use a separate instance for concurrent measurements, since one object's fields would otherwise be overwritten.

The extra syntax buys explicitness rather than magic. A class can validate configuration in __init__, preserve diagnostic attributes and integrate with existing object-oriented code. It also avoids the single-use limitation of a decorated generator manager, although separate instances remain the clearest choice for independent operations.

Combining several resources with ExitStack

Several resources often belong to one operation: a report may read a source file, write a destination and record temporary state. ExitStack manages these as a group, entering each manager and registering its exit before moving to the next. It replaces deeply nested with statements when the number of resources is dynamic.

from contextlib import ExitStack

with ExitStack() as stack:
    source = stack.enter_context(open("sales.csv", encoding="utf-8"))
    target = stack.enter_context(open("processed.txt", "w", encoding="utf-8"))

    for line in source:
        target.write(line)

enter_context accepts any context manager, not just file objects. Resources leave the stack in reverse order, matching nested with semantics, and a resource already entered is cleaned up if a later acquisition fails. stack.callback can also register a plain callable for teardown when no context-manager protocol is needed.

For a fixed pair of resources, nested with blocks remain the simplest choice because their ownership is immediately visible. ExitStack earns its complexity when resources are conditional, numerous or assembled by a helper. It is part of the standard library, so this composition adds no package dependency.

Selecting the right contextlib helper

contextlib already provides several useful managers. closing calls close() on objects that offer it but does not decide whether work should commit. nullcontext supplies a no-op manager for optional resources, and suppress catches only the listed exception types inside its block. redirect_stdout temporarily changes standard output, which is convenient for captured diagnostics but affects shared process state.

For asynchronous code, aclosing awaits an object's aclose() method, while @asynccontextmanager adapts an asynchronous generator. A synchronous manager cannot simply be used with async with; the acquisition and release operations must support asynchronous cleanup. Keeping these variants separate prevents accidental blocking or un-awaited teardown.

Large data pipelines make the same lifecycle equally important. A pandas pipeline review illustrates how readers, writers and transformations need explicit closing points. This discipline is especially valuable in Australian workflows covered by the Privacy Act 1988, where temporary personal information, open handles and idle connections all need finite, deliberate lifetimes.

Recommendations for dependable context managers

Choose the simplest implementation that expresses the required behaviour. A generator works well for a single acquisition-and-release cycle, while a class suits richer state and explicit lifecycle methods. Standard helpers should handle common cases before a bespoke manager is introduced.

Design and test the failure paths as carefully as the successful path. Cleanup is most valuable when the body fails, and the manager should preserve useful diagnostics rather than replace them with vague errors.

• Keep acquisition small and place release in finally, so every successful setup has a matching cleanup path.
• Yield exactly once in a generator-based manager and create a fresh manager instance for each with block.
• Avoid suppressing exceptions unless suppression is an explicit part of the operation, and preserve the original traceback when translating errors.
• Close resources in reverse acquisition order, using ExitStack when the number of resources varies.
• Test normal completion, exceptions inside the block, cleanup failures and repeated use with separate instances.

These choices make ownership visible and keep failure behaviour predictable. A dependable custom context manager states what is acquired, what happens during use and which teardown runs before the caller continues.