Opening a file is easy. Deciding exactly when it stops being usable is the engineering problem. The same is true for locks, transactions, temporary directories, tracing spans, decimal settings, and test patches. Each has a region where some condition holds and an obligation when that region ends.
A context manager packages both sides of that boundary. A with statement does not merely save a call to close(). It connects acquisition and cleanup through a protocol that covers normal completion, early return, loop exits, and exceptions.
Version note. The context management protocol is a Python language guarantee. Examples target Python 3.10 through 3.14 and were run on CPython 3.14. Details such as file object classes, reference counting, and bytecode are CPython implementation details and are not cleanup contracts.
Experiment 1: cleanup survives every exit
class Marker:
def __enter__(self):
print("[event] enter context")
return "resource"
def __exit__(self, exc_type, exc, traceback):
print("[event] exit context:", exc_type.__name__ if exc_type else "normally")
return False
def work(stop_early):
with Marker() as value:
print("[state] acquired value:", value)
if stop_early:
return "stopped"
return "finished"
print("[result] early-return work:", work(True))
print("[result] normal work:", work(False))
Both calls execute __exit__. Python evaluates the expression after with, calls its __enter__, binds that result after as, and runs the suite. Once entry succeeds, Python calls __exit__ when the suite leaves. The reason can be ordinary fall-through, return, break, continue, or an exception.
This is the central guarantee. It is stronger than hoping an object is destroyed soon. CPython commonly destroys an unreferenced object immediately because it uses reference counting, but cycles, tracebacks, alternate implementations, and lingering references make finalization timing unsuitable for correctness. Explicit context boundaries describe lexical ownership: this block owns the resource for this interval.
There is one important edge. If __enter__ itself raises, that manager's __exit__ is not called because entry never completed. Acquisition involving several steps must undo earlier steps if a later step fails, or delegate those steps to already-safe managers.
Experiment 2: as receives the entry result
class Session:
def __init__(self, name):
self.name = name
self.connected = False
def __enter__(self):
self.connected = True
return {"session": self.name, "send": self.send}
def send(self, message):
assert self.connected
return f"{self.name}: {message}"
def __exit__(self, exc_type, exc, traceback):
self.connected = False
return False
manager = Session("primary")
with manager as api:
print("[result] session send:", api["send"]("ready"))
print("[check] session connected after exit:", manager.connected)
The bound value need not be the manager. Files return themselves, but a manager may return a transaction, cursor, facade, token, or nothing. Separate these roles when callers should use a narrow capability while the manager retains lifecycle machinery.
Do not read with expression as target as assigning expression to target. The expression produces the context manager; __enter__() produces the target value. That distinction explains APIs such as decimal.localcontext(), where the manager establishes temporary state and returns a context object that can be adjusted inside the block.
An object's useful lifetime should agree with its boundary. If callers store api and invoke it afterward, this example correctly fails its assertion. Production APIs should raise an informative exception, but the deeper fix is documentation and design that make escaped, invalid handles unlikely.
Experiment 3: exceptions are presented, not hidden
class ReportFailure:
def __enter__(self):
return self
def __exit__(self, exc_type, exc, traceback):
if exc is not None:
print(f"[event] observed error: {exc_type.__name__}: {exc}")
return False
try:
with ReportFailure():
raise ValueError("bad row")
except ValueError as error:
print(f"[error] caller received: {error}")
On exceptional exit, Python passes the exception type, exception instance, and traceback to __exit__. Returning a false value, including None, propagates the original exception. Returning a true value suppresses it and continues after the with statement.
Suppression is therefore part of an API's error semantics, not a cleanup convenience. A database transaction may roll back and still propagate. A tracing span may record failure and still propagate. Most cleanup managers should return False or None.
Suppress only a deliberately narrow, documented condition. contextlib.suppress(FileNotFoundError) can express idempotent removal, where absence already satisfies the desired state. A manager that returns True for every exception can hide KeyboardInterrupt, memory failures, programming mistakes, and corrupted state. If __exit__ raises a new exception, that new failure replaces the active one, with exception chaining preserving context. Keep cleanup paths small and dependable.
Experiment 4: generator managers put cleanup beside setup
from contextlib import contextmanager
@contextmanager
def temporary_setting(settings, key, value):
missing = object()
previous = settings.get(key, missing)
settings[key] = value
try:
yield settings
finally:
if previous is missing:
del settings[key]
else:
settings[key] = previous
config = {"mode": "safe"}
with temporary_setting(config, "mode", "fast"):
print("[state] temporary config:", config)
print("[state] restored config:", config)
@contextmanager adapts a generator function to the same protocol. Code before yield performs entry, the yielded object becomes the as value, and code after yield performs exit. The function must yield exactly once. The try/finally is essential: an exception from the with suite is thrown into the generator at yield, so code after a bare yield could be skipped.
Use this form when setup and teardown are short and share local state. Use a class when the manager has several operations, substantial state, reusable methods, or when explicit protocol methods improve diagnosis. Neither is more fundamentally Pythonic; both implement the same semantics through standard machinery.
Be careful catching exceptions around yield. If the generator receives an exception and then completes without re-raising it, the generated manager treats that exception as handled. Catch only errors you truly mean to translate or suppress. A finally block avoids that accidental policy.
Experiment 5: compose a dynamic number of resources
from contextlib import ExitStack
from io import StringIO
closed = []
def note_close(name):
closed.append(name)
with ExitStack() as stack:
streams = [stack.enter_context(StringIO(text)) for text in ("alpha", "beta")]
stack.callback(note_close, "callbacks run too")
print("[result] stream contents:", [stream.read() for stream in streams])
print("[event] callbacks invoked:", closed)
print("[check] streams closed:", [stream.closed for stream in streams])
Nested with statements work when the resource count is fixed. ExitStack handles resources discovered at runtime and callbacks that do not implement the protocol. Registered exits run in last-in, first-out order, matching nested statements. If the fourth acquisition fails, the first three already-entered resources are released.
This makes ExitStack an acquisition transaction. Build the stack, and call pop_all() only when ownership should transfer elsewhere. Without that explicit transfer, every registered obligation remains tied to the block.
Avoid turning ExitStack into an unstructured cleanup drawer. Registration order affects correctness, callbacks receive no automatic exception arguments, and a callback can fail. Prefer native managers and use the stack at orchestration boundaries where dynamic composition is the actual problem.
Experiment 6: async lifetime is a separate protocol
import asyncio
class AsyncConnection:
async def __aenter__(self):
await asyncio.sleep(0)
print("[event] async connection opened")
return self
async def request(self):
await asyncio.sleep(0)
return "response"
async def __aexit__(self, exc_type, exc, traceback):
await asyncio.sleep(0)
print("[event] async connection closed")
return False
async def main():
async with AsyncConnection() as connection:
print("[result] async response:", await connection.request())
asyncio.run(main())
async with uses __aenter__ and __aexit__, awaiting both. This is a distinct language protocol for setup or teardown that must suspend, such as acquiring a pool connection or closing an async stream. A synchronous with cannot await cleanup; an async manager cannot be used by ordinary with merely because the method names look similar.
Cancellation also reaches __aexit__. Cleanup should preserve cancellation semantics and avoid unbounded waits. Libraries may shield a small critical cleanup operation, but broadly swallowing cancellation makes shutdown unreliable. contextlib.asynccontextmanager and AsyncExitStack provide async counterparts to the generator and composition tools.
Async context-manager syntax was introduced in Python 3.5. contextlib.chdir, by contrast, arrived in 3.11 and illustrates a manager with process-wide side effects: changing the current directory is not suitable around a yield or await where unrelated work may run. Always separate protocol safety from resource semantics. A manager guarantees its exit hook runs; it cannot make global mutable state local.
Design resource ownership, not punctuation
Use with when a caller can state where ownership begins and ends. Open files close at the block boundary. Locks release there. Transactions commit or roll back there. Temporary mutations are restored there. This visible lifetime is more valuable than shorter syntax.
Do not return an iterator that depends on a resource opened inside a completed with block. Either consume the iterator inside the block, have a generator own the block for the duration of iteration, or let the caller provide the open resource. Lazy values and lexical lifetime must agree.
Do not routinely combine acquisition and broad exception recovery. Cleanup belongs close to the resource; recovery often belongs at a service boundary that understands retries, user messages, and observability. A context manager can annotate an exception without deciding the whole application's response.
Thread and task safety are properties of the managed resource, not of with. Entering the same manager concurrently is safe only if its contract says so. Reentrancy is likewise explicit: reusable managers can support multiple separate uses, while reentrant managers support nesting the same instance. Many generator-based managers are one-shot.
Transactions need a declared outcome
Managers around transactions must define what normal exit means. A common policy commits when the suite completes and rolls back when it raises. That is convenient, but it makes catching an exception inside the block significant: once caught, the manager sees normal completion and may commit later changes. If every failed statement poisons the transaction, the transaction object must track that state rather than relying only on exc_type.
Nested transaction managers need equally precise semantics. The inner boundary may create a savepoint, join the outer transaction, or reject nesting. Calling all three behaviors "a transaction context" does not make them interchangeable. Document where commit becomes durable and which layer owns retry.
Cleanup ordering is part of correctness. Release dependent resources before their dependencies: close a cursor before its connection, finish a compressed stream before its destination file, and end a tracing child before its parent. Lexically nested with statements and ExitStack both provide last-in, first-out exit for exactly this reason.
Observability should not destabilize release. If recording a cleanup metric can fail, isolate that failure according to an explicit policy so it does not replace the application exception or skip essential teardown. Logging an exception in __exit__ also does not mark it handled; propagation still depends only on the return value or a new raised exception.
Exercises: make the boundary explicit
- Write a timer manager that reports elapsed time but never suppresses an exception. Test normal and exceptional exits.
- Build a manager that temporarily adds an environment variable and restores absence separately from an empty previous value.
- Use
ExitStackto open a runtime-sized list ofStringIOobjects, then force one acquisition to fail and verify earlier resources close. - Change
ReportFailure.__exit__to suppress onlyValueError. Explain why checkingexc_type is ValueErrordiffers fromissubclass(exc_type, ValueError). - Implement the same temporary-setting manager as a class and with
@contextmanager. Compare state visibility and misuse diagnostics. - Create an async manager whose exit awaits a completion event. Cancel its body and record which exception reaches the caller.
Keep this model
A context manager defines a region in which a condition is established and a cleanup obligation is owned. __enter__ establishes and optionally returns a capability. __exit__ receives the outcome, performs cleanup, and decides only through its return value whether an exception continues.
That protocol is a Python guarantee. Immediate destruction, particular file types, and generated bytecode are implementation details. Version-added helpers are conveniences layered on the stable model. Design around the model: pair acquisition with release, keep suppression narrow, compose dynamic ownership deliberately, and use the async protocol when cleanup must await.