Calling something a container sounds as though Python has one container interface. It does not. A list supports iteration, membership, length, integer indexing, slicing, reversed traversal, equality, concatenation, and mutation. A set chooses a different bundle. A dictionary iterates keys but indexes values by keys. A generator supports iteration while refusing nearly everything collection-shaped.
That decomposition is useful when designing domain collections. Instead of imitating list, choose the operations whose semantics you can honor. Syntax such as in, [], len(), and for dispatches through separate data-model protocols.
Version note. Special-method lookup and the documented fallback rules are Python guarantees. The examples run on Python 3.10 through 3.14 and were verified on CPython 3.14. Built-in storage layouts, operation timings, and slot dispatch machinery are CPython details.
Experiment 1: one object, independent operations
class Batch:
def __init__(self, items):
self._items = tuple(items)
def __iter__(self):
print("[event] __iter__ called")
return iter(self._items)
def __len__(self):
print("[event] __len__ called")
return len(self._items)
def __contains__(self, item):
print("[event] __contains__ called")
return item in self._items
batch = Batch(["queued", "running"])
print("[result] list(batch):", list(batch))
print("[result] len(batch):", len(batch))
print("[result] contains 'running':", "running" in batch)
Each expression asks a different question. iter(batch) invokes __iter__; len(batch) invokes __len__; membership prefers __contains__. Implementing one does not conceptually require the others, although Python has compatibility fallbacks.
__len__ must return a non-negative integer that fits the platform's Py_ssize_t; invalid results raise. Length should describe the current number of items, not capacity, bytes, or an expensive estimate. Python also uses length for truth testing when __bool__ is absent: zero is false, nonzero true. If emptiness is expensive or unknowable, omitting length may be more honest than making if collection perform hidden I/O.
Protocol methods are semantic promises. Adding __len__ because a remote service can execute COUNT(*) makes innocent truth checks potentially expensive. Adding iteration to a query object may trigger network requests. Python permits those designs, but their costs should be explicit in names and documentation.
Experiment 2: membership has a fallback ladder
class LoggedValues:
def __init__(self, values):
self.values = values
def __iter__(self):
print("[event] membership requested iteration")
return iter(self.values)
values = LoggedValues([2, 4, 6])
print("[check] contains 4:", 4 in values)
print("[check] contains 5:", 5 in values)
Without __contains__, in tries iteration. It compares each yielded value for equality and stops at the first match. If iteration is unavailable, legacy sequence indexing from zero can provide another fallback until IndexError.
That default is correct for many small collections but can be algorithmically misleading. A set-like object backed by a hash index should implement __contains__ so membership expresses its direct operation. A range-like object can test mathematically without iterating. Conversely, a streaming source may be iterable but should make users confront that membership consumes values and might never terminate.
Membership must answer whether the object contains the candidate according to its advertised element semantics. Dictionaries contain keys, not values. Strings search substrings rather than individual-character equality. Those are intentional contracts, not universal behavior derived from storage.
If __contains__ returns a non-Boolean object, Python truth-tests it. Return an actual bool unless a compelling interoperability convention says otherwise. Raising TypeError for unsupported candidate kinds can be appropriate, but returning False is often friendlier when the types simply cannot match.
Experiment 3: indexing receives keys and slice objects
class Window:
def __init__(self, values):
self._values = tuple(values)
def __len__(self):
return len(self._values)
def __getitem__(self, key):
if isinstance(key, slice):
start, stop, step = key.indices(len(self))
return type(self)(self._values[start:stop:step])
return self._values[key]
def __repr__(self):
return f"Window({self._values!r})"
window = Window(range(6))
print("[result] last item:", window[-1])
print("[result] items at slice 1:5:2:", window[1:5:2])
print("[result] reversed window:", window[::-1])
Subscription calls __getitem__ with whatever appears between brackets. An integer is not guaranteed; slices arrive as slice objects, mappings receive arbitrary hashable keys, and user code can pass tuples through comma-separated subscriptions. Supporting [] therefore requires deciding a key language.
For sequence-like slicing, slice.indices(length) normalizes omitted and negative bounds for a particular length. Delegating to an internal tuple is even simpler when the return type can be a tuple. Here slices preserve the domain type while scalar indexes return elements. That is one reasonable policy, not a language requirement.
Integer sequences should support negative indexes only if they claim ordinary sequence expectations. Raise IndexError for an out-of-range positional index; iteration's old __getitem__ fallback depends on it. A KeyError communicates missing mapping keys. Exception type is observable API behavior used by callers and standard machinery.
Experiment 4: iteration can exist without __iter__
class Squares:
def __init__(self, count):
self.count = count
def __getitem__(self, index):
if index < 0 or index >= self.count:
raise IndexError(index)
return index * index
squares = Squares(4)
print("[result] squares from sequence fallback:", list(squares))
print("[check] contains 9:", 9 in squares)
Python's legacy sequence protocol lets iter() request indexes 0, 1, and onward until IndexError. This is guaranteed fallback behavior, but new classes should usually implement __iter__ explicitly. It states intent, can avoid repeated indexing, and can choose an iterator independent of random access.
The fallback exposes a dangerous bug: if __getitem__ never raises IndexError, iteration is infinite. Returning None for missing positions does not stop it. A class intended only as a mapping can also accidentally iterate numeric keys if it accepts them. Explicit protocols prevent unrelated capabilities from emerging by accident.
collections.abc.Iterable runtime checks look for __iter__; they do not discover an object that works only through sequence fallback. This is a useful reminder that successful operation and ABC recognition are related but not identical. Prefer trying iter(obj) when the immediate operation matters, and use an ABC when its declared contract matters.
Experiment 5: mutation needs stricter invariants
class UniqueList:
def __init__(self, values=()):
self._values = []
for value in values:
self.append(value)
def __len__(self):
return len(self._values)
def __getitem__(self, index):
return self._values[index]
def __setitem__(self, index, value):
candidate = self._values.copy()
candidate[index] = value
if len(candidate) != len(set(candidate)):
raise ValueError("values must remain unique")
self._values[index] = value
def append(self, value):
if value in self._values:
raise ValueError("values must remain unique")
self._values.append(value)
values = UniqueList([1, 2, 3])
values[1] = 4
print("[state] values after valid replacement:", list(values))
try:
values[1] = 3
except ValueError as error:
print("[error] duplicate replacement rejected:", error)
Mutation is not just __setitem__. A list-like surface includes deletion, append, insert, extension, in-place addition, and slice assignment. Every route must preserve domain invariants. Subclassing list and overriding only append does not intercept all changes; built-in methods can bypass the policy you imagined.
Composition narrows the surface. Expose only operations you can define consistently. The example relies on __getitem__ fallback for iteration, but production code could add explicit __iter__. Slice assignment also reaches __setitem__; the copied candidate lets validation happen before mutation and handles scalar and slice keys uniformly.
Mutable containers must decide whether iterators observe later changes, reject structural mutation, or traverse snapshots. Built-ins differ. Python does not impose one global concurrent-mutation policy. Document it, especially when callbacks or async code can mutate between steps.
Experiment 6: ABC mixins create methods from primitives
from collections.abc import Sequence
class Cards(Sequence):
def __init__(self, cards):
self._cards = tuple(cards)
def __len__(self):
return len(self._cards)
def __getitem__(self, index):
result = self._cards[index]
return type(self)(result) if isinstance(index, slice) else result
cards = Cards(["A", "K", "Q", "J"])
print("[result] reversed cards:", list(reversed(cards)))
print("[result] index of Q:", cards.index("Q"))
print("[result] count of A:", cards.count("A"))
collections.abc.Sequence declares a recognizable contract and supplies mixins such as index, count, iteration, and reversed traversal from __len__ and __getitem__. This can remove boilerplate, but generated operations inherit the complexity of those primitives. If __getitem__ is expensive, a mixin that repeatedly indexes may be unexpectedly expensive. Override with a better implementation where it matters.
Registering a class as a virtual subclass makes isinstance recognize it without adding methods. Registration is an assertion, not an adapter. If the object does not really satisfy the contract, downstream failures become harder to understand.
ABCs and protocols serve different audiences. ABCs combine runtime recognition, inheritance, and optional mixins. Static typing.Protocol describes structural requirements to type checkers and, optionally, limited runtime checks. Neither replaces behavioral documentation: a Sequence should have stable positional ordering and coherent length, not merely methods with matching names.
Equality, hashing, and representation
Container equality is another independent design choice. Lists compare element-by-element and order matters. Sets compare members without order. Mappings compare key-value pairs. A domain collection should choose semantics from the domain rather than delegating blindly to internal storage.
Returning NotImplemented from __eq__ for unsupported types allows reflected comparison and the language's fallback. Returning False immediately can prevent another type from recognizing a meaningful cross-type comparison. If mutable contents participate in equality, instances generally should not be hashable. Stable hashes and mutable value equality do not mix.
__repr__ should aid diagnosis without consuming one-shot inputs or issuing remote calls. A container representation that enumerates millions of elements can turn logging into an outage. Truncate deliberately and indicate omission.
Choose the smallest honest bundle
Start from caller questions. Do callers need traversal, exact size, positional lookup, key lookup, membership, reversal, or mutation? Add each independently. Do not add random access because an internal list happens to provide it; changing storage later would then break a public promise.
Preserve conventional semantics where you adopt conventional syntax. Positional misses raise IndexError; key misses raise KeyError; slices handle omitted and negative bounds; membership returns a Boolean answer; iteration eventually raises StopIteration. Surprising costs still need names or documentation even when behavior is correct.
Separate views from snapshots. A view reflects later mutation and may reject mutation during iteration. A tuple or list copy fixes values at a moment and costs memory. Returning an iterator promises one-shot traversal. These are different ownership and consistency choices, not interchangeable optimizations.
Complexity belongs in the interface
Familiar syntax creates familiar complexity expectations even though Python does not enforce them. Users generally expect sequence length and indexing to be cheap, set membership to be cheap on average, and iteration to perform work proportional to values produced. A database-backed object can technically implement all of these while issuing one query per operation, but it turns harmless-looking expressions into distributed-system boundaries.
Prefer verbs such as fetch_page, count_remote, or contains_key when latency, failure, authorization, or billing deserves visibility. Reserve container syntax for operations that behave enough like local collection operations to support ordinary composition. This is an API judgment rather than a language restriction.
Concurrency adds a second complexity dimension. if key in mapping: return mapping[key] performs two protocol operations and is not an atomic claim that the key remains present. Built-in dictionaries under ordinary CPython execution do not turn compound application logic into a transaction, and alternate implementations need not share incidental atomicity. Provide an operation such as get_or_create under the appropriate lock when the domain requires one decision.
Finally, distinguish ordering from sorting. Iteration order may be insertion order, priority order, storage order, or deliberately unspecified. If clients need sorted traversal, expose that guarantee or require an explicit sorted(...) at the presentation boundary. Depending on an observed order that the type does not promise creates tests tied to implementation accidents.
Exercises: design the surface
- Create a read-only paged collection. Decide whether
len, truth testing, and negative indexing justify remote requests. - Add
__contains__toSquaresusing arithmetic rather than iteration. Handle negative and non-integer candidates. - Implement a mapping whose iteration yields keys, then demonstrate membership,
keys, and value lookup with conventional exceptions. - Extend
Windowwith__reversed__and compare it with the sequence fallback based on length and indexing. - Find every mutation path in
MutableSequence. Implement the required primitives for a bounded-capacity collection. - Make two domain collections compare equal only when their semantic kinds and elements match. Return
NotImplementedfor other types.
Keep this model
A container is not one protocol. It is a deliberate bundle of capabilities selected by special methods and, sometimes, an ABC. Python provides fallback connections among those capabilities, but fallback convenience can hide costs and accidental behavior.
Choose the smallest honest surface, preserve the conventions of syntax you adopt, and treat complexity, consistency, and mutation policy as part of the contract. CPython may optimize built-ins through internal slots and specialized storage. User code should depend on documented behavior, not those representations.