Tests run in one interpreter by default. Imports, registries, caches, logging handlers, environment snapshots, and monkey patches can therefore survive from one test into another. A test that passes alone but fails in the suite often has an ownership problem disguised as an ordering problem.
The import system has more than one cache. sys.modules stores loaded module objects. Finders may cache directory or archive information. Modules themselves may cache values in globals. Deleting one layer does not necessarily reset the others, and importlib.reload() does not create a pristine module.
These experiments use unique temporary module names and always restore interpreter state. That discipline is not decoration: experiments about global state can contaminate the very process verifying them.
Version note. Blocks were run on CPython 3.14.7. The import protocol and
sys.modulesbehavior are Python guarantees documented by the language andimportlib. Dictionary identity, extension-module reinitialization, finder internals, and bytecode caches include implementation- and version-specific details.
Experiment 1: import returns the cached module
import importlib
import sys
import tempfile
from pathlib import Path
name = '_deepcuts_cache_demo'
with tempfile.TemporaryDirectory() as directory:
Path(directory, name + '.py').write_text('values = []\n', encoding='utf-8')
sys.path.insert(0, directory)
try:
first = importlib.import_module(name)
first.values.append('changed')
second = importlib.import_module(name)
assert second is first
assert second.values == ['changed']
finally:
sys.path.remove(directory)
sys.modules.pop(name, None)
Import first checks sys.modules. If the name maps to a module, import normally returns it without searching or executing source again. Module globals are process-global state indexed by import name.
This cache is necessary for identity and circular imports, not merely speed. If every import created a new module, class identity, singleton registries, and shared configuration would fracture.
Tests should not casually clear all of sys.modules. The running test framework, plugins, codecs, and standard library depend on stable module identities. Own a unique name and remove only entries you created.
Experiment 2: deleting the entry creates a second module
import importlib
import sys
import tempfile
from pathlib import Path
name = '_deepcuts_second_module'
with tempfile.TemporaryDirectory() as directory:
Path(directory, name + '.py').write_text('token = object()\n', encoding='utf-8')
sys.path.insert(0, directory)
try:
first = importlib.import_module(name)
del sys.modules[name]
second = importlib.import_module(name)
assert second is not first
assert second.token is not first.token
finally:
sys.path.remove(directory)
sys.modules.pop(name, None)
The old module remains alive because first references it. Code that executed from module import token can likewise retain objects from the old generation while later imports use the new one. Removing sys.modules[name] does not globally unload a module or rewrite aliases.
This split identity is especially dangerous for classes: an instance of the old Widget is not an instance of a newly imported Widget, even if source is unchanged. Avoid delete-and-reimport as a generic reset strategy in application tests.
Python guarantee. A module may be removed from
sys.modules, but other references are unaffected. Importing it afterward can create a new module object.
Experiment 3: reload reuses the module dictionary
import importlib
import sys
import tempfile
from pathlib import Path
name = '_deepcuts_reload_demo'
with tempfile.TemporaryDirectory() as directory:
source = Path(directory, name + '.py')
source.write_text("survivor = globals().get('survivor', 0) + 1\n", encoding='utf-8')
sys.path.insert(0, directory)
try:
module = importlib.import_module(name)
module.extra = 'retained'
reloaded = importlib.reload(module)
assert reloaded is module
assert module.survivor == 2
assert module.extra == 'retained'
finally:
sys.path.remove(directory)
sys.modules.pop(name, None)
Reload recompiles and re-executes module code while retaining the module dictionary. Names redefined by source replace old bindings. Names omitted by the new execution remain. The survivor expression deliberately observes prior state.
Reload is therefore not "import as if for the first time." It is useful for a narrow test of reload-aware behavior or interactive development. It is a weak fixture reset when the desired contract is clean startup.
Imports elsewhere that used from module import name are not rebound by reload. Instances retain old class objects. Extension-module initialization may not be rerun and many extension modules are not designed for repeated initialization.
Experiment 4: import-time environment is a snapshot
import importlib
import os
import sys
import tempfile
from pathlib import Path
from unittest.mock import patch
name = '_deepcuts_environment_demo'
with tempfile.TemporaryDirectory() as directory:
Path(directory, name + '.py').write_text(
"import os\nMODE = os.environ.get('DEEPCUTS_MODE', 'default')\n",
encoding='utf-8',
)
sys.path.insert(0, directory)
try:
with patch.dict(os.environ, {'DEEPCUTS_MODE': 'first'}):
module = importlib.import_module(name)
with patch.dict(os.environ, {'DEEPCUTS_MODE': 'second'}):
assert importlib.import_module(name).MODE == 'first'
finally:
sys.path.remove(directory)
sys.modules.pop(name, None)
The module copied environment state during execution. Patching the environment later cannot change MODE. To test both startup configurations in-process, establish the environment before first import and carefully remove the owned module afterward. In pytest, monkeypatch.context() or a fixture can guarantee restoration.
The cleaner production design often reads configuration in an explicit load_settings(environ) function. Tests then call a pure boundary with controlled mappings, while application startup imports once and invokes it. Avoid import-time network calls, thread creation, and irreversible registration; they make import ordering part of system correctness.
Experiment 5: invalidating finder caches is a different operation
import importlib
import sys
import tempfile
from pathlib import Path
name = '_deepcuts_created_later'
with tempfile.TemporaryDirectory() as directory:
sys.path.insert(0, directory)
try:
try:
importlib.import_module(name)
except ModuleNotFoundError:
pass
Path(directory, name + '.py').write_text('answer = 42\n', encoding='utf-8')
importlib.invalidate_caches()
module = importlib.import_module(name)
assert module.answer == 42
finally:
sys.path.remove(directory)
sys.modules.pop(name, None)
invalidate_caches() asks finders on sys.meta_path to discard discovery caches. It does not remove loaded modules from sys.modules or reset their globals. Use it when a test or plugin system creates modules after a finder may have scanned the location.
Filesystem timestamp granularity can make rapid source replacement and bytecode cache reuse surprising on some systems. Creating unique module names avoids much of that ambiguity. Tests of dynamic module discovery should call cache invalidation explicitly after writing files.
Experiment 6: failed imports clean up, with caveats
import importlib
import sys
import tempfile
from pathlib import Path
name = '_deepcuts_failed_import'
with tempfile.TemporaryDirectory() as directory:
Path(directory, name + '.py').write_text("state = 'partial'\nraise RuntimeError('boom')\n", encoding='utf-8')
sys.path.insert(0, directory)
try:
try:
importlib.import_module(name)
except RuntimeError:
pass
assert name not in sys.modules
finally:
sys.path.remove(directory)
sys.modules.pop(name, None)
The import machinery inserts a module before executing it so circular imports can see a partially initialized object. If loading fails, it removes the failing module entry. Side effects already performed are not rolled back. A dependency successfully imported during the attempt can remain cached, a registry can remain mutated, and a file can remain written.
This is why import-time side effects should be minimal and reversible. Import provides namespace construction, not a transaction.
Circular-import failures may expose messages about partially initialized modules. Test public architecture, not exact CPython error wording.
Experiment 7: replacing sys.modules can supply a controlled dependency
import importlib
import sys
import tempfile
from pathlib import Path
from types import ModuleType
from unittest.mock import patch
dependency = '_deepcuts_optional_dependency'
consumer = '_deepcuts_optional_consumer'
fake = ModuleType(dependency)
fake.answer = 73
with tempfile.TemporaryDirectory() as directory:
Path(directory, consumer + '.py').write_text(
f'import {dependency}\nvalue = {dependency}.answer\n',
encoding='utf-8',
)
sys.path.insert(0, directory)
try:
with patch.dict(sys.modules, {dependency: fake}):
module = importlib.import_module(consumer)
assert module.value == 73
finally:
sys.path.remove(directory)
sys.modules.pop(consumer, None)
sys.modules.pop(dependency, None)
Preloading a fake module can test optional-dependency paths without installing that dependency. The fake must provide the interface consumed and, for packages or more advanced imports, appropriate metadata such as __spec__, __path__, and submodule entries.
Use this technique sparingly. It couples the test to import mechanics and can leave fake identities in aliases. Often a wrapper module or injected adapter is a cleaner seam. When import behavior itself is the subject, scoped patch.dict and unique names keep ownership clear.
Experiment 8: subprocesses provide real interpreter isolation
import subprocess
import sys
code = "import sys; sys.modules['_deepcuts_child_only'] = object(); print('ok')"
completed = subprocess.run(
[sys.executable, '-c', code],
check=True,
capture_output=True,
text=True,
)
assert completed.stdout.strip() == 'ok'
assert '_deepcuts_child_only' not in sys.modules
A fresh interpreter has a separate sys.modules, environment snapshot, import hooks, logging state, and extension-module state. Subprocess tests are slower but are the honest boundary for startup behavior, command-line entry points, interpreter flags, site initialization, and imports that cannot safely repeat.
Pass sys.executable so the child uses the tested interpreter. Control cwd, environment, and PYTHONPATH explicitly; otherwise the child may succeed because of the developer's installation. Capture stderr and return code as part of the assertion.
Test runners that distribute tests among worker processes provide partial isolation between workers, not necessarily between tests assigned to one worker. Order dependence can remain.
Experiment 9: explicit reset beats magical unloading
from functools import lru_cache
calls = 0
@lru_cache
def settings(profile):
global calls
calls += 1
return {'profile': profile}
assert settings('test') is settings('test')
assert calls == 1
settings.cache_clear()
assert settings('test') == {'profile': 'test'}
assert calls == 2
When application state is intentionally cached, expose or use its documented reset operation. lru_cache.cache_clear() states exactly which cache a test owns. A registry can offer clear_for_testing() if reset is a legitimate lifecycle operation, though dependency injection often avoids global reset entirely.
Do not reload a module just to clear one function cache. Reload may duplicate classes, handlers, and registrations while retaining names omitted by source. Reset the smallest state with clear ownership.
Isolation strategies
Choose the cheapest boundary that is semantically honest:
- Pass values or dependencies for ordinary behavior; no global reset is needed.
- Use fixture cleanup for mappings, registries, environment, and documented caches.
- Import a unique temporary module when testing import machinery itself.
- Use reload only when reload behavior is the subject or the module explicitly supports it.
- Use a subprocess for startup, import crashes, extension state, interpreter flags, and irreversible side effects.
- Randomize test order to reveal leaks, but fix ownership rather than accepting random failures.
An isolation bug has two parties: a test that leaves state and a later test that assumes freshness. Diagnose by finding the earliest mutation, not by adding a reset to whichever test failed. Fixtures should pair acquisition and cleanup, as part one of this series explains.
Parallel tests add another dimension. Process globals are isolated, but filesystem paths, ports, databases, environment inherited at worker start, and external services remain shared. Give each worker unique resources and make cleanup robust after crashes.
Collection imports before tests run
Test frameworks import test modules during collection. A fixture that patches the environment during test execution is therefore too late for module-level imports and decorators already evaluated at collection. Pytest plugins and conftest.py files can import application modules even earlier than expected.
If startup configuration must be controlled before collection, set it in the test command's environment, a dedicated test configuration entry point, or a subprocess. Better, move behavior out of import time so collection does not configure the application. A test file should be importable without opening databases, parsing production secrets, or starting workers.
Module-level parametrization, skip conditions, and decorator arguments are also evaluated during collection. They should depend on stable discovery facts, not mutable state another test plans to establish.
Diagnose order dependence systematically
When a failure appears only in the suite, first run the failing test with its immediate predecessor, then bisect the earlier tests to find the polluter. Snapshot likely registries, environment keys, logging handlers, warnings filters, and selected sys.modules entries before and after the suspect. The goal is to identify ownership, not add a blanket reset.
Order randomization broadens detection, while repeated runs reveal leaks that need accumulation. Neither proves isolation. A suite can pass thousands of random orders while one untried sequence remains invalid. Once the mutation is found, add a direct regression test showing that the responsible fixture restores it.
Import timings can help locate unexpected eager imports. CPython's -X importtime option reports timing and nesting, and verbose import tracing can show resolution paths. Those outputs are diagnostic and version-specific; do not freeze them as golden tests. Assert the intended application boundary instead, such as "importing this package does not connect to the network" in a controlled subprocess.
Exercises
- Reload a module after removing a name from its source. Observe the retained global and explain why reload did not delete it.
- Create a module defining a class, hold an instance, delete and reimport the module, then test both class identities.
- Refactor import-time environment parsing into a pure function and remove the need for module deletion in its tests.
- Write a subprocess test for an import that exits or configures logging. Assert exact externally relevant behavior only.
- Find one suite-global cache and document its owner, lifetime, reset API, and behavior under parallel test workers.
Keep this model
Import is stateful namespace construction. sys.modules maps names to loaded module objects; finder caches answer where modules may exist; module globals and application registries add further layers. Reload re-executes in an existing namespace, while deletion can create split module identities without updating old references.
Use explicit seams and targeted cleanup for ordinary tests. Reserve import manipulation for import behavior, and use a fresh interpreter when clean startup is the contract. Test isolation is not the absence of state; it is clear ownership of every state lifetime.