Python's import statement is syntax over a public protocol. After checking sys.modules, the import machinery asks finders whether they can locate a fully qualified name. A successful finder returns a ModuleSpec. The spec identifies a loader and records facts needed to create, initialize, execute, and represent the module. The loader supplies the code or other contents.

That division lets Python import source files, bytecode, extension modules, frozen modules, zip archives, namespace packages, and application-defined resources through one operation. It also makes import hooks sharp: they run in a global resolution path, can shadow real packages, participate in recursion, and must preserve invariants expected by tooling.

Version boundary. The finder/loader protocol and importlib abstract base classes are documented Python APIs. The default contents and order of sys.meta_path, frozen bootstrap implementation, extension loading details, and private helpers are implementation and version behavior. Experiments were run on CPython 3.14.7 and use unique names to avoid collisions.

Experiment 1: inspect the default meta path

Meta path finders are consulted for an unresolved fully qualified name.

import sys


for finder in sys.meta_path:
    print(type(finder).__name__, getattr(finder, "__module__", None))

On CPython 3.14 this normally reveals finders for built-in, frozen, and path-based modules. Exact representation and order belong to this interpreter installation. Applications may add finders, and test tools can modify the list.

The protocol is ordered: the first finder returning a non-None spec wins. A custom finder should decline unrelated names immediately. A broad hook that performs network access or expensive probing for every import slows startup and can interfere with standard modules.

sys.path is not itself the whole import mechanism. CPython's path-based finder interprets entries using path hooks and an importer cache. Meta path hooks sit one level earlier and can implement schemes unrelated to filesystem paths.

Experiment 2: ask for a spec without importing

find_spec() exposes discovery results.

import importlib.util
import sys


before = "email.parser" in sys.modules
spec = importlib.util.find_spec("email.parser")
print(before)
print(spec.name)
print(type(spec.loader).__name__)
print(spec.origin is not None)
print(spec.submodule_search_locations is None)

The final value is True because email.parser is a module, not a package. Finding a dotted child may import its parent to obtain the parent's search path, so find_spec() is not universally side-effect free. Discovery is less than execution, but it can still trigger package initialization.

Useful spec attributes include name, loader, origin, loader_state, submodule_search_locations, cached, and has_location. Not every loader gives every field the same meaning. Tooling must support modules without filesystem locations.

Experiment 3: create and execute from a spec

The public utility functions expose the loading steps, but callers must manage the cache invariant.

import importlib.util
import sys


name = "fractions"
spec = importlib.util.find_spec(name)
module = importlib.util.module_from_spec(spec)
existing = sys.modules.get(name)
sys.modules[name] = module
try:
    spec.loader.exec_module(module)
    print(module.__name__)
    print(module.__spec__ is spec)
    print(module.Fraction(1, 2))
finally:
    if existing is None:
        sys.modules.pop(name, None)
    else:
        sys.modules[name] = existing

module_from_spec() calls create_module() when supplied and initializes import attributes. exec_module() populates the namespace. In normal imports, bootstrap code handles insertion before execution and cleanup on failure. Manual loading assumes those responsibilities and can conflict with an already imported module, which is why normal import_module() is preferable unless a tool explicitly needs protocol control.

Experiment 4: implement one in-memory module

A loader can execute content that has no file.

import importlib.abc
import importlib.util
import sys


class MemoryLoader(importlib.abc.Loader):
    def create_module(self, spec):
        return None

    def exec_module(self, module):
        module.answer = 42
        module.describe = lambda: f"loaded by {type(self).__name__}"


loader = MemoryLoader()
spec = importlib.util.spec_from_loader("memory_demo", loader, origin="memory")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
try:
    loader.exec_module(module)
    print(module.answer)
    print(module.describe())
    print(module.__spec__.origin)
finally:
    sys.modules.pop(spec.name, None)

Returning None from create_module() asks import machinery to perform standard module creation. Most loaders should do that unless they need a specialized module type. exec_module() receives an initialized object and must populate it or raise.

Compiling text is another option, but then the loader must choose meaningful filenames, encoding behavior, and trust boundaries. Import hooks execute code with application privileges. Loading remote or user-controlled source is arbitrary code execution, not configuration parsing.

Experiment 5: connect a meta path finder

Install a finder temporarily and remove it by identity.

import importlib.abc
import importlib.util
import sys


class AnswerLoader(importlib.abc.Loader):
    def exec_module(self, module):
        module.value = 42


class AnswerFinder(importlib.abc.MetaPathFinder):
    def find_spec(self, fullname, path, target=None):
        if fullname != "virtual_answer":
            return None
        return importlib.util.spec_from_loader(fullname, AnswerLoader(), origin="generated")


finder = AnswerFinder()
sys.meta_path.insert(0, finder)
try:
    import virtual_answer
    print(virtual_answer.value, virtual_answer.__spec__.origin)
finally:
    sys.meta_path.remove(finder)
    sys.modules.pop("virtual_answer", None)

This is a complete import hook for one module. The machinery handles module creation, cache insertion, attributes, execution, and return. The finder handles recognition and specification; the loader handles population.

Use a reserved namespace such as myapp_generated.*, not common top-level names. In production, install hooks during single-threaded startup and remove test hooks in finally. Mutating sys.meta_path while other threads import creates process-wide races.

Experiment 6: package specs need search locations

A package is represented by non-None submodule_search_locations.

import importlib.machinery


package_spec = importlib.machinery.ModuleSpec(
    "virtual_package",
    loader=None,
    is_package=True,
)
module_spec = importlib.machinery.ModuleSpec(
    "virtual_leaf",
    loader=None,
    is_package=False,
)

print(package_spec.submodule_search_locations)
print(module_spec.submodule_search_locations)
print(package_spec.parent, module_spec.parent)

The package receives an empty list; the ordinary module receives None. A finder asked for virtual_package.child also receives the parent package's path argument. Custom package finders must use that context and return coherent specs for children.

Namespace packages can combine portions from multiple path entries and have no ordinary __init__.py loader. Code that assumes every package has a __file__ or one directory is incorrect. Use importlib.resources to access package data and spec search locations when discovery truly requires them.

Experiment 7: failure cleanup is part of import

Let normal machinery execute a loader that fails.

import importlib
import importlib.abc
import importlib.util
import sys


class BrokenLoader(importlib.abc.Loader):
    def exec_module(self, module):
        module.started = True
        raise RuntimeError("cannot load")


class BrokenFinder(importlib.abc.MetaPathFinder):
    def find_spec(self, fullname, path, target=None):
        if fullname == "broken_virtual":
            return importlib.util.spec_from_loader(fullname, BrokenLoader())
        return None


finder = BrokenFinder()
sys.meta_path.insert(0, finder)
try:
    try:
        importlib.import_module("broken_virtual")
    except RuntimeError:
        pass
    print("broken_virtual" in sys.modules)
finally:
    sys.meta_path.remove(finder)
    sys.modules.pop("broken_virtual", None)

The result is False: bootstrap machinery removes the failing entry. It cannot undo external effects performed by the loader. Loaders should minimize irreversible work before successful validation and make repeated attempts safe where practical.

Do not catch every loader exception and convert it to ModuleNotFoundError. "No finder accepts this name" differs from "the accepted module is broken." Preserving that distinction makes deployment errors diagnosable.

Experiment 8: cache invalidation reaches finders

Finders may cache discovery data and can participate in global invalidation.

import importlib
import importlib.abc
import sys


class CountingFinder(importlib.abc.MetaPathFinder):
    def __init__(self):
        self.invalidations = 0

    def find_spec(self, fullname, path, target=None):
        return None

    def invalidate_caches(self):
        self.invalidations += 1


finder = CountingFinder()
sys.meta_path.append(finder)
try:
    importlib.invalidate_caches()
    print(finder.invalidations)
finally:
    sys.meta_path.remove(finder)

This prints 1. Invalidation asks participating finders to discard stale search state. It does not unload modules from sys.modules. A dynamic finder should implement invalidation if backing resources can change after startup, and document freshness and concurrency behavior.

Path-based importing has another cache, sys.path_importer_cache, mapping path entries to path entry finders or None. Use public invalidation rather than editing it casually. Custom path hooks are appropriate when a new kind of sys.path entry is the abstraction; meta path finders are appropriate for namespace-wide policy or non-path discovery.

Design rules for real hooks

Keep recognition cheap and exact. Return None for names outside the hook's ownership. Do not recursively import the name currently being found. If implementation needs helper modules, import them before installing the hook or ensure its namespace rules exclude them.

Give specs honest metadata. origin should help diagnostics; package status must match child behavior; loader_state can pass finder data to the loader without global side channels. Support reload deliberately: find_spec receives target, and exec_module may run against an existing namespace. If reload is unsupported, document that rather than silently accumulating state.

Consider whether import is the right API. Import hooks are appropriate when users should naturally write import generated_schema.customer, when module identity and normal caching are valuable, and when code executes under trusted application control. A registry, factory, resource API, or plugin entry point is simpler when the artifact is data or when lifecycle must be explicit.

Security boundaries deserve particular skepticism. A signed archive can establish provenance, but the loader still executes code. Network retrieval introduces availability, caching, update, and supply-chain concerns into startup. Prefer installing verified distributions through normal packaging infrastructure.

Source loaders and diagnostics

A source-producing loader has more obligations than calling exec(text, module.__dict__). It should compile with a stable, meaningful filename so tracebacks, coverage, and debuggers can identify content. If no real file exists, it may need to cooperate with line caching or implement loader methods that return source. Encoding and newline handling should follow a documented format rather than ambient defaults.

importlib.abc.SourceLoader supplies a richer base for byte-oriented sources, including code generation and bytecode-cache hooks. Implementing its required data access methods can be safer than recreating source-import behavior. Cache writing is optional and environment-dependent; correctness must never require permission to create __pycache__.

Resource access should remain separate. A module's __file__ may be absent, virtual, or inside an archive. Loaders and packages can expose resources through importlib.resources protocols. Application code should request package resources through that API instead of joining paths beside __file__.

Reload and existing module state

importlib.reload() finds a spec and executes code in the existing module namespace. A loader can observe target during finding, but it should not assume a clean dictionary. Names omitted by the new execution can survive, and objects imported elsewhere keep old identities. Supporting reload means defining how registries, classes, external resources, and stale names behave.

Most custom loaders should promise ordinary first import and treat production upgrades as process restarts. Interactive tooling may offer reload with documented limitations. Tests should not use reload as a substitute for isolation; a subprocess provides a clearer first-import environment.

If a finder uses loader_state, build a fresh immutable or privately owned value for each spec. Shared mutable state can couple simultaneous modules and reloads. The module's spec is publicly inspectable, so do not place credentials or sensitive source material in it.

Concurrency and reentrancy

Import machinery coordinates loading, but a loader can call arbitrary Python and trigger nested imports. Keep loader state reentrant or scoped to the module spec. Do not hold an application lock while importing helpers if those helpers can call back into the hook. That pattern can create deadlocks outside importlib's own protection.

Long I/O in a finder or loader blocks whichever thread requested import and makes startup availability depend on a remote system. Download and verify artifacts before process startup, then let a deterministic local loader consume them. This also gives deployment systems a chance to audit and roll back artifacts.

For observability, record hook decisions at a debug level without logging every unrelated miss. Include fully qualified name, selected origin, loader identity, and duration. Never rely on logs as cache state; expose a diagnostic method for the hook's owned cache and invalidation generation.

Practical decisions

  • Use importlib.import_module() for ordinary dynamic imports and find_spec() for discovery questions.
  • Reserve a unique namespace and decline unrelated names immediately.
  • Separate finding from loading; communicate through a truthful ModuleSpec.
  • Let standard bootstrap machinery manage cache insertion and failure cleanup.
  • Install and remove hooks deterministically, preferably during single-threaded startup.
  • Implement cache invalidation and reload behavior only when backing resources require them.
  • Support packages and namespace packages without assuming __file__ exists.
  • Choose a simpler registry or resource API unless module semantics are genuinely useful.

Exercises

  1. Extend the answer hook to expose two modules under one reserved prefix without claiming unrelated names.
  2. Add a virtual package and child, using the parent path argument to validate child lookup.
  3. Compile trusted source in a loader with a meaningful virtual filename and inspect its traceback.
  4. Add a finder cache, prove stale discovery, then implement invalidate_caches().
  5. Make exec_module() fail after an external side effect and design an idempotent retry.
  6. Compare a meta path hook, path hook, and explicit registry for one application scenario.

Keep this model

On a cache miss, finders answer whether and how a fully qualified name can be loaded. A ModuleSpec carries that answer. A loader creates or populates the module, while bootstrap machinery establishes attributes, inserts the object before execution, and cleans up a failing entry. Packages add search locations that guide child discovery.

An import hook is therefore a participant in global name resolution, not a fancy file reader. Keep its namespace narrow, metadata coherent, lifecycle deterministic, and failure transparent.

Primary sources