Python applications constantly cross boundaries. A call to sum enters implementation code written in C in CPython. json.dumps traverses Python objects through an accelerator when available. A socket write enters the operating system. A process-pool submission serializes a request and sends it to another interpreter. NumPy, database drivers, compression modules, and native extensions each place a different boundary around work.

"Move the loop into C" is useful advice only when it identifies the right loop. Crossing a boundary has fixed costs: calls, argument conversion, validation, serialization, copying, scheduling, and error translation. Native code wins when enough suitable work happens on the far side. Crossing once per scalar can lose to a simple Python loop; crossing once per batch can transform throughput.

This tutorial uses standard-library experiments so every result is reproducible without optional dependencies. Run Python blocks as .venv/bin/python experiment.py. Replace the payloads with production shapes before making a production decision.

Test environment. Recorded results came from CPython 3.14.7, arm64, macOS 26.5.2, traditional GIL enabled, optimization level 0, non-debug build. Built-in implementation paths, specialization, extension accelerators, process startup, and operating-system calls vary by Python implementation, exact version, build, and platform. Timings are observations, not language guarantees.

Experiment 1: let a built-in own the reduction

Start with equivalent integer addition:

import timeit


setup = "values = range(10_000)"
python_loop = """
total = 0
for value in values:
    total += value
"""

for label, statement in (("Python loop", python_loop), ("sum", "sum(values)")):
    runs = timeit.repeat(statement, setup, number=1_000, repeat=5)
    print(label, [round(run / 1_000 * 1e6, 1) for run in runs])

Our best results were 199 us for the explicit loop and 68.9 us for sum. The language guarantee is that sum adds the input from left to right with its documented start value. The speed difference is a CPython 3.14 result: the built-in controls iteration and accumulation in implementation code instead of dispatching the Python loop body for every item.

Built-ins also communicate intent and have tested edge behavior. Prefer sum, min, max, any, all, sorted, bytes.join, and str.join when their semantics exactly match. Do not replace a loop that performs validation, short-circuit policy, logging, or domain-specific error handling merely to reach a built-in.

Input type matters. Summing range integers, arbitrary objects with __add__, floats with accuracy requirements, and decimal values can take different paths and have different semantics. CPython has changed sum implementations across releases. Benchmark the exact types you carry.

Experiment 2: one operation can replace thousands of crossings

Joining byte strings is not just prettier concatenation:

.venv/bin/python -m timeit -r 5 -n 1000 -s 'parts = [b"abcd"] * 10000' 'b"".join(parts)'
.venv/bin/python -m timeit -r 5 -n 100 -s 'parts = [b"abcd"] * 10000' 'out = b""' 'for part in parts: out += part'

Our best results were 123 us for join and 21.6 ms for repeated concatenation. Immutable byte concatenation repeatedly creates and copies growing results. join can inspect all parts, calculate the final size, allocate once, and copy into that destination.

This is more than "C is faster than Python." The algorithm and allocation strategy changed. A native extension that performed the same repeated-growing-copy algorithm would still do avoidable work. Look first for an operation that gives the far side enough information to choose a better algorithm.

For a small number of pieces, readability dominates and the difference may disappear. For incrementally produced output that cannot be retained as parts, use io.BytesIO, a buffered writer, or protocol-aware streaming. A giant join trades repeated copies for retaining all inputs and one final contiguous allocation.

Experiment 3: tiny C calls still leave a Python loop

Calling a C-backed operation once per element does not move the surrounding loop:

.venv/bin/python -m timeit -r 5 -n 1000 -s 'parts = [b"abcd"] * 10000' 'sum(len(part) for part in parts)'
.venv/bin/python -m timeit -r 5 -n 1000 -s 'parts = [b"abcd"] * 10000' 'len(b"".join(parts))'

Our best results were 277 us for summing individual lengths and 138 us for joining then measuring. These candidates answer the same total-length question for these inputs, but the second also allocates and copies 40,000 bytes. It happens to win here because one operation owns the traversal; that does not make it a sensible way to count bytes in production.

The experiment isolates granularity. len(part) is fast, yet Python still resumes a generator and invokes it 10,000 times. A good API accepts a collection, buffer, iterator, or column and performs substantial work per call. A poor optimization builds an unnecessary aggregate merely to reduce calls.

When designing your own extension or service client, benchmark payload size across several orders of magnitude. Plot total time and time per element. The fixed crossing cost appears where tiny batches have poor per-element throughput; memory or latency limits appear as batches grow.

Experiment 4: serialization has an envelope cost

Encoding one JSON document per record repeats encoder setup and creates many result strings. Compare that with one JSON array:

import json
import timeit


records = [{"id": value, "ok": True} for value in range(1_000)]
cases = {
    "one document each": "[json.dumps(record) for record in records]",
    "one array": "json.dumps(records)",
}

for label, statement in cases.items():
    runs = timeit.repeat(statement, globals=globals(), number=100, repeat=5)
    print(label, [round(run / 100 * 1e6, 1) for run in runs])

Our best results were 1,369 us for 1,000 separate documents and 251 us for one array. The outputs are not wire-compatible: newline-delimited JSON, independent queue messages, and a JSON array have different framing, streaming, retry, and consumer behavior. The timing demonstrates repeated boundary and envelope work; it does not authorize a protocol change.

Batch within the protocol you actually control. A service might accept up to 100 independent records in one request while preserving per-record statuses. A stream might retain newline framing but encode records into a buffered chunk. Include UTF-8 encoding, compression, network writes, parsing, and response handling when they are paid per production operation.

CPython normally imports the _json accelerator, but that is an implementation detail. The documented behavior belongs to json; another implementation or build may choose a different engine. Confirm accelerator availability only when diagnosing an environment-specific regression, not as application logic.

Experiment 5: batch writes without pretending buffering is delivery

io.BytesIO gives a deterministic way to observe Python-to-writer call granularity:

import io
import timeit


parts = [b"abcd"] * 10_000


def individual():
    sink = io.BytesIO()
    for part in parts:
        sink.write(part)
    return sink.getvalue()


def batched():
    sink = io.BytesIO()
    sink.writelines(parts)
    return sink.getvalue()


assert individual() == batched()
for function in (individual, batched):
    runs = timeit.repeat(function, number=100, repeat=5)
    print(function.__name__, min(runs) / 100 * 1e6)

Our best observations were 220 us for individual writes and 130 us for writelines. BytesIO makes no system calls, so the result measures method-call and in-memory writer behavior, not socket or disk performance. On a real unbuffered file or socket, many tiny writes can add much larger syscall and packet costs. On an already buffered stream, the buffer may coalesce them.

writelines does not insert separators, and a successful buffered write does not mean bytes are durable on disk or acknowledged by a peer. Keep flush, fsync, transaction, timeout, and partial-write semantics explicit. Batch size should respect latency: waiting indefinitely to fill a large buffer can improve throughput while violating a request deadline.

For network experiments, run against a controlled local and remote endpoint, record payload sizes, and report throughput plus latency percentiles. A BytesIO microbenchmark is evidence about one layer only.

Experiment 6: database APIs make the boundary visible

The standard-library SQLite driver offers both one-execute-per-row and batched parameter binding:

import sqlite3
import timeit


rows = [(value, str(value)) for value in range(1_000)]


def insert(individual):
    connection = sqlite3.connect(":memory:")
    connection.execute("create table records (id, value)")
    if individual:
        for row in rows:
            connection.execute("insert into records values (?, ?)", row)
    else:
        connection.executemany("insert into records values (?, ?)", rows)
    connection.commit()
    connection.close()


for individual in (True, False):
    runs = timeit.repeat(lambda: insert(individual), number=100, repeat=5)
    print(individual, min(runs) / 100 * 1e6)

Our in-memory best times were 974 us for the Python execute loop and 534 us for executemany. Both use one transaction, which is crucial: committing each row would mostly benchmark transaction durability and would answer a different question.

On a client/server database, network round trips, server planning, locks, constraints, and transaction logging dominate differently. Driver executemany implementations also vary: some loop internally, some rewrite statements, and some support dedicated bulk protocols. Read the driver's primary documentation and inspect server-side evidence.

Batches alter failure semantics. Determine which row failed, whether earlier rows committed, how retries avoid duplicates, and whether one pathological row rejects the whole batch. Throughput is not correctness.

Experiment 7: process boundaries need coarse work

Processes can run CPU-bound Python in separate interpreters, but submission and result transfer are not free. This complete script measures one worker after paying startup once:

from concurrent.futures import ProcessPoolExecutor
from time import perf_counter


def square(value):
    return value * value


def square_batch(values):
    return [value * value for value in values]


if __name__ == "__main__":
    values = list(range(10_000))
    with ProcessPoolExecutor(max_workers=1) as pool:
        pool.submit(square, 0).result()  # Start the worker outside measurement.

        start = perf_counter()
        small = list(pool.map(square, values, chunksize=1))
        small_time = perf_counter() - start

        start = perf_counter()
        large = pool.submit(square_batch, values).result()
        large_time = perf_counter() - start

    assert small == large
    print(f"10,000 submissions: {small_time:.4f}s")
    print(f"one submission:      {large_time:.4f}s")

Run it as a file, not from an interactive heredoc: process startup imports the main module, and the if __name__ == "__main__" guard is required on spawn-based platforms. Results vary dramatically by OS and start method. The durable lesson is the shape of the experiment: separate startup from steady state when workers are long-lived, preserve startup when jobs are one-shot, and compare equal outputs.

One worker cannot make the calculation itself parallel. It reveals serialization, queueing, scheduling, and result-transfer overhead. Add workers only after each task is coarse enough, then measure wall time, CPU utilization, and memory. Large arguments may be copied or serialized; shared memory removes some copies but adds lifetime and synchronization complexity.

Threads are often appropriate when a library releases the GIL around blocking I/O or native computation. Processes help CPU-bound Python at the cost of isolation. CPython's free-threaded build changes the tradeoff but remains a distinct build with extension-compatibility and workload-specific performance considerations. State which build you tested.

Experiment 8: measure the call before writing an extension

Native does not mean zero overhead. timeit can compare a minimal C-backed built-in call with a Python call:

import timeit


value = []


def identity(argument):
    return argument


for statement in ("len(value)", "identity(value)"):
    runs = timeit.repeat(statement, globals=globals(), number=1_000_000, repeat=7)
    print(statement, min(runs) / 1_000_000 * 1e9, "ns")

On our CPython 3.14 environment both are only tens of nanoseconds, and exact results depend on specialization and the harness. That is the wrong scale for deciding whether to maintain C, C++, or Rust integration. Argument conversion, buffer acquisition, callback frequency, error handling, output construction, and the useful native kernel must be measured together.

Before writing an extension, look for a standard-library primitive or a mature native-backed package whose operation matches the domain. For dense numeric or tabular workloads, vectorized libraries can place whole arrays behind one call and operate on packed memory. They are optional dependencies, so evaluate installation size, supported platforms, data-conversion copies, dtype semantics, missing-value behavior, and whether your deployment already uses them. Do not add a large dependency to accelerate a cold loop.

When no existing operation fits, prototype the API boundary before its implementation. Prefer buffers, arrays, or batches over one callback per element. Keep ownership explicit and test exceptions. The stable Python/C API and limited API can reduce CPython-version coupling, while direct internals may offer capabilities at greater maintenance cost. Tools such as Cython, pybind11, cffi, and Rust bindings make different portability and build-system trades; consult their current primary documentation when selecting one.

Choosing the far side

Use the least operationally expensive boundary that removes meaningful work:

  1. Built-in or standard library: first choice when semantics fit; minimal dependency and packaging cost.
  2. Batch an existing I/O API: often the largest application win because it reduces calls, round trips, and envelopes.
  3. Native-backed library: strong for established domains and packed data; include conversion and deployment costs.
  4. Threads: useful when blocking operations or native kernels release the GIL; shared state remains shared.
  5. Processes: useful for coarse isolated work; pay startup, serialization, transfer, and extra memory.
  6. Custom extension: justified for a stable hot kernel not served elsewhere; budget for builds, security, debugging, and releases.
  7. Alternate implementation: consider PyPy or another runtime when the whole application's dependencies and workload fit, not from one loop benchmark.

An alternate interpreter can optimize long-running Python loops differently, while C-extension compatibility and warmup can reverse the choice. Run the full test suite and representative service benchmark on the exact runtime. The Python language defines behavior; no implementation promises the same performance profile.

A boundary benchmark checklist

For every proposed crossing, record:

  1. The useful work performed per call or message.
  2. Input and output types, sizes, ownership, and copies.
  3. Cold startup, warmup, and steady-state boundaries.
  4. Serialization, conversion, queueing, and synchronization included in production.
  5. Equivalent results, errors, transaction behavior, ordering, and retry semantics.
  6. Throughput, median and tail latency, CPU, and peak memory at realistic concurrency.
  7. Dependency, packaging, portability, observability, and maintenance costs.

Microbenchmarks identify fixed overhead and promising batch sizes. End-to-end tests decide whether users benefit. If a loop consumes 2% of request time, making it infinitely fast cannot recover the other 98%.

Exercises

  1. Compare any(predicate(x) for x in values) with a hand-written loop. Test an early match, late match, and no match; preserve short-circuit behavior.
  2. Benchmark JSON batches of 1, 10, 100, and 1,000 representative records. Include encoded byte size and choose a batch under a latency limit.
  3. Replace BytesIO with a temporary buffered file. Measure write time separately from flush and fsync, and explain which metric maps to durability.
  4. Repeat the SQLite experiment with commits per row, one transaction, and executemany. Attribute the differences to boundaries rather than labeling all of them "Python overhead."
  5. Run the process experiment with increasing CPU work per task and multiple workers. Find where parallel wall time first beats local sequential execution on your deployment platform.
  6. Choose one optional vectorized library already present in a real project. Include conversion into and out of its native representation in the benchmark.

Keep this model

A boundary is valuable when substantial useful work happens beyond it. Built-ins can own a loop. Batch APIs can amortize calls, envelopes, transactions, and system calls. Packed native data can improve both execution and locality. Processes can provide parallel execution while charging for isolation.

Count crossings, but also inspect what each crossing does. A faster implementation of the wrong algorithm remains wrong; a giant batch can violate latency and recovery requirements; a native extension can cost more to ship than it saves at runtime. Start with the highest-level operation whose semantics fit, make work coarse, include conversion and failure behavior, and decide from end-to-end evidence.

Primary sources