← Research

The copy nobody needed

2026-07-30 · speed · 8 min

Compiled Mojo was slower than CPython on a million-element list. More than half of that was a copy back the kernel had provably not earned — plus three name-collision bugs found by reading compiler stderr.

mojosub's lab notebook had a finding in it that we had stopped questioning: list arguments make the compiled path lose. A million-element list[float] has to be copied into a contiguous buffer to cross the C ABI, and for a cheap kernel that copy costs more than the native code saves. The recorded number was dot over 1e6 elements going from 36x faster on a numpy array to 3x slower on a list.

It was true. It was also, in more than half, our own fault.

Where the time went

Marshalling a sequence across the boundary is two copies, not one:

buf = array.array("d", obj)      # in:  boxed floats -> 64 contiguous bits
addr, length = buf.buffer_info()
result = fn(addr, length)         # the kernel
obj[:] = buf.tolist()             # out: in case the kernel wrote to it

The copy in is unavoidable. A Python list is an array of pointers to individually allocated float objects; there is no address you can hand a compiled function. The copy out is a different matter. It exists because the kernel might have stored through the buffer — and for a reduction, it provably has not.

And the transpiler already knows which, because it has just finished deciding what to emit. If it never generated a store through a parameter, nothing can have been written through it. That is a fact available for free, at transpile time, that was being thrown away.

Recording it

Each parameter a store was emitted for goes on the signature, and the runtime skips the write-back for the rest:

@dataclass
class Signature:
    name: str
    params: list[tuple[str, Ty]]
    ret: Ty
    writes: frozenset[str] | None = None   # None means "not analysed"

    def may_write(self, param: str) -> bool:
        return self.writes is None or param in self.writes

None having to mean "may write everything" is the important part: a hand-built signature must not silently lose a write-back. The default is the safe answer, and only an actual analysis can narrow it.

What it was worth

Same kernel, same lists, copy-back forced on and off:

Read the bottom row twice. 259 ms against CPython's 253 ms is the original finding — the compiler losing. Two thirds of it was a tolist() and a slice assignment undoing nothing.

With it gone there is no size at which a list argument loses:

The heuristic we then did not build

This measurement was supposed to feed a cost model. The plan was to predict, before spending five seconds of compiler, whether a call would be copy-dominated and lose — using the interpreted time the JIT has already measured and a copy cost per element, giving a ceiling on the achievable speedup so it could only ever reject a case that provably could not win.

Then the copy-back fix removed the losses it was meant to avoid. A model whose job is to predict a 0.94x that no longer happens is a model with no job.

So the finding was reported instead of acted on. The gap between a list and a buffer is still two orders of magnitude, and unlike everything else in the pipeline it is the caller's to close. mojosub counts the elements it had to copy, and a run says so:

{"accel": {"native_calls": 2, "hints": [
  "dot() copied 800,000 elements across the native boundary because a, b
   arrived as Python lists. Passing numpy arrays (or array.array('d', ...))
   hands them over by address instead — measured up to 100x faster on the
   same kernel."]}}

Escape analysis stops early on purpose

A buffer handed to a helper is assumed written, whatever the helper does with it. Following it properly is a real escape analysis, and the failure mode of getting that wrong is not a slow program — it is a silently stale array in the caller's hands. So the conservative case keeps a copy back it may not need, and the aggressive case does not exist yet.

Three bugs in a variance

The same corpus run turned up something worse, by the crude method of reading what the compiler said instead of only whether it succeeded. This is ordinary statistics code:

def variance(xs) -> float:
    total = 0.0
    for i in range(len(xs)):
        total += xs[i]
    mean = total / len(xs)
    var = 0.0                 # <- 1
    for i in range(len(xs)):
        fn = xs[i] - mean     # <- 2
        var += fn * fn
    return var / len(xs)

It emitted this:

var var: Float64 = 0.0
var fn: Float64 = 0.0

var and fn are reserved words in Mojo and perfectly ordinary names in Python. So do alias, struct, ref, owned and inout. Naming a variance var is not exotic — and the cost of it was five seconds of host compiler per submission, for ever, with the kernel silently staying interpreted, because a failed build is precisely what the tiered design is built to shrug off.

The third bug is the one that should worry you:

def count_above(xs, cut: float) -> float:
    xs_len = 3.0          # the user's own variable
    ...
    for i in range(len(xs)):   # len(xs) emits `xs_len`
        ...

xs_len is the parameter we synthesise for a buffer's length. A program with its own variable of that name reassigned it, so every later len(xs) returned 3. Not a crash, not a compile error: a wrong answer, silently, in whatever direction the collision happened to point.

The fix is not a keyword list

Blacklisting Mojo's reserved words is a race against a language still adding them, and it would not have caught xs_len at all. Two namespaces instead: every user identifier is emitted with a u_ prefix, every synthesised length with n_.

def u_variance(u_xs: UnsafePointer[Float64, AnyOrigin[mut=True]], n_xs: Int) -> Float64:
    var u_var: Float64 = 0.0
    var u_fn: Float64 = 0.0
    ...

Both maps are injective and their images are disjoint, so no user name can collide with a keyword, with a generated length, or with another user name — including a program that has both xs and xs_len, which a suffix scheme cannot handle at all. Generated temporaries keep leading underscores and live in neither namespace. The exported symbol keeps the Python name, because that is a string the runtime looks up, not an identifier.

One kernel in the corpus — a rolling Sharpe ratio, which computes a variance, which calls it var — went from never compiling to 423x (3597 ms → 8.5 ms).

And a bug that only exists at fleet scale

While measuring vector widths we read this in mojo build --help:

--target-cpu <CPU> — Sets the compilation target CPU. Defaults to the host CPU.

The compile cache is content-addressed on the Mojo source, the compiler version and the optimization level. Not the machine. On one box that is invisible. But this service spills over onto rented capacity and the cache syncs between machines through an object store — so a unit built where AVX-512 exists and dlopened where it does not will load fine, run fine, and die on the first vector instruction it was compiled to use.

The host's CPU feature flags are now part of the key — the flags rather than the model name, because the same model number ships with features fused off. Machines that can run each other's code still share cache entries; machines that cannot get their own, which is what they always needed. Nobody had hit it, which is the only good thing about it.

Reproducing

bench/run.sh bench/fallback_study.py --copyback     # the write-back saving
bench/run.sh bench/fallback_study.py --crossover    # list vs array, by size
bench/run.sh bench/fallback_study.py --opsweep      # arithmetic per element

The namespacing is in mojosub/transpile.py, the write-back elimination spans transpile.py and runtime.py, and the cache key is in mojosub/cache.py. Each has a named regression test in tests/test_transpile.py — including one asserting a read-only list argument comes back unmodified, since that is the only thing skipping the copy risks.

Measured on the machine that served this page: Intel Xeon E5-2697 v4 (Broadwell, AVX2), CPython 3.13, Mojo 1.0.0b3.dev2026072406. Numbers move between machines; the shape of the result is what we are claiming.