Skip to content

Numba's on-disk cache

cache=True (the numba-utils default) stores compiled binaries next to the source (__pycache__/*.nbi + *.nbc), keyed by signature and compilation flags. Later runs load the binary instead of recompiling — for scripts this turns a seconds-long warmup into milliseconds. This is what Numba recommends, and it is the right default for most users.

This page exists because of the environments where it is NOT safe.

Where the cache breaks

The cache assumes a stored binary is valid for any process that loads it. In practice, these setups have produced intermittent crashes (access violation 0xC0000005 on Windows, SIGSEGV on Linux) when a process loads a binary compiled by a different process:

  • Multi-process worker farms — many workers importing the same module, one of them compiled the cache, the rest load it. This has crashed even on pure, non-parallel kernels.
  • Network / shared filesystems — the cache directory is shared between machines that don't share an ABI.
  • Ephemeral container storage — stale caches surviving image layers or volume mounts they weren't compiled in.

Recognizing it

The failure signature is distinctive:

  • The first run (the one that compiled) works; later runs crash.
  • Crashes look random — same code, same data, different outcome.
  • Deleting __pycache__ "fixes" it… until the next run repopulates it.
  • The crash is a hard process death (no Python traceback).

If that pattern matches, stop debugging your kernel: it's the cache.

The fix

Disable caching globally — this is a policy decision about the environment. One import-time detail decides which switch you need: numba-utils' own kernels are compiled with cache=True the moment numba_utils is imported, so only a switch that is already active at that point can reach them.

The full kill-switch is the environment variable, set before the first import:

NUMBA_UTILS_CACHE=0                # exported in the shell / CI / farm

or, equivalently, at the very top of the entry script:

import os
os.environ["NUMBA_UTILS_CACHE"] = "0"   # BEFORE importing numba_utils

Either form overrides everything — explicit cache=True arguments, your kernels, and the library's own — because the environment is read each time a function is decorated.

The code-level override covers your kernels only:

import numba_utils as nu
nu.configure(cache=False)          # functions decorated after this call

By the time configure() can run, the library's kernels are already decorated, so it cannot reach them. On an affected machine, use the environment variable.

Cost: each process recompiles fresh (seconds to a couple of minutes for large kernels). For a worker farm, amortize by running long-lived worker processes rather than fanning out a process per task.

Hygiene for affected machines: clear __pycache__ before runs, and avoid chaining multiple Numba-heavy script invocations in one shell.

A separate cache gotcha on Windows: running from long, mapped or otherwise unusual paths can break Numba's cache locator entirely — every cached function fails with RuntimeError: cannot cache function ... no locator available. If a suite that passes from the repo fails wholesale with that error, copy the tree to a short real path (or set NUMBA_UTILS_CACHE=0) before concluding anything about the code.

Upgrading numba-utils: the stale-binary window

Numba invalidates a cached binary when the source file's (mtime, size) stamp changes — not by content, and not by package version. What that means in practice:

  • A plain pip install --upgrade is safe. pip rewrites the .py files, so the mtime is fresh and the old binaries are invalidated.
  • Deployments that PRESERVE mtime are the dangerous case. docker COPY (build-context files keep their timestamps), tar -x, rsync -a, cp -p: if the new file's size happens to match the old one's, the stamp is identical and the process silently keeps LOADING the binary compiled from the previous version. A compile-time gate added in the release (a new dtype-validation check, a changed kernel) never runs; nothing warns. This scenario has been reproduced, not just theorized.

Two fixes:

  • Hygiene: clear __pycache__ on deploy (or run once with NUMBA_UTILS_CACHE=0). Works, but every deploy has to remember it.
  • Structural: stamp by content instead of (mtime, size), via Numba's official locator hook, set before the first numba import:
NUMBA_CACHE_LOCATOR_CLASSES=numba_utils.cache_locator.ContentHashLocator

numba_utils.cache_locator.ContentHashLocator stamps sources with a SHA-256 of their bytes, so a changed file always invalidates — regardless of what the deployment did to timestamps. Caveats in the module docstring: the variable replaces Numba's locator chain process-wide (append Numba's own locators by bare name if you need zip-import fallbacks), and it costs one hash per cached function per process.

Note diagnostics.check(fn) flags that caching is enabled (the crash gotcha above) — it does not and cannot detect a stale binary.

Trade-off summary

Environment Recommendation
Single-process scripts, notebooks, dev loops Keep the default (cache=True)
Multi-process farms, shared FS, containers NUMBA_UTILS_CACHE=0 set before the first import