Skip to content

Persist the numba compile of the built-in UDF binops to disk - #611

Open
eriknw wants to merge 1 commit into
30-udf-import-costfrom
31-numba-cache-builtin-udfs
Open

eriknw wants to merge 1 commit into
30-udf-import-costfrom
31-numba-cache-builtin-udfs

Conversation

@eriknw

@eriknw eriknw commented Aug 4, 2026

Copy link
Copy Markdown
Member

floordiv, rfloordiv, absfirst, abssecond, and rpow are UDFs, so every
process that touched one paid its numba compile again. They are plain
module-level functions, which is exactly the case numba can cache, so
that cost belongs once per machine rather than once per process.

Thread the flag through as a private keyword-only _cache on
register_new into _build, where it reaches numba.njit alongside the
existing error_model. Keeping it private avoids widening the public API
for something no caller outside this module should set. Ops registered by
users stay uncached: numba keys a cache entry on a stable on-disk source,
which lambdas and interactively defined functions do not have.

Measured here, fresh process, medians of 3, load average ~4 on a shared
machine so read these as indicative. First-touch floordiv alone: 434 ms
uncached, 347 ms warm (-20%). First-touch of all five: 1337 ms uncached,
733 ms warm (-45%). Writing a cold cache costs about 107 ms once for all
five (1444 ms on the run that populates it).

What stays: the remaining ~733 ms is numba/LLVM startup and per-process
object linking, which no on-disk cache removes. In-process dispatch once
an op is built is untouched. The first process on a machine is slightly
slower, not faster.

Read-only installs are safe, verified rather than assumed. With
graphblas/core/operator and its pycache chmod'd a-w, the run wrote
zero files in-tree and 66 .nbi/.nbc files under ~/Library/Caches/numba,
with empty stderr under python -W always. The NumbaWarning in
numba/core/caching.py is a source-content check, not a filesystem one.

docs/user_guide/udf.rst gains a "Compilation caching" section covering the
cache location, the read-only fallback, and why user ops are excluded.


Stack created with GitHub Stacks CLI • Give Feedback 💬

@eriknw
eriknw marked this pull request as ready for review August 4, 2026 16:07
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 4130be1 to e848303 Compare August 4, 2026 16:12
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from e848303 to 408900a Compare August 5, 2026 00:06
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 408900a to e4f6717 Compare August 5, 2026 03:18
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from e4f6717 to 6aecc12 Compare August 5, 2026 17:44
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 6aecc12 to a934b2d Compare August 5, 2026 18:03
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from a934b2d to f8c1826 Compare August 5, 2026 18:05
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from f8c1826 to 188b387 Compare August 6, 2026 07:59
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 188b387 to d57105f Compare August 6, 2026 15:39
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch 2 times, most recently from 2265b21 to c4a03f3 Compare August 6, 2026 20:36
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from c4a03f3 to 5fc2928 Compare August 6, 2026 20:42
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 5fc2928 to e2a01e7 Compare August 7, 2026 02:49
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from e2a01e7 to ae91f10 Compare August 7, 2026 05:09
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from ae91f10 to 75c4d36 Compare August 26, 2026 17:30
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch 2 times, most recently from 005294d to 42bf923 Compare September 18, 2026 03:46
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 42bf923 to 3dfcd58 Compare September 18, 2026 17:34
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 3dfcd58 to 2416d28 Compare September 18, 2026 17:39
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 2416d28 to 0c3b23e Compare September 23, 2026 16:22
@eriknw
eriknw removed this pull request from stack #627 September 23, 2026 18:18
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 0c3b23e to 466cd68 Compare September 23, 2026 18:19
@eriknw
eriknw added this pull request to stack #634 September 23, 2026 18:19
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch 2 times, most recently from efc3e64 to 9a1923c Compare September 24, 2026 20:07
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 9a1923c to 46643af Compare September 25, 2026 03:33
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch 2 times, most recently from 408fc61 to 4210f18 Compare September 26, 2026 05:23
floordiv, rfloordiv, absfirst, abssecond, and rpow are UDFs, so every
process that touched one paid its numba compile again. They are plain
module-level functions, which is exactly the case numba can cache, so
that cost belongs once per machine rather than once per process.

Thread the flag through as a private keyword-only `_cache` on
register_new into _build, where it reaches numba.njit alongside the
existing error_model. Keeping it private avoids widening the public API
for something no caller outside this module should set. Ops registered by
users stay uncached: numba keys a cache entry on a stable on-disk source,
which lambdas and interactively defined functions do not have.

Measured here, fresh process, medians of 3, load average ~4 on a shared
machine so read these as indicative. First-touch floordiv alone: 434 ms
uncached, 347 ms warm (-20%). First-touch of all five: 1337 ms uncached,
733 ms warm (-45%). Writing a cold cache costs about 107 ms once for all
five (1444 ms on the run that populates it).

What stays: the remaining ~733 ms is numba/LLVM startup and per-process
object linking, which no on-disk cache removes. In-process dispatch once
an op is built is untouched. The first process on a machine is slightly
slower, not faster.

Read-only installs are safe, verified rather than assumed. With
graphblas/core/operator and its __pycache__ chmod'd a-w, the run wrote
zero files in-tree and 66 .nbi/.nbc files under ~/Library/Caches/numba,
with empty stderr under `python -W always`. The NumbaWarning in
numba/core/caching.py is a source-content check, not a filesystem one.

docs/user_guide/udf.rst gains a "Compilation caching" section covering the
cache location, the read-only fallback, and why user ops are excluded.
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 4210f18 to e8f5737 Compare September 26, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant