Problem
Every top-level binding lives in the M table, so every inter-binding reference is a hash lookup: the linked Data.Array output has 681 M.* references across 124 bindings. Measured: reading M.field in a loop is 2.4x slower than a local under PUC Lua 5.1; in a LuaJIT trace it is a guarded load where a local is a register.
History (the constraint that shaped the current design): early codegen emitted top-level bindings as chunk locals and real programs hit Lua 5.1 limits — 200 locals per chunk, and the 60-upvalue limit amplified by 5.1's pass-through accumulation, where a nested function reading an outer local costs an upvalue slot in every intermediate function (#19). The M table was the fix: unbounded fields, and M itself is the single upvalue everywhere. Any revisit must handle the limits by construction, not assume programs stay small.
Approach
Budget-aware emission with M as the overflow valve (the design principle). Every budget overflow degrades the specific reference or binding back to today's M form, so the worst case is exactly the current output and the limits stop being correctness cliffs.
Stage 1: per-function caching of M fields (limit-proof by construction)
Storage untouched: M remains the single source of truth and the single upvalue, so neither limit can be approached. A Lua.Optimizer pass caches M.x fields used two or more times in a generated function body into locals at function entry (capped at ~30 per function, prioritized by use count):
M.Data_Array_span = function(p)
return function(arr)
local index, add = M.Data_Array_index, M.Data_Array_add
local go
go = function(i)
local v = index(arr)(i)
-- ...
end
Semantics: the read moves to function entry — by the time any generated function is called, module init has assigned the bindings it reads, and M is not mutated within a call; the runtime-lazy scheme is unaffected. This is the classic "localize globals" optimization from LuaJIT guides.
Stage 2: two-tier storage with budget accounting
Top-K bindings by static reference count are emitted as real chunk locals; the tail stays in M; exported bindings are mirrored into the export surface. Two budgets are computed over the finished Lua AST before printing:
- Locals: a chunk-local counter with a ceiling of ~180 (200 minus fixture locals and headroom). Overflow keeps the binding in
M.
- Upvalues: the killer of the old design. Accounting runs bottom-up over the function tree:
upvals(f) = |own outer-local references ∪ children's pass-through demands|. When a function proto would exceed ~55, individual references are demoted — printed as M.x — while the binding stays a local for everyone else.
A program that fits the budgets (like Data.Array with 124 bindings) loses the M table entirely: pure locals, with the module export table referencing them directly. K for larger programs is chosen from #172 measurements.
Rejected alternatives
Per-module closure scopes wired by upvalues — the road that led to #19; transitive pass-through accumulation makes the explosion structural. Globals/setfenv — GETGLOBAL in 5.1 is the same hash lookup, on _G. Integer-indexed storage (B[42]) — still a guarded load in traces, and unreadable output.
ADR
This changes the codegen structure, so the decision gets an ADR — docs/adr/0001-top-level-binding-storage.md, the first in the repo, establishing the practice. It records the full history (locals, then M, then two-tier), the budget model, and the rejected alternatives above.
Prerequisites / Relations
Independent in the dependency graph — pure codegen, no hard prerequisite. It builds on the #19 history above (the constraint the design must respect). The K threshold for large programs is chosen from #172's measurements, and #172's M.field-vs-local microbenchmark is the tool that scores the win. The multiplier grows once uncurrying (#24) and loopification (#181) land: real loops then exist and a cached local becomes a loop-hoisted register in a LuaJIT trace.
Verification / Measurement
Modest immediately: 2.4x on multi-use function bodies and module init (the #172 M.field-vs-local microbenchmark plus macro timings). The headline structural observable: a program that fits the budgets — Data.Array, 124 bindings — emits no M table at all, only chunk locals with the export table referencing them directly. Structural goldens churn mechanically (PSLUA_GOLDEN_ACCEPT + diff review); as a pure-codegen change, eval goldens must not move.
Problem
Every top-level binding lives in the
Mtable, so every inter-binding reference is a hash lookup: the linkedData.Arrayoutput has 681M.*references across 124 bindings. Measured: readingM.fieldin a loop is 2.4x slower than a local under PUC Lua 5.1; in a LuaJIT trace it is a guarded load where a local is a register.History (the constraint that shaped the current design): early codegen emitted top-level bindings as chunk locals and real programs hit Lua 5.1 limits — 200 locals per chunk, and the 60-upvalue limit amplified by 5.1's pass-through accumulation, where a nested function reading an outer local costs an upvalue slot in every intermediate function (#19). The
Mtable was the fix: unbounded fields, andMitself is the single upvalue everywhere. Any revisit must handle the limits by construction, not assume programs stay small.Approach
Budget-aware emission with
Mas the overflow valve (the design principle). Every budget overflow degrades the specific reference or binding back to today'sMform, so the worst case is exactly the current output and the limits stop being correctness cliffs.Stage 1: per-function caching of M fields (limit-proof by construction)
Storage untouched:
Mremains the single source of truth and the single upvalue, so neither limit can be approached. ALua.Optimizerpass cachesM.xfields used two or more times in a generated function body into locals at function entry (capped at ~30 per function, prioritized by use count):Semantics: the read moves to function entry — by the time any generated function is called, module init has assigned the bindings it reads, and
Mis not mutated within a call; the runtime-lazy scheme is unaffected. This is the classic "localize globals" optimization from LuaJIT guides.Stage 2: two-tier storage with budget accounting
Top-K bindings by static reference count are emitted as real chunk locals; the tail stays in
M; exported bindings are mirrored into the export surface. Two budgets are computed over the finished Lua AST before printing:M.upvals(f) = |own outer-local references ∪ children's pass-through demands|. When a function proto would exceed ~55, individual references are demoted — printed asM.x— while the binding stays a local for everyone else.A program that fits the budgets (like
Data.Arraywith 124 bindings) loses theMtable entirely: pure locals, with the module export table referencing them directly.Kfor larger programs is chosen from #172 measurements.Rejected alternatives
Per-module closure scopes wired by upvalues — the road that led to #19; transitive pass-through accumulation makes the explosion structural. Globals/
setfenv—GETGLOBALin 5.1 is the same hash lookup, on_G. Integer-indexed storage (B[42]) — still a guarded load in traces, and unreadable output.ADR
This changes the codegen structure, so the decision gets an ADR —
docs/adr/0001-top-level-binding-storage.md, the first in the repo, establishing the practice. It records the full history (locals, thenM, then two-tier), the budget model, and the rejected alternatives above.Prerequisites / Relations
Independent in the dependency graph — pure codegen, no hard prerequisite. It builds on the #19 history above (the constraint the design must respect). The
Kthreshold for large programs is chosen from #172's measurements, and #172'sM.field-vs-local microbenchmark is the tool that scores the win. The multiplier grows once uncurrying (#24) and loopification (#181) land: real loops then exist and a cached local becomes a loop-hoisted register in a LuaJIT trace.Verification / Measurement
Modest immediately: 2.4x on multi-use function bodies and module init (the #172
M.field-vs-local microbenchmark plus macro timings). The headline structural observable: a program that fits the budgets —Data.Array, 124 bindings — emits noMtable at all, only chunk locals with the export table referencing them directly. Structural goldens churn mechanically (PSLUA_GOLDEN_ACCEPT+ diff review); as a pure-codegen change, eval goldens must not move.