Skip to content

Closing audit: prove the cumulative win of the optimisation series #187

Description

@Unisay

Problem

The optimisation and correctness series (#172-#186 plus the standing
correctness bugs and #24) is motivated by a set of one-off measurements taken
against a single commit. Each individual issue defends its own local win, but
nothing yet ties the series together into a single before/after statement about
the generated code as a whole. Without that, "we made the backend faster" stays
an anecdote spread across a dozen issues rather than one reproducible number.

This issue is the closing bracket of the series: it records the baseline as it
stands today so that, once the series lands, the same harness can be re-run and
the cumulative effect stated in one table. It is deliberately blocked by every
issue in the series, so it naturally sorts last and cannot be closed until the
work it audits is done.

Baseline (today)

Timings are from the generated-Lua performance analysis (PUC Lua 5.1.5,
os.clock) and from the harness motivation in #172 (LuaJIT 2.1). Each row
compares the current generated shape against the idiomatic / uncurried
equivalent.

Pattern Current generated Ideal Slowdown Runtime
fib(30), arithmetic via IIFE lambdas 0.457 s 0.055 s 8.3× PUC 5.1.5
>= via ordInt dictionary (5e6) 1.165 s 0.027 s 42.4× PUC 5.1.5
Monadic bind ×3 chain (1e6) 0.736 s 0.012 s 63.3× PUC 5.1.5
Curried application f(a)(b) (5e6) 0.339 s 0.079 s 4.3× PUC 5.1.5
M.field access vs local (1e7) 0.090 s 0.037 s 2.4× PUC 5.1.5
Maybe alloc + match, hash-part vs array-part (2e6) 0.158 s 0.113 s 1.4× PUC 5.1.5
Curried hot loop acc = add(acc)(i) (2e7) 0.91 s 0.017 s 54× LuaJIT 2.1
Data.Array.foldl driven by a curried step 0.99 s 0.012 s 85× LuaJIT 2.1

Two LuaJIT facts from #172 anchor the trace-compilation story: the curried hot
loop runs the same with the JIT on (0.91 s) and with -joff (0.90 s). The JIT
buys nothing because every partial application emits an FNEW (closure
creation), which is NYI, so the trace recording aborts and the loop start is
blacklisted. The uncurried loop compiles to a single trace. The same blacklist
hits the Data.Array.foldl FFI for loop because the trace has to inline the
curried callee.

FNEW census (to capture via the harness)

The deterministic LuaJIT counter from #172's approach is the CI-friendly
regression metric: the number of FNEW bytecodes in luajit -bl of a linked
artifact, split into main-chunk (runs once at init) vs function-body (runs per
call) occurrences. It is not in this baseline because the #172 harness does not
exist yet and luajit is not in the dev shell. Capturing the Data_Array.lua
FNEW split is part of running the harness for the before snapshot.

Approach

  1. Land Benchmark harness: PUC Lua timings and LuaJIT trace/FNEW counters for generated code #172 (the benchmark harness); it is a prerequisite of this audit.
  2. Once the series is closed, re-run the Benchmark harness: PUC Lua timings and LuaJIT trace/FNEW counters for generated code #172 harness over the same linked
    artifacts and micro/macro patterns.
  3. Fill the "after" column of the table above and the FNEW census, and state
    the cumulative multiplier per pattern.

No code change ships from this issue; its deliverable is the before/after
table and the FNEW deltas.

Verification / Measurement

The filled-in table above is the verification. The pass criterion is that the
hot patterns move from the double-digit slowdowns recorded here toward small
single-digit factors over idiomatic Lua, and that the function-body FNEW count
in Data_Array.lua drops materially. The exact target numbers are set when the
harness produces the "after" snapshot.

Prerequisites / Relations

Blocked by the whole series: every correctness bug and performance
optimisation whose effect this audit measures. Feature issues (#27, #72) are
out of scope and are not blockers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: toolchainnix / spago / purs / buildchoreMaintenance / infrastructure

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions