Repository navigation
fix(engine): bound request-supplied sizes so huge values can't abort the server - #287
Merged
Merged
Conversation
…ocation One query with a huge client-supplied count, such as lsh_probes(0, 4, 300000000000000000), made the engine ask the allocator for that many bytes; the allocation failed and the process aborted, before the per-query memory limit could act. hnsw_nearest's k and ef_search, top_k's K, and an index's ef_construction were the same class, and hnsw_rs exits the process outright for m over 256. Each size is now checked twice. The parser refuses a literal over its bound with a validation error that names the parameter and the bound (new size_limits module): top_k and top_k_threshold K at most 1,000,000, hnsw_nearest k and ef_search at most 10,000 (ef_search 0 is refused too), lsh_probes and lsh_multi_probe num_probes at most 39,774 (every distinct probe for 62 bits), and .index create ef_construction and ef_search at most 10,000. The evaluator bounds each allocation by what its input can produce, so a size bound at run time is safe as well: probe vectors by the distinct probe count, top-k heaps by the group size, the HNSW beam by the index size (k * 4 and the widening loop saturate), and saved index definitions are clamped when their graph is built. replace and concat return null rather than build a string over 16 MiB, the LSH functions return null for vectors over 16,384 dimensions (their cached hyperplanes are bits x dimension floats), and a client timeout too far out for Instant no longer panics.
This was referenced Oct 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
I want you to really hyper focus on making the symbolic engine, you know, crisp and really truly production ready. (Context: the adversarial audit found a P0: one query with a huge client-supplied size (e.g. lsh_probes = 3e17) makes the engine try to allocate that much memory and abort; hnsw_nearest ef_search and top_k have the same class.)
The audit's P0 item, by reference: on an empty knowledge graph with +a[(1)], lsh_probes with num_probes=4 returns [0,1,2,4], but num_probes=300000000000000000 closes the connection and kills the server process ("memory allocation of 2400000000000000000 bytes failed"). lsh_probes(bucket, num_hp, num_probes) calls Vec::with_capacity(num_probes) with the client's value; an allocation beyond the address space makes Rust abort, which cannot be caught. The optional ef_search argument of hnsw_nearest (parsed as any usize, reaching BinaryHeap::with_capacity in hnsw_rs once an index has over 1,024 rows) and top_k<K, ...> (BinaryHeap::with_capacity(k+1)) are the same class. Any principal that may run a query can do this, and the memory budgets cannot intercept it because the abort happens before any accounting. The audit's fix: clamp num_probes (to the number of distinct probes), ef_search and top_k's K at parse or evaluation time and reject larger values with validation, and audit every with_capacity whose argument comes from a value.
What Changed
src/size_limits.rswith upper bounds:top_k/top_k_thresholdK ≤ 1,000,000;hnsw_nearestk andef_search≤ 10,000;ef_construction≤ 10,000;lsh_probes/lsh_multi_probenum_probes≤ 39,774 (every distinct probe for 62 bits). The parser now rejects literals over these bounds, andhnsw_nearestef_search= 0, with avalidationerror that names the parameter and its bound. Index creation now rejectsef_construction/ef_searchoutside 1–10,000.efat the entry count. HNSWfetchdoubling now saturates instead of overflowing. Saved index definitions are clamped to safem/ef_constructionwhen the graph is rebuilt.concat/replacereturn null when the result would exceed 16 MiB. Request deadlines usechecked_add, so a huge timeout means no deadline instead of a panic.tests/size_limit_tests.rscovering the oversized values from the audit.Risk Assessment
✅ Low: The fix round removed the LSH dimension cap cleanly from the constant, the five evaluator checks and the docs. Every other clamp is still in place: num_probes, ef_search, top_k and hnsw k are rejected at parse time and capped at evaluation time, and saved HNSW parameters are clamped. No new defects were found in the changed code.
Testing
I built the server and client binaries and ran the real server on a free local port with a disposable data dir. I then sent IQL scripts over WebSocket with the real client: the audit's lsh_probes repro, runtime-bound probe counts, hnsw_nearest k and ef_search on a 1,100-row index (above the 1,024-row graph-walk threshold), index ef parameters, top_k and top_k_threshold, the oversized string builders, and 20,000-dimension LSH. Every live scenario behaved as intended, and the same server PID answered queries afterwards. The targeted size_limit_tests integration file also passes, but its fixed-seed fuzz runs against an in-process handler, so it is recorded as untested live. I did not rebuild the base commit to reproduce the original crash live. Everything I set up was torn down; only the evidence files remain.
cargo test --test size_limit_testsagainst an in-process Handler, not the running server binary. To dri…Evidence: Live client transcript: runtime probes, HNSW, index params, top_k
Evidence: Live client transcript: high-dim LSH and string builders
Evidence: Server log for the live run
Evidence: Audit P0 repro against live server
Evidence: size_limit_tests output
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 1 issue found → auto-fixed ✅
src/code_generator/mod.rs:2655- This adds a new rule: the LSH builtins (lsh_bucket, lsh_bucket_int8, lsh_bucket_with_distances, lsh_multi_probe, lsh_multi_probe_int8 at src/code_generator/mod.rs:2655, 2820, 2854, 2890, 2908) now return null for any vector over MAX_LSH_DIMENSION = 16,384 (src/size_limits.rs:38). The intent doesn't call for this. The hyperplane allocation isbits * dimensionwith bits capped at 62, so it is at most 62 times the size of a vector that is already in memory. It can't request an amount the address space can't hold, which is the abort the intent is about. Meanwhile a legitimate embedding over 16,384 dimensions now silently loses its LSH bucket or probes and drops out of LSH joins without any error. Recommended fix: remove the dimension check. If the concern is the size of the hyperplane cache, deal with that separately, since this cap doesn't bound the number of cache entries anyway.🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
cargo test --test size_limit_testsagainst an in-process Handler, not the running server binary. To dri…target/debug/inputlayer-server --host 127.0.0.1 --port 27493 --data-dir <tmp>+inputlayer-client --server http://127.0.0.1:27493 s1.iql(audit repro: lsh_probes 4 vs 3e17)inputlayer-client ... s2.iql: runtime num_probes from data, lsh_multi_probe literal, hnsw_nearest k/ef_search on a 1,100-row index, .index create ef_construction/ef_search, top_k / top_k_threshold over and at boundinputlayer-client ... s3.iql: 20,000-dim lsh_bucket / lsh_multi_probe, oversized replace/concat return null, normal replace still workspgrep after each script to confirm the server process survivedcargo test --test size_limit_tests(9 handler-level tests including the fixed-seed fuzz over every size position)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.