Repository navigation
test: cargo-fuzz harness for the IQL parser and /ws frames; refuse non-finite float literals (#301, #397) - #405
Merged
Merged
Conversation
Three targets run the server's own code through a test-support seam (src/fuzzing.rs) on an engine-sized stack at the nesting ceiling: iql_statement, iql_program (split, parse, bind params, authorize) and ws_client_frame (pre-auth decode, id probe, request classification). Beyond crashes they check that a persistent rule reloads unchanged and that a request run read-only holds only queries. fuzz/seeds.py seeds them from the example programs, every /ws frame type and the #295 deep-nesting and oversized-body cases; fuzz/run.sh (make fuzz) runs a campaign that keeps going past findings.
… docs make doc-check failed on main since #390: rustdoc denies a public doc linking to a private item.
A persistent rule holding 1e400 (or inf, NaN) was stored with a JSON null, which failed to decode on the next start: WalUnreadable, and the engine would not start. Every float literal now goes through float_literal: decimal syntax only, and a literal beyond f64's range is a validation error. Rust's inf/NaN spellings are identifiers again. Negating i64::MIN in a literal no longer overflows. Found by the iql_statement fuzz target; its inputs are kept in fuzz/regressions/.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #301. Closes #397.
What
Fuzz harness (#301).
fuzz/is a cargo-fuzz crate with its own workspace, likeperf-gate. It has three targets. Each runs the server's own code through atest-supportseam (src/fuzzing.rs), on a thread with the server's 64 MiB engine stack, at the highest nesting limit an operator may configure (1024).iql_statementparse_statementvalidate_ruleand reloads unchanged from its catalog/WAL JSON (nested_json)iql_programexecuteprogram, optionally\0+paramsJSON$parambinding andauthorize_statementfor every role, as the handler does; a program the connection runs read-only (is_query_program) holds only queries; the rule-program parserws_client_frame/wstext frameClientFramedecode agrees withprobe_request_id; a decoded frame re-encodes to itself;Request::from_text's reply echoes the frame's id; an immediate reply encodes; a request classifiedAccess::Sharedholds only queriesfuzz/seeds.pygenerates them; they are not committed. They cover all 1,170 example programs and their statements, one frame of every client type (withparamsandexpect_*), the fix(parser): reject over-deep and oversized input instead of overflowing the stack #295 cases, and past findings fromfuzz/regressions/. The fix(parser): reject over-deep and oversized input instead of overflowing the stack #295 cases are nested calls, groups, operator chains,list[...]and records in every statement form at depths 1, 127-129, 1023-1025 and 4,000, plus bodies of 4,095-4,097 elements.iql.dictandws_frame.dictare libFuzzer dictionaries.fuzz/run.sh SECONDS [TARGET...](make fuzz) builds with ASan and debug assertions, then runs the targets in parallel withFUZZ_FORKSworkers each. Workers keep going past a crash, timeout (10 s) or OOM (2 GiB) and list every finding at the end. The script uses nightly when installed, otherwise stable withRUSTC_BOOTSTRAP=1.fuzz/README.md, plus a Fuzzing section and amake fuzzrow in TESTING.md.make fmtandmake fmt-checknow coverfuzz/.P0 found and fixed (#397). In the first local campaign,
iql_statementfound that a float literal beyond f64's range (1e400), or Rust'sinf/NaNspellings, parses into a non-finite float. JSON has no form for those, so a persistent rule holding one was written to the WAL asnull. On the next startStorageEngine::newfailed withWalUnreadableand the engine did not start. Any editor could cause this with one statement. All float literals inparser,statement::parser(terms, negatives, vectors, arithmetic) and type refinements now go through one function,parser::float_literal. It accepts decimal syntax only and refuses an out-of-range literal withvalidation. While there, negatingi64::MINin a literal no longer overflows (checked_neg). The four crash inputs are kept infuzz/regressions/iql_statement/.tests/float_literal_tests.rsreproduces the restart failure on main and passes here. The syntax reference says float literals must be finite (docs bundle regenerated). The SDKs already refuse non-finite floats before writing IQL.Also:
make doc-checkwas red on main. Since #390, a public doc instorage_engine/snapshot.rslinks the privaterevisionsmodule. This PR unlinks it.Campaigns
iql_statement: 4 crashes, all #397 (907k execs, cov 6227).iql_program: 590k execs, cov 6740, none.ws_client_frame: 730k execs, cov 5918, none.Local checks (GitHub CI is paused, #396)
Run on the devbox under
MemoryMax=16G,CARGO_BUILD_JOBS=8, after rebasing on d5df1ba:make fmt-check(engine, perf-gate, fuzz)make lint(clippy, all targets, all features,-D warnings), plus clippy on the fuzz cratemake doc-checkmake test-budget:make unit-test(every workspace test, including scenarios and oracle)make test-budget:run_snapshot_tests.sh --debug --skip-buildmake test-budget: budget checkcargo fuzz build -O+ replay of the 4 crash inputs