Fuzzing RustPython with fusil #8325

Activity

1. youknowone commented on Jul 19, 2026

   

   youknowone

   on Jul 19, 2026

   Member

   More actions

   Hello, Thank you for contacting. Could you try a pilot run to find 5-20 bugs? I'd like to check it will be worth or not.
   I believe fuzzer will find a lot of bugs. But if the reports are duplication of CPython unittests, which probably already has the case reported by your tool, that means we are not ready to seriously run fuzzer.
2. devdanzin commented on Jul 19, 2026

   

   devdanzin

   on Jul 19, 2026

   Last edited by devdanzin

   Author

   More actions

   The first run against RustPython 0.5.0, for about 50 minutes on 4 instances, turned up a batch of pure-Python programs that panic or segfault the interpreter where CPython raises a normal exception.

   Here's a sample. Each row is a complete reproducer — paste it into rustpython -c "…":

   reproducer	crash	CPython does
   import pwd; pwd.struct_passwd().pw_name	panic types/structseq.rs:311 — index out of bounds (a struct-sequence built with fewer elements than its named fields; the field getter indexes with no bounds check)	TypeError at construction
   import _md5; _md5.md5()	panic class.rs:87 — static type has not been initialized	returns an md5 object
   import _csv; _csv.reader([]).__next__()	panic stdlib/src/csv.rs:805 — unwrap() on None (unregistered dialect line terminator)	works
   import _typing; _typing._idfunc()	panic stdlib/_typing.rs:43 — index out of bounds (args[0], no arity check)	TypeError
   eval(chr(0xd800))	panic stdlib/builtins.rs:557 — PyStr contains surrogates (expect_str on a lone-surrogate string)	ValueError
   import re; M=type(re.match('a','a')); M.__new__(M)[0]	SIGSEGV in _sre Match::as_mapping — an uninitialized Match (via __new__) has garbage regs/string; the mapping-subscript path reads them with no init guard (group()/__repr__ do guard it)	TypeError

   That last one is memory-unsafety (a segfault, not a clean panic). Two more that take a few lines rather than one:

   • The most common panic (48 hits): stdlib/_thread.rs:977 RefCell already borrowed. cleanup_thread_local_data holds LOCAL_GUARDS.borrow_mut() across .clear(), which drops the guards; a dropped thread-local value whose __del__ re-enters _thread._local borrows LOCAL_GUARDS again → BorrowMutError. (Full reproducer + root cause in the catalog.)
   • A segfault class: native stack overflow from unbounded recursion — hashing/comparing a deep or cyclic object recurses on the native stack with no depth guard → SIGSEGV/SIGABRT, where CPython raises RecursionError.

   Full catalog — one report per crash, each with a root cause and a fix sketch:
   https://github.com/devdanzin/rustpython-findings
   An example of what one report looks like (the re.Match segfault):
   https://gist.github.com/devdanzin/6e8f53711ea6b9c311e2c8fa043b17ca

   Due to how the fuzzer works, it's expected that many more issues are found in the first hours of fuzzing than on the long run.

   If this is useful I'm happy to file these individually or as an umbrella issue, minimize the remaining segfaults, or run more fuzzing, whatever's most helpful.

   Found with fusil. Investigation, root causes, fixes, reproducers and draft created with help from Claude Code

   👍React with 👍1Reacted by Jeong, YunWon
3. devdanzin commented on Jul 19, 2026

   

   devdanzin

   on Jul 19, 2026

   Author

   More actions

   Regarding whether these findings are duplications of CPython unittests: 2 of 9 are directly covered by a CPython test; the other 7 exercise inputs/behaviors the suite doesn't — including one where RustPython passes the relevant CPython tests.

   finding	CPython does	CPython test that exercises it	covered?
   RUSTPY-0008 re.Match.__new__ segfault	re.Match() → TypeError: cannot create 're.Match' instances	test_re.py:3160 check_disallow_instantiation(re.Match) asserts it can't be instantiated. RustPython lets re.Match() construct → the assertion fails, and [...] on the instance segfaults.	YES — running test_re catches it.
   RUSTPY-0004 csv reader unwrap	reader iterates fine	test_csv.py iterates readers pervasively. In RustPython every reader iteration panics (csv.reader(['a,b']) panics via get_lineterminator).	YES — test_csv would crash immediately.
   RUSTPY-0002 structseq short-ctor field OOB	pwd.struct_passwd() → TypeError: missing required argument 'sequence'	No test constructs a too-short / no-arg struct-seq and then reads a field. test_structseq tests the evil-__getitem__ case (correct length) and copy/replace, not this.	no
   RUSTPY-0005 _typing._idfunc() arity OOB	TypeError (wrong arity)	test_typing never calls the internal _typing._idfunc with zero args.	no
   RUSTPY-0006 eval/compile surrogate source	UnicodeEncodeError	No test_compile/test_builtin test compiles/evals a lone-surrogate source string (surrogate tests there are for ascii()/repr(), not compile).	no
   RUSTPY-0001 _thread._local teardown re-entrancy	works	test_threading_local has test_derived_cycle_dealloc (a local subclass with a ref cycle) — adjacent, but not the "a stored value's __del__ re-enters _local while the owning thread runs cleanup" trigger.	no (adjacent only)
   RUSTPY-0003 "static type not initialized"	n/a — CPython has no such failure mode	It's a RustPython-internal type-init issue (tracked as #5210), not a CPython behavior; nothing in CPython's suite targets it.	no (RustPython-specific)
   RUSTPY-0007a recursion → native stack overflow	RecursionError	test_json/test_recursion (incl. test_highly_nested_objects_encoding/decoding, "doesn't segfault"), test_richcmp, test_isinstance test recursion. But RustPython PASSES these — it guards cyclic json.dumps (raises), deep-nested json (raises), recursive __eq__/hash/compare (RecursionError). The fleet's segfaults come from other, more specific object graphs.	no — RustPython passes the standard recursion tests; the fuzzer found triggers they don't cover. (Known class = #2796.)

   👍React with 👍1Reacted by Jeong, YunWon
4. YangSiJun528 commented on Jul 26, 2026

   

   YangSiJun528

   on Jul 26, 2026

   Last edited by YangSiJun528

   Contributor

   More actions

   Thank you for running this pilot. It found cases where RustPython crashed but CPython did not. Some of them led to fixes in #8358 and #8371, so I think this approach is useful.

   My concern is that it may be too early to run a lot of fuzzing while CPython compatibility are still improving. If some parts already behave like CPython in most cases, would it make sense to start fuzzing those parts?
5. devdanzin commented on Jul 26, 2026

   

   devdanzin

   on Jul 26, 2026

   Author

   More actions

   Thanks for the feedback!

   I can restrict the findings to hard crashes and/or panics, or to specific code areas worth fuzzing, but would have to be guided towards which areas make the most sense.

   Alternatively, since I totally jumped the gun and already ran a full fuzzing campaign against RustPython (about 20 crashes found), I could just show you what I have found and you pick and choose.

   Happy to proceed however you want me to.
6. YangSiJun528 commented on Jul 28, 2026

   

   YangSiJun528

   on Jul 28, 2026

   Contributor

   More actions

   @youknowone, do you think it makes sense to focus fuzzing on a few modules first? If so, are there any modules with good CPython compatibility that would be useful fuzzing targets?
   An AI-assisted check of the current CPython test results for RustPython suggested operator, math, random, array, and set, but I’m not sure whether they are suitable.
7. youknowone commented on Aug 4, 2026

   

   youknowone

   on Aug 4, 2026

   Member

   More actions

   @devdanzin Thank you for much for trying this! I reviewed the reports and it looks very useful. In general fuzzing is more useful for mature codebase. In that pov, I think threading/multiprocessing/gc is not ready to fuzzy.

   I also like the format "finding | CPython does | CPython test that exercises it | covered?", this is very useful.
   To leverage AI agents to automate to solve it, creating issue (or subtask) for each bug will be more convenient.
8. youknowone commented on Aug 14, 2026

   

   youknowone

   on Aug 14, 2026

   Member

   More actions

   Status on the catalog, from working through it in #8514 and #8518.

   One lead turned out to be live: RPYR-0008

   The record has it as "LEAD — not yet reproduced", with the monkeypatch left as a TODO. It does reproduce. The attribute new_invalid_state_error consults is asyncio.InvalidStateError — the lookup imports asyncio.exceptions at level 0, which hands back the package — so patching that name is what triggers it:

   import _asyncio, asyncio
   asyncio.InvalidStateError = lambda *args: 42
   _asyncio.Future(loop=object()).result()
   # thread 'main' panicked at crates/stdlib/src/_asyncio.rs:2753:
   # called `Result::unwrap()` on an `Err` value: [PyObject PyInt { value: 42 }]

   CPython is immune because it caches the class in module state at init and raises it with PyErr_SetString, so a later rebinding is never consulted.

   Fixed in #8518: the type is looked up the way get_cancelled_error_type looks its own up, and a lookup that does not produce an exception type falls back to RuntimeError, which is what the neighbouring arms of that function already did.

   Four records no longer reproduce on main

   id	current behaviour
   RUSTPY-0001 thread-local RefCell reentrancy	runs to completion — cleanup_thread_local_data takes the vector out of LOCAL_GUARDS before dropping it (_thread.rs:646), so a __del__ that registers a new guard cannot re-borrow
   RUSTPY-0009 repr(staticmethod(obj))	propagates ValueError: boom; the unwrap is gone
   RUSTPY-0011 repr(classmethod(obj))	same
   RUSTPY-0010 binascii.b2a_qp(b'\n')	returns; the scan is guarded with in_idx > 0 (binascii.rs:507)

   Each was re-run from the catalog's own repro.py against a build of current main. Nothing to fix; the records can be closed as fixed.

   One I could not reproduce: RUSTPY-0007 face 7c

   The object::core::PyInner face, reported in the selectors and asyncio_queues vehicles. The report marks it as needing a per-crash-dir gdb pass to separate it from 7a, and the crash dirs are not public, so this is only what could be tried from outside:

   • 7a's guards landed in Fix 33 reproduced fuzzing and static-review defects #8514 (hash-slot dispatch, genericalias.make_parameters). 18 deep-nesting shapes — gc.collect, copy.deepcopy, marshal, json, pickle, repr, hash, ==, and dropping 100k-deep list/tuple/dict/instance chains on the main thread and on a worker — all raise RecursionError or complete normally.
   • Both named vehicle surfaces, driven hard. selectors: fileno() implementations that lie (a different answer every call), raise, return out-of-range or non-int descriptors, close the descriptor they just handed out, or unregister the object from inside fileno(), with register/modify/unregister/get_key/select interleaved from four threads. asyncio queues: Queue/LifoQueue/PriorityQueue carrying items whose __lt__ mutates the collection, with _asyncio._register_task/_unregister_task/_enter_task/_leave_task/all_tasks/current_task fed non-task objects from three threads. No crash.
   • 10 object-core abuse shapes: resurrection in __del__ (plain and inside a cycle), weakref callbacks that resurrect, a __del__ that calls gc.collect(), __class__ and __bases__ reassignment across layouts, instances of a type that has been deleted, frame.clear() on a suspended frame, slot-wrapper descriptors called on foreign objects. No crash.

   If a backtrace or a crash dir for one of the 7c vehicles can be shared, I'll take another pass at it. For what it is worth, my guess is that it is the PyAtomicRef Debug type confusion fixed in #8514 — that one formatted an object header as payload bytes and segfaulted inside the object core, and the way to reach it was _asyncio._enter_task's error message, which fits an asyncio vehicle — but that is a guess, not a diagnosis.

   Everything else

   #8514 (merged) covers 34 records in 33 commits — RPYR-0013 and RPYR-0014 share one fix. #8518 covers RUSTPY-0019 (contextvars under threads), RUSTPY-0023 (concurrent generator resumption) and RPYR-0008, plus an itertools.tee collector leak that turned up beside them and is not in the catalog. That leaves 7c as the only record without a fix.

   — commented by Claude