Repository navigation
Free-threaded (no-GIL): dumps() crashes when another thread mutates the dict being serialized (PyDict_Next iterator invalidation) #234
Description
Ahead of the free-threading work in #225: dumps() walks the object being
serialized with PyDict_Next, which is documented as unsafe against concurrent
mutation. On a free-threaded interpreter, another thread structurally mutating that
dict while dumps() is walking it corrupts the walk and crashes.
Latent today, not live. The extension does not declare Py_MOD_GIL_NOT_USED,
so importing it on a free-threaded build re-enables the GIL and this is safe as
shipped. It becomes reachable when the GIL is forced off (PYTHON_GIL=0) — i.e.
the moment free-threading support (#225) lands. So this is a concrete, reproducible
instance of what #225 needs to address, not a bug in current default behaviour.
Site (current master, rapidjson.cpp)
static inline bool all_keys_are_string(PyObject* dict) { // 2323
Py_ssize_t pos = 0; PyObject* key;
while (PyDict_Next(dict, &pos, &key, NULL)) // walks the user dict, no copy, no lock
if (!PyUnicode_Check(key)) return false;dumps(d) walks d here and again in dumps_internal (2556 / 2596). pos indexes
the dict's internal table; a structural mutation from another thread (an insert that
triggers a resize, or a delete) reallocates or reorders that table, so pos then
points into freed / relocated memory.
Reproducer
python3.14.0rc1t, python-rapidjson 1.23 built from source, PYTHON_GIL=0. 6
threads call dumps(shared) while 3 threads structurally mutate shared. 10
rounds per arm:
| arm | result |
|---|---|
mutate, GIL off (PYTHON_GIL=0) |
10/10 SIGSEGV |
| control — no mutator thread | clean 10/10 |
control — mutate a different dict dumps() never sees |
clean 10/10 |
| control — same mutation, GIL left on | clean 10/10 |
The GIL-on control staying clean is the whole point: this is a free-threading
fault, not a generic race, and it is safe under the GIL that ships today.
ft_rapidjson.py
"""python-rapidjson 1.23 — iterator-invalidation in dumps() over a shared dict.
`rapidjson.cpp:2320` all_keys_are_string() and dumps_internal() (2553/2593) walk
the user dict with `PyDict_Next`, no copy and no lock:
while (PyDict_Next(dict, &pos, &key, NULL)) // 2320
if (!PyUnicode_Check(key)) return false;
`PyDict_Next` is documented unsafe against concurrent mutation: `pos` indexes the
dict's internal table, and a structural mutation (insert that triggers a resize,
or a delete) reallocates or reorders that table, so a `pos` captured before the
mutation now points into freed/relocated memory.
Under the GIL, `dumps()` never yields mid-walk, so it is safe. Under
`Py_GIL_DISABLED`, another thread structurally mutating the same dict while
`dumps()` walks it invalidates the iteration → crash or garbage read.
This is the iterator-invalidation class, not borrow-then-use: the corrupted thing
is the *walk's cursor over the container*, not a single borrowed item.
Run with PYTHON_GIL=0 (the module does not declare Py_MOD_GIL_NOT_USED, so import
re-enables the GIL otherwise).
"""
import os
import sys
import threading
import time
import rapidjson
N_DUMPERS = int(os.environ.get("N_DUMPERS", 6))
N_MUTATORS = int(os.environ.get("N_MUTATORS", 3))
SIZE = int(os.environ.get("SIZE", 64))
SECONDS = float(os.environ.get("SECONDS", 6))
MUTATE = os.environ.get("MUTATE", "1") == "1"
DECOY = os.environ.get("DECOY", "0") == "1"
def fresh(n):
return {f"k{i}": i for i in range(n)}
def main():
gil = getattr(sys, "_is_gil_enabled", lambda: True)()
print(f"py={sys.version.split()[0]} gil={gil} "
f"dumpers={N_DUMPERS} mutators={N_MUTATORS if MUTATE else 0} "
f"decoy={DECOY}", flush=True)
shared = fresh(SIZE)
decoy = fresh(SIZE)
stop = threading.Event()
barrier = threading.Barrier(N_DUMPERS + (N_MUTATORS if MUTATE else 0) + 1)
dumped = [0] * N_DUMPERS
def dumper(t):
barrier.wait()
n = 0
while not stop.is_set():
try:
rapidjson.dumps(shared)
except Exception:
pass # value/type errors from racing data are irrelevant
n += 1
dumped[t] = n
def mutator():
barrier.wait()
target = decoy if DECOY else shared
i = 0
while not stop.is_set():
# STRUCTURAL mutation: grow (insert → resize) and shrink (delete),
# which is what invalidates the PyDict_Next cursor.
target[f"x{i}"] = i
if i % 3 == 0 and len(target) > SIZE:
try:
target.pop(next(iter(target)))
except (KeyError, StopIteration, RuntimeError):
pass
i += 1
threads = [threading.Thread(target=dumper, args=(t,)) for t in range(N_DUMPERS)]
if MUTATE:
threads += [threading.Thread(target=mutator) for _ in range(N_MUTATORS)]
for th in threads:
th.start()
barrier.wait()
time.sleep(SECONDS)
stop.set()
for th in threads:
th.join()
print(f"clean: {sum(dumped)} dumps survived", flush=True)
return 0
if __name__ == "__main__":
raise SystemExit(main())A possible fix (when #225 is worked)
Take a critical section on the dict for the duration of the walk
(Py_BEGIN_CRITICAL_SECTION(dict) around the PyDict_Next loops), or snapshot the
keys/items before serializing. all_keys_are_string and the two dumps_internal
walks are the sites.
Not claimed
No severity — reachable only with the GIL disabled, which is not the default today.
No exploitability — the observed faults are consistent with the mechanism; I did not
build a primitive.
Activity
lelit commentedon Aug 6, 2026
Thank you.
The FT branch definitely requires some focused effort, something I cannot grant these days... so any help would be highly appreciated!
espressolee commentedon Aug 6, 2026
Happy to help — here is what I found looking at the
free-threadedbranch, with themeasurements behind it.
The branch makes this bug reachable by default
The branch's only change to
rapidjson.cppis the module declaration:That tells the interpreter not to re-enable the GIL, but the container walks are still
unlocked — so it converts this issue from latent into reachable as built. Measured on
python3.14.0rc1twith the module built from8798483, noPYTHON_GILoverride,10 runs per arm:
PYTHON_GIL=1On
masterthe same script needsPYTHON_GIL=0to crash. The GIL-on control stayingclean is what makes it a free-threading fault rather than a generic race.
A second site, same class, not previously reported
dumps_internalwalks lists the same way — size captured once, then the unchecked macro:A concurrent shrink of the list being dumped makes the loop read past the end. Same
4-arm result: 10/10 SIGSEGV on the branch with no override, all three controls clean.
So there are four walk sites to protect:
all_keys_are_string(2326), the two dictwalks in
dumps_internal(2559, 2599), and this list walk (2513–2518).What I checked and found not to be a problem
So the list above is not just the scary-looking greps:
static PyObject*globals (46–79) are assigned once inmodule_execand only read afterwards, which is safe under free-threading. They are asubinterpreter concern (
m_size = 0), i.e. Extension isolation and subinterpreters support #226, not this issue.PyHandlermembers (root,sharedKeys, thedecoder*fields) and thePy_SETREF/Py_CLEARon them: aPyHandleris per decode call, so those areper-call state, not shared between threads.
the captured size cannot go stale.
Suggested fix, and one ordering note
For each walk, either hold a critical section on the container:
or snapshot before walking (
PyDict_Items()/PySequence_Tuple()). StringZilla justtook the snapshot route for exactly this class of bug. A critical section is cheaper,
but note it is released around re-entrant calls, and the loop body here can re-enter
Python through a
default=callable — so for thedumps_internalwalks a snapshot isthe more robust choice, while
all_keys_are_string(which calls no Python) is fine witha critical section.
The one ordering I would avoid: landing
Py_MOD_GIL_NOT_USEDbefore the walks areprotected, since that is what removes the interpreter-level protection currently keeping
these latent.
Offer
If it is useful I can put up a PR against
free-threadedwith those four sites fixedplus a 3.14t CI job, and leave the
Py_MOD_GIL_NOT_USEDdecision to you as a separatecommit you can flip when you are satisfied. Reproducers for both crashes are small and
I can include them. Entirely your call on shape — happy to follow whatever you prefer,
or to just hand over the reproducers if you would rather do the fix yourself.
(Analysis done with AI assistance; the measurements are mine and reproducible.)