Repository navigation
Optimize buffer usage in encode/encode_into - #37
Merged
Merged
Conversation
This applies a few buffer-oriented optimizations to speedup encoding (by up to a factor of 30x). - Move resizing of write buffer to a separate function, and inline `mp_write`. This allows gcc to do a better job of optimizing small memcpy operations with little code bloat. - Change buffer growth factor from 1.5x(ish) to 2x. This comes with a slight increase in memory usage, but significantly improves performance. - When returning a `bytes` object from `encode`, don't hit realloc to shrink the buffer down to the exact output size. Instead, we just shrink the view presented to Python. This requires a bit of mucking in the internals of the bytes object, but nothing too onerous. Since the output of `encode` will usually be immediately written to a socket/file and then dropped, this shouldn't result in an increase in application memory usage.
Member
Author
|
Benchmark: import gc
import timeit
import msgspec
def time(func, data):
gc.collect()
timer = timeit.Timer("func(data)", globals={"func": func, "data": data})
n, t = timer.autorange()
return t / n
data = b"x" * 10
msg = [data] * 100000
enc = msgspec.Encoder()
t = time(enc.encode, msg)
print(f"msgspec took {t * 100000} us per call")Master This PR |
sobolevn
added a commit
to Andarius/msgspec
that referenced
this pull request
Oct 12, 2026
`MS_CONSTR_STR_REGEX` slot clear was missing. It was not properly
traversed and freed. Now it is.
It used to fail with memory sanitized with:
```
==2637==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x50e000e8e350 in thread T0
#0 0x7f07a10fc4d8 in free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52
msgspec#1 0x7f079c42ddf4 in TypeNode_Free src/msgspec/_core.c:3474
msgspec#2 0x7f079c42dbdf in TypeNode_Free src/msgspec/_core.c:3468
msgspec#3 0x7f079c55e2eb in JSONDecoder_dealloc src/msgspec/_core.c:17235
msgspec#4 0x7f07a0999132 in _PyEval_EvalFrameDefault Python/bytecodes.c:1398
msgspec#5 0x7f07a0a15f73 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92
msgspec#6 0x7f07a0a15f73 in method_vectorcall Objects/classobject.c:69
msgspec#7 0x7f07a098d82f in _PyEval_EvalFrameDefault Python/bytecodes.c:3263
msgspec#8 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419
msgspec#9 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144
msgspec#10 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508
msgspec#11 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791
msgspec#12 0x7f07a0a14206 in _PyObject_MakeTpCall Objects/call.c:240
msgspec#13 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715
msgspec#14 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419
msgspec#15 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144
msgspec#16 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508
msgspec#17 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791
msgspec#18 0x7f07a0a1550e in PyObject_Call (/opt/hostedtoolcache/Python/3.12.15/x64/lib/libpython3.12.so.1.0+0x21550e) (BuildId: d5ce433e66c26342bfaf6b82ab1a95541a8ebade)
msgspec#19 0x7f07a098d82f in _PyEval_EvalFrameDefault Python/bytecodes.c:3263
msgspec#20 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419
msgspec#21 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144
msgspec#22 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508
msgspec#23 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791
msgspec#24 0x7f07a0a14206 in _PyObject_MakeTpCall Objects/call.c:240
msgspec#25 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715
msgspec#26 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419
msgspec#27 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144
msgspec#28 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508
msgspec#29 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791
msgspec#30 0x7f07a0a14206 in _PyObject_MakeTpCall Objects/call.c:240
msgspec#31 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715
msgspec#32 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419
msgspec#33 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144
msgspec#34 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508
msgspec#35 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791
msgspec#36 0x7f07a0a14206 in _PyObject_MakeTpCall Objects/call.c:240
msgspec#37 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715
msgspec#38 0x7f07a0ae8f35 in PyEval_EvalCode Python/ceval.c:580
msgspec#39 0x7f07a0ae7a3c in builtin_exec_impl Python/bltinmodule.c:1093
msgspec#40 0x7f07a0ae7a3c in builtin_exec Python/clinic/bltinmodule.c.h:543
msgspec#41 0x7f07a0a345a2 in cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:438
msgspec#42 0x7f07a0a14532 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92
msgspec#43 0x7f07a0a14532 in PyObject_Vectorcall Objects/call.c:325
msgspec#44 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715
msgspec#45 0x7f07a0ae8f35 in PyEval_EvalCode Python/ceval.c:580
msgspec#46 0x7f07a0b09f76 in run_eval_code_obj Python/pythonrun.c:1757
msgspec#47 0x7f07a0b09aaa in run_mod Python/pythonrun.c:1778
msgspec#48 0x7f07a0b099f1 in pyrun_file Python/pythonrun.c:1674
msgspec#49 0x7f07a0b09852 in _PyRun_SimpleFileObject Python/pythonrun.c:459
msgspec#50 0x7f07a0b09667 in _PyRun_AnyFileObject Python/pythonrun.c:78
msgspec#51 0x7f07a0b12943 in pymain_run_file_obj Modules/main.c:361
msgspec#52 0x7f07a0b12943 in pymain_run_file Modules/main.c:380
msgspec#53 0x7f07a0b12943 in pymain_run_python Modules/main.c:634
msgspec#54 0x7f07a0b12943 in Py_RunMain Modules/main.c:714
msgspec#55 0x7f07a0b1243a in Py_BytesMain Modules/main.c:768
msgspec#56 0x7f07a042a1c9 (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: a4a7992a8e66555c8141ab2a08a8465ff6e0ea65)
msgspec#57 0x7f07a042a28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: a4a7992a8e66555c8141ab2a08a8465ff6e0ea65)
msgspec#58 0x558976af7094 in _start (/opt/hostedtoolcache/Python/3.12.15/x64/bin/python3.12+0x1094) (BuildId: d664fd041bd9df9902c326eb80717fa04325aaaa)
0x50e000e8e350 is located 16 bytes inside of 152-byte region [0x50e000e8e340,0x50e000e8e3d8)
allocated by thread T0 here:
#0 0x7f07a10fd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
msgspec#1 0x7f07a0a7c2fc in gc_alloc Modules/gcmodule.c:2330
msgspec#2 0x7f07a0a7c2fc in _PyObject_GC_NewVar Modules/gcmodule.c:2364
SUMMARY: AddressSanitizer: bad-free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 in free
==2637==ABORTING
```
See
https://github.com/msgspec/msgspec/actions/runs/38243375356/job/114791328508
Now it is properly cleaned.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This applies a few buffer-oriented optimizations to speedup encoding (by
up to a factor of ~2x).
Move resizing of write buffer to a separate function, and inline
mp_write. This allows gcc to do a better job of optimizing smallmemcpy operations with little code bloat.
When returning a
bytesobject fromencode, don't hit realloc toshrink the buffer down to the exact output size. Instead, we just shrink
the view presented to Python. This requires a bit of mucking in the
internals of the bytes object, but nothing too onerous. Since the output
of
encodewill usually be immediately written to a socket/file andthen dropped, this shouldn't result in an increase in application memory
usage.