Skip to content

perf: several small builtin fast paths - #12

Draft
moreal wants to merge 1633 commits into
mainfrom
perf/quick-wins
Draft

moreal wants to merge 1633 commits into
mainfrom
perf/quick-wins

Conversation

@moreal

@moreal moreal commented Sep 2, 2026

Copy link
Copy Markdown
Owner

What changed

Five small, independent fast-path fixes to hot builtin code paths, picked from a
performance audit of 31678962e (section 7.16). Each is individually justified
and individually benchmarked below.

  1. math.floor/ceil/trunc (crates/stdlib/src/math.rs) — add exact
    float/int fast paths ahead of the __floor__/__ceil__/__trunc__
    dunder lookup. PyFloat and PyInt both define these dunders, so the
    has_attr check was always true and every call allocated a bound method
    and made a second call through it.
  2. str.translate (crates/vm/src/builtins/str.rs) — for a dict
    translation table (the common case, e.g. built by str.maketrans), use
    PyDict::get_item_opt (a non-raising lookup, still correct for __missing__
    on dict subclasses) instead of building and catching a KeyError exception
    for every unmapped character.
  3. bytes.join (crates/vm/src/bytes_inner.rs) — copy an exact
    bytes/bytearray element directly instead of always going through the
    buffer protocol (which allocates a dim_desc plus a to_vec copy per
    element).
  4. str() (crates/vm/src/builtins/str.rs) — return the interned
    empty-string singleton (vm.ctx.empty_str) instead of allocating a fresh
    empty string, matching CPython's guarantee that str() is str().
  5. dict(pairs) / dict.update (crates/vm/src/builtins/dict.rs) — use
    vm.get_attribute_opt (a non-raising lookup) with the interned keys
    identifier instead of raising and catching an AttributeError to decide
    whether the argument is a mapping.

Why it's faster

All five avoid allocating and/or catching an exception object (or a bound
method) on a path that is taken on essentially every call. None change
observable behavior: math.trunc, bytes.join, and dict merge fall back to
the original generic path for non-exact types (subclasses, non-dict mapping
tables, etc.), and str.translate still calls table.get_item/catches
KeyError when the table isn't an exact dict.

Benchmark scripts

Under /tmp/choco-pi/perf/perf-quick-wins/ (one script per fix, plus a
neutral control workload unrelated to any of the five fixes):

  • bench_math_floor_ceil_trunc.py
  • bench_str_translate.py
  • bench_bytes_join.py
  • bench_str_empty.py
  • bench_dict_construct.py
  • bench_control_neutral.py (control — no fix touches this workload)
  • run_bench.py — interleaved A/B harness (baseline binary, patched binary,
    alternating, min-of-N and median over 11 reps each)

Benchmark results (interleaved A/B, 11 reps each, min/median wall time)

Baseline: unmodified 31678962e release binary. Patched: this branch's
release binary. Measured on a shared 10-core machine with other concurrent
builds running, hence interleaving + min-of-N; two independent full passes
gave consistent results (min-based speedups here are from the second,
post-cargo fmt pass):

Benchmark Baseline min / median Patched min / median Speedup (min / median)
bench_math_floor_ceil_trunc.py 697.82ms / 706.80ms 417.86ms / 433.22ms 1.67x / 1.63x
bench_str_translate.py 1240.81ms / 1251.62ms 198.52ms / 202.37ms 6.25x / 6.18x
bench_bytes_join.py 552.98ms / 555.73ms 240.65ms / 241.41ms 2.30x / 2.30x
bench_str_empty.py 514.91ms / 529.76ms 397.71ms / 408.40ms 1.29x / 1.30x
bench_dict_construct.py 248.25ms / 255.05ms 218.88ms / 220.86ms 1.13x / 1.15x
bench_control_neutral.py (no fix applies) 474.80ms / 478.64ms 475.87ms / 478.20ms 1.00x / 1.00x

The control workload (fib + sum(range(20)) loop, touching none of the five
changed paths) stays at 1.00x across runs, confirming the measured speedups
above aren't noise from the shared build machine. All five fixes reproduced a
clear, consistent improvement across two independent full A/B passes, so all
five are kept.

Test results

  • cargo clippy --workspace --all-targets --exclude rustpython_wasm --exclude rustpython-venvlauncher --exclude rustpython-capi — clean, no new warnings.
  • cargo test --workspace --exclude rustpython_wasm --exclude rustpython-venvlauncher --exclude rustpython-capi — all pass.
  • (cd crates/capi && cargo test) — 103 passed, 0 failed.
  • cargo run --release -- -m test test_math — 89 run, 5 skipped (pre-existing), SUCCESS.
  • cargo run --release -- -m test test_str — 138 run, 10 skipped (pre-existing), SUCCESS.
  • cargo run --release -- -m test test_bytes — 317 run, 13 skipped (pre-existing), SUCCESS.
  • cargo run --release -- -m test test_dict — 121 run, 10 skipped (pre-existing), SUCCESS.
  • cargo run --release -- -m test test_unicode — module doesn't exist in this
    Lib/test snapshot (pre-existing, unrelated to this change); test_str
    covers str.translate.
  • extra_tests: pytest -q — 443 passed, 1 pre-existing failure
    (test_rustpython_stdlib_sqlite, ModuleNotFoundError: No module named '_sqlite3') reproduced identically on the unmodified baseline binary, so
    it's unrelated to this change.

Caveats

  • Benchmarks were run on a shared machine with several other concurrent
    builds/agents, which adds noise; interleaving plus min-of-N and a
    neutral control workload (1.00x, unaffected) were used to keep the
    reported speedups honest.
  • test_unicode doesn't exist as a module in this Lib/test snapshot, so it
    couldn't be run as such; test_str (which includes str.translate
    coverage) was run instead and passes.

dependabot Bot and others added 30 commits July 21, 2026 01:36
Bumps [which](https://github.com/harryfei/which-rs) from 8.0.4 to 8.0.5.
- [Release notes](https://github.com/harryfei/which-rs/releases)
- [Changelog](https://github.com/harryfei/which-rs/blob/master/CHANGELOG.md)
- [Commits](harryfei/which-rs@8.0.4...8.0.5)

---
updated-dependencies:
- dependency-name: which
  dependency-version: 8.0.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.82.9 to 2.83.2.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](taiki-e/install-action@4684b84...43aecc8)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.83.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [bitflags](https://github.com/bitflags/bitflags) from 2.13.0 to 2.13.1.
- [Release notes](https://github.com/bitflags/bitflags/releases)
- [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md)
- [Commits](bitflags/bitflags@2.13.0...2.13.1)

---
updated-dependencies:
- dependency-name: bitflags
  dependency-version: 2.13.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.118 to 2.0.119.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](dtolnay/syn@2.0.118...2.0.119)

---
updated-dependencies:
- dependency-name: syn
  dependency-version: 2.0.119
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Add more unicode functions

Review

Add unicode decode functions

Add more functions

* Remove

* Add helper
* bytes: support keyword arguments in hex()

* bytes: unmark test_memoryview_hex_separator
…_ATTR specializations (RustPython#8350)

* Use dict keys-version stamps and entry-index hints in attr specializations

Add a keys-version stamp to Dict: a globally unique u32 assigned lazily
and reset on any key-set change (new key, deletion, clear). Value-only
updates keep the stamp.

- LoadAttrMethodWithValues / LoadAttrNondescriptorWithValues: cache the
  instance dict's stamp in the pointer cache and skip the shadow probe
  while the stamp matches.
- LoadAttrWithHint: cache the entry index at specialization time; a hit
  is an identity check on the entry key instead of a hash probe, with
  self-refresh on miss.
- StoreAttrWithHint / StoreAttrInstanceValue: replace values through the
  cached entry index via set_item_with_hint.

Fixes youknowone#42

Assisted-by: Claude

* Share keys-version stamps between dicts with identical layouts

Derive the keys-version stamp from a shape — the ordered interned-key
sequence of a hole-free dict — instead of always allocating a unique
stamp. Dicts with identical layouts (instances of the same class built
by the same __init__) now carry equal stamps, so a LOAD_ATTR cache
entry populated by one instance skips the shadow probe for every
instance sharing the layout.

Shapes are held in a fixed-size lock-free table keyed by interned key
addresses; dicts with holes, non-interned keys, or more than 32 keys
fall back to dict-unique stamps.

Assisted-by: Claude

* Address clippy and review feedback for keys-version stamps

- Use BuildHasher::hash_one for shape hashing (clippy manual_hash_one)
- Fix set_item_with_hint doc: a refreshed hint is returned on any hint
  miss, not only when the hinted slot was vacant
- Route both holey-dict instances through a single LOAD_ATTR cache site
  in the snippet test

Assisted-by: Claude

* Restore try_read_cached_descriptor doc comment to its function

The doc block and #[inline] were left attached to store_attr_dict_hinted
when it was inserted above try_read_cached_descriptor.

Assisted-by: Claude

* Specialize LoadAttrWithHint even when the entry index exceeds u16

hint_for_key returns None both for an absent key and for a present key
whose entry index does not fit in u16, so very large instance dicts
stopped specializing entirely. Use get_item_opt_refresh_hint to decide
presence, degrading an unrepresentable hint to 0: the handler then
keeps taking its full-probe fallback path.

Assisted-by: Claude
Part of RustPython#8245 to reduce the amount of work needed to review. I
simplified the embedded nul errors by forwarding to the implementations
in `vm::exceptions`.
* Keep StringIO text operations CPython-compatible

Constraint: email.feedparser depends on StringIO(newline='') recognizing CR, LF, and CRLF boundaries.
Rejected: an email-specific parser workaround | StringIO is the shared root cause.
Confidence: high
Scope-risk: moderate
Directive: Keep internal storage byte-based; convert only at StringIO API boundaries.
Tested: prek run --all-files; test_memoryio; test_email; full workspace verification by contributor
Assisted-by: Codex:gpt-5.6-sol

* Run restored StringIO conformance tests

Constraint: upstream StringIO tests already cover the repaired behavior.
Rejected: new RustPython-only regression tests | existing CPython coverage is sufficient.
Confidence: high
Scope-risk: narrow
Directive: Remove expected-failure markers only while the inherited tests pass.
Tested: prek run --all-files; test_memoryio; test_email
Assisted-by: Codex:gpt-5.6-sol

* Keep StringIO newline handling consistent

Constraint: StringIO must apply its newline mode consistently when constructing, writing, and reading text.\nRejected: a readline-only fix | it leaves CR and CRLF modes internally inconsistent.\nConfidence: high\nScope-risk: moderate\nDirective: Keep StringIO buffer contents valid WTF-8 before using unchecked views.\nTested: prek run --all-files; cargo clippy -p rustpython-vm -- -D warnings; test_memoryio; test_shlex; test_email; workspace excluding rustpython-capi; full workspace manually verified by contributor\nNot-tested: local rustpython-capi workspace test crashes with a pre-existing macOS SIGSEGV\nAssisted-by: Codex:gpt-5.6-sol
…thon#8362)

When a Connection is explicitly closed via con.close(), subsequent
operations (cursor(), commit(), rollback(), create_function(), etc.)
should raise ProgrammingError with 'Cannot operate on a closed database.'
to match CPython behaviour.

Previously, _db_lock() always returned 'Base Connection.__init__ not
called.' when self.db was None, without distinguishing between a
connection that was never initialised (subclass before __init__) and
one that was initialised and then explicitly closed.

Fix: inspect the initialized atomic flag — if True but db is None,
the connection was closed; if False, it was never initialised.

Assisted-by: GitHub Copilot:claude-sonnet-4-6
…stPython#8364)

Row(cursor, data) raised ValueError when cursor.description was None.
CPython allows this case and returns an empty key list.

- Use empty tuple when description is None instead of raising
- Include the key name in the IndexError when a string key is not found

Assisted-by: GitHub Copilot:claude-sonnet-4-6
* Fix AST identifier interning

Ensure parsed name and function identifier fields use interned strings,
matching CPython behavior.

Assisted-by: Codex:gpt-5.6-sol

* Apply PEP 8 import ordering
Raise OverflowError when a finite f64 value overflows while being converted for the struct f format. Remove the expected-failure marker from the corresponding regression test.

Assisted-by: Codex:gpt-5
* time: use C altzone when available

* Modified to also check TARGET_CC/CC_target.
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.2 to 3.1.4.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](fastify/fast-uri@v3.1.2...v3.1.4)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ustPython#8363)

CPython raises ProgrammingError with "Binding N is a named parameter"
when a query uses named placeholders (, , ) but
the caller passes a sequence instead of a mapping.

RustPython's bind_parameters_sequence() did not check whether each
binding slot is a named parameter, so no error was raised.

Fix: call sqlite3_bind_parameter_name() for each slot in
bind_parameters_sequence(). If the first byte of the returned name is
not '?' (i.e. it is a named placeholder), raise ProgrammingError before
attempting to bind.

Assisted-by: GitHub Copilot:claude-sonnet-4-6
…stPython#8367)

Bumps [postcss](https://github.com/postcss/postcss) from 8.5.10 to 8.5.23.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](postcss/postcss@8.5.10...8.5.23)

---
updated-dependencies:
- dependency-name: postcss
  dependency-version: 8.5.23
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#8368)

Bumps [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware) from 2.0.9 to 2.0.10.
- [Release notes](https://github.com/chimurai/http-proxy-middleware/releases)
- [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/v2.0.10/CHANGELOG.md)
- [Commits](chimurai/http-proxy-middleware@v2.0.9...v2.0.10)

---
updated-dependencies:
- dependency-name: http-proxy-middleware
  dependency-version: 2.0.10
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ustPython#8370)

Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.8.4 to 1.10.0.
- [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md)
- [Commits](ljharb/shell-quote@v1.8.4...v1.10.0)

---
updated-dependencies:
- dependency-name: shell-quote
  dependency-version: 1.10.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ustPython#8359)

* Replace malformed unicode error helper calls with the _real variants

new_unicode_decode_error / new_unicode_encode_error build the exception
from a bare message without running the initializer, so the result has
none of the five attributes a unicode error must carry and str() renders
as an empty string. Convert the non-Windows call sites in csv, socket,
getlogin and the fs-path decoders to the _real constructors, passing the
source bytes/str and the failing offset from the captured Utf8Error.

The Windows-gated sites (nt, mbcs/oem codecs) and the sites whose source
object is not reachable (uname, array) are left for follow-ups.

Assisted-by: Claude Code:claude-fable-5

* Report the full invalid UTF-8 span in the converted decode errors

The conversions used valid_up_to() + 1 for the decode-error end offset,
which under-reports multi-byte invalid sequences. Take the span from the
Utf8Error instead: valid_up_to() + error_len(), or the input length for a
truncated sequence (error_len() == None), matching CPython.

Assisted-by: Claude Code:claude-opus-4-8
* Implement termios.tcgetwinsize/tcsetwinsize

Adds ioctl(TIOCGWINSZ/TIOCSWINSZ) wrappers in host_env and the
corresponding Python-facing functions in the termios stdlib module,
removing the associated expectedFailure markers in test_termios.py.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT

* Use ret < 0 convention for ioctl error checks in termios winsize

POSIX ioctl only guarantees -1 on failure, not exactly 0 on success;
matches the existing check_libc_neg convention used elsewhere in
host_env (e.g. fcntl.rs, posix.rs::get_terminal_size).

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT

* Reject non-sequence size in tcsetwinsize

extract_elements_with also accepted dicts (treating keys as elements).
Switch to try_sequence(), matching CPython's PySequence_Check, so only
real sequences (tuple/list/etc.) are accepted.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT

* Use `core::mem::zeroed` instead of `std::mem::zeroed`

`std::mem` just re-exports `core::mem`, but clippy prefers importing
from core when there's no OS dependency. This was breaking the wasm CI build.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT

* Use try_index for tcsetwinsize row/col conversion

CPython's PyLong_AsLong calls __index__ on non-int objects before
converting, so downcast_ref::<PyInt> was stricter than CPython.
try_index matches that behavior and is more concise.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT

* Mark test_fork/test_spawn_doesnt_hang as expected failures

Both were hidden by a skipIf on tty.tcgetwinsize, which now exists.
Root cause: pty.fork() calls os.login_tty(), which isn't implemented,
so the forked child crashes before the test body runs.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT

* Use unittest.skip instead of expectedFailure for pty.fork() tests

expectedFailure still runs the test body, so pty.fork()'s real fork()
still happens and the child crashes on missing os.login_tty inside the
parallel test runner's worker process, corrupting its JSON reporting
channel ("worker bug"). skip prevents the method from running at all,
avoiding that. Verified locally with -j 2.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT

* Clarify skip reason for pty.fork() tests

* Simplify tcgetwinsize/tcsetwinsize fd argument with Fildes destructuring

* Explain why pty.fork() tests use skip instead of expectedFailure

---------

Co-authored-by: Claude Sonnet 5 <[email protected]>
Support the CPython 3.13+ format spec syntax that puts a grouping option
after the precision, so `format(1234.56789, '.6,f')` gives
'1234.567,890'. The integer and fractional parts group independently and
may use different separators (`,.6_f` -> '1,234.567_890'), so the spec
carries a separate `frac_grouping_option`.

`parse_precision` now consumes the separator that follows the precision
digits and rejects a `,`/`_` mix. A repeated separator is deliberately
left in the spec so the trailing-text check reports it, which is how
CPython arrives at a different message there. A dot followed by neither
digits nor a separator now raises "Format specifier missing precision"
rather than an unrelated error.

Fraction digits group away from the decimal point, so the last group may
be short, and any exponent or percent tail is left intact. The
separators count toward the field width, so zero padding of the integer
part reserves room for them.

'n' takes its separators from the locale and so cannot carry one. The
complex locale path rewrites 'n' to 'g' before delegating, so it has to
validate first; that also makes `format(1+2j, ',n')` fail as it does in
CPython, which it previously did not.

Reference (CPython 3.14), Python/formatter_unicode.c:
- parsing: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L257-L299
- 'n' rejection: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L361-L367
- number split: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L488-L516
- zero-pad width: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L604-L606

Remove `@unittest.expectedFailure` from tests that now pass
(2 in test_format, 1 in test_float).

Assisted-by: Claude:claude-opus-5

Co-authored-by: Claude Opus 5 <[email protected]>
codspeed Bot and others added 24 commits August 30, 2026 15:18
Alias criterion to codspeed-criterion-compat so the existing benchmark suites can be measured by CodSpeed with CPU simulation on every pull request, and add the workflow running them.

Co-authored-by: CodSpeed Bot <[email protected]>
)

* uuid: expose stable hardware node

Cache native MAC address detection and advertise an extractable node only when a hardware address is available. This lets uuid.getnode() use the native UUID path while preserving its existing fallbacks.

Assisted-by: Codex:gpt-5.6-sol

* Prefer universally administered MAC addresses

Match uuid.getnode()'s documented interface-selection rule by scanning available addresses before falling back to a locally administered one. Ignore zero addresses and cover the selection behavior with Rust unit tests.

Assisted-by: Codex:gpt-5.6-sol
* Implement PEP 667 frame locals semantics

Assisted-by: Codex:GPT-5

* Fix C API test CString import

Assisted-by: Codex:GPT-5

* Address PEP 667 review findings

Assisted-by: Codex:GPT-5
* Fix tuple subclass preservation in type bases

Assisted-by: Codex:gpt-5.6-sol

* Refine type bases representation

Store validated base classes in the original typed tuple, while retaining a bootstrap vector only until a Python tuple can be materialized.

Assisted-by: gpt-5.6-sol
Assisted-by: Codex:gpt-5.6-sol

* Bootstrap tuple type with type hierarchy

Initialize tuple and the canonical empty tuple alongside type and object so every type stores an actual typed bases tuple from construction. Remove the bootstrap bases representation and keep raw initialization Miri-safe.

Assisted-by: Codex:gpt-5.6-sol
…ite (RustPython#8621)

* Leave a byte that starts no UTF-8 sequence at the end of a console write

`find_last_utf8_boundary` read every byte at or above 0xc0 as the start of a
sequence, giving 0xf8..=0xff an expected length of 4 and cutting the tail back
to before it.  `_find_last_utf8_boundary` stops the four-byte range at 0xf8 and
returns the full length for anything above it, leaving the byte for
MultiByteToWideChar to answer with U+FFFD.

`write_console_utf8` therefore reported fewer bytes written than it was given
whenever one of those bytes fell in the last three, and for a buffer that began
with one it cut the length to 0, converted nothing and returned Ok(0) -- a
write that never advances.

The scan also ran a fourth iteration the C loop does not; the arms happen to
answer the same there, but the bound is now the same 3.

* Drop the expectedFailure on test_winconsoleio's test_write

The marker went stale with the boundary fix in the parent commit.  The test
writes `b'\xff'*10` and `data + b'\xff'` to CONOUT$ and asserts the returned
count, which is the case `find_last_utf8_boundary` was answering wrong, so
regrtest now reports it as an unexpected success.
…c__` (RustPython#8614)

* Strip the signature prefix from builtin __doc__

A function's signature and its documentation share one string, separated
by `\n--\n\n`, as in CPython. __text_signature__ read the front of it,
but __doc__ returned the whole thing:

    >>> len.__doc__
    'len(obj, /)\n--\n\nReturn the number of items in a container.'   # was
    'Return the number of items in a container.'                      # now

44 of the 46 builtin functions leaked the prefix.

get_doc_from_internal_doc already stripped it but was reachable only from
PyType.__doc__. Split it the way CPython does, into doc_without_signature
(_PyType_DocWithoutSignature) and a wrapper mapping an empty result to
None (_PyType_GetDocFromInternalDoc), and call it from PyNativeFunction
and PyMethodDescriptor as well.

Assisted-by: Claude Code:claude-opus-5

* Terminate a generated __text_signature__ that carries no documentation

A signature and its documentation share one string, separated by
`\n--\n\n`, and readers find the signature by searching for that marker.
A #[pyfunction] with no doc comment stored the bare signature, which no
reader could parse:

    >>> time.clock_getres.__doc__
    'clock_getres(clk_id, /)'
    >>> time.clock_getres.__text_signature__
    None

Emit the marker in that case too, as Argument Clinic does for an
undocumented function. 136 of the 915 functions reachable from the
importable modules were affected.

__doc__ now reports None for them, matching CPython, because nothing
follows the marker.

Assisted-by: Claude Code:claude-opus-5

* Report a __text_signature__ for every method

A #[pymethod] built its doc only when the method carried a doc comment,
so one without a comment dropped the generated signature too:

    >>> list.append.__text_signature__
    None

Build it from either part, as #[pyfunction] does. All 1101 method
descriptors reachable from the builtin types now report a signature, up
from 1, and 713 of the 1030 that CPython also describes match it exactly.

A method that takes its receiver as an ordinary first argument instead of
`&self` reported that argument by name, so binding the method could not
drop it:

    >>> inspect.signature([].__dir__)
    (obj, /)     # was
    ()           # now

Mark it $self, or $type for a classmethod, as CPython does. 310 methods
were affected.

test_unbound_builtin_method_noargs and test_bound_builtin_method_noargs
pass now.

Assisted-by: Claude Code:claude-opus-5

* Drop the expectedFailure on test_simple_completion

rlcompleter closes the parenthesis for a callable whose signature takes
no parameters, and leaves it open when the signature cannot be read:

    def _callable_postfix(self, val, word):
        if callable(val):
            word += "("
            try:
                if not inspect.signature(val).parameters:
                    word += ")"
            except ValueError:
                pass

os.getpid reports `()` since the generated signature carries the `--`
terminator, so completing `os.getpid` yields `os.getpid()` and the test
passes.

Assisted-by: Claude Code:claude-opus-5

* Name every #[pymethod] receiver zelf

A method that needs an owned handle cannot take `&self`, so it takes the
receiver as its first argument and names it `zelf`, `self` being a Rust
keyword. 24 of the 163 such methods named it something else: `instance`
(11), `obj` (4), `exc` (3), `_self` (3), `_instance` (2).

Rename them. The generated signature is unaffected either way, since the
macro marks the receiver by position rather than by name.

Assisted-by: Claude Code:claude-opus-5

* Drop the unreachable zelf name check in func_sig

The receiver is now marked by position, so the name it carries no longer
matters, and no path reaches this check with an argument named zelf:

- a #[pymethod] or #[pyclassmethod] without a `&self` receiver has its
  first argument replaced by the marker before the name is read
- a `&self` receiver is handled as syn::FnArg::Receiver
- a #[pymethod(raw)] is passed to static_raw_func, whose PyNativeFn bound
  fixes its signature to (&VirtualMachine, FuncArgs)
- no #[pystaticmethod] or #[pyfunction] names an argument zelf
- #[pymember], #[pygetset] and #[pyslot] never reach func_sig

Every generated signature is byte for byte identical without it.

Assisted-by: Claude Code:claude-opus-5

* Keep the argument bundle when marking the receiver

A #[pyclassmethod] that takes FuncArgs receives the class inside the
bundle along with every other argument, but the marker was spent on the
bundle and the arguments went unreported:

    >>> object.__subclasshook__.__text_signature__
    '($type, /)'                    # was, claims it takes nothing
    '($type, *args, **kwargs)'      # now

Report both, the shape CPython uses for __new__. Two methods take their
arguments this way, __subclasshook__ on object and on type.

Assisted-by: Claude Code:claude-opus-5
* common: add PyPy CJK codec engines

Assisted-by: Codex:GPT-5

* common: mark CJK mapping tables generated

Assisted-by: Codex:GPT-5

* common: add reusable byte and text engines

Assisted-by: Codex:GPT-5

* common: record opt-in engine dependencies

Assisted-by: Codex:GPT-5

* stdlib: add `_multibytecodec` and the six `_codecs_*` modules

Wire the CJK codec engines in `rustpython-common` up to the VM. `_multibytecodec`
holds `MultibyteCodec`, `MultibyteIncrementalEncoder`, `MultibyteIncrementalDecoder`,
`MultibyteStreamReader` and `MultibyteStreamWriter`, the encode/decode drivers and
the error-handler protocol; each `_codecs_*` module exposes `getcodec` for the
encodings of its region.

`rustpython-stdlib` now enables the `cjk-codecs` feature of `rustpython-common`.

Add `cjk::decode_reset`, which the decoder `reset()` methods need. It differs from
building a fresh decoder state: ISO 2022 keeps its G1..G3 designations and the
escape-throughout flag, and resets only G0 and the shifted flag.

Drop the markers from the CJK codec tests and from the `test_codecs`, `test_io`,
`test_email`, `test_zipfile` and `test_traceback` cases that needed a CJK codec.
`test_codecs.BasicUnicodeTest.{test_basics,test_decoder_state}` stay expected
failures for an unrelated reason: `codecs.charmap_encode` rejects a `None` mapping.

Assisted-by: Claude Code:claude-opus-5

* stdlib: use the byte and text engines from `rustpython-common`

`binascii`, `_json` and `_socket`'s address converters now call the engines in
`rustpython-common` instead of carrying their own copies, and `rustpython-stdlib`
enables the `binascii`, `inet` and `json` features for them. That leaves
`base64` and `crc32fast` with no caller in `rustpython-stdlib`, and
`crates/stdlib/src/json/machinery.rs`, the last of the json_in_type-derived code,
goes away with its last caller.

The address converters change behaviour. `inet_aton` reads the lenient forms
(`127.1`, `16909060`, `0x7f.1`, `0177.0.0.1`), `inet_pton` rejects an octet with a
redundant leading zero and a group of five hex digits, `inet_ntop` writes the
IPv4-compatible form as `::c000:201`, and an address family with no converter
raises `EAFNOSUPPORT` instead of a bare `OSError`.

Give the JSON engine the closing-quote scan and the allocation-free `\uXXXX`
escape it needs to carry `_json`'s workload. Against the code it replaces,
`json.dumps` runs 2.3x faster with `ensure_ascii=True` and 1.7x without it, and
`json.loads` 1.2x.

`binascii.a2b_base64` now reports the messages `binascii.c` does: no
`error decoding base64: ` prefix, and the data-character count in parentheses.

Assisted-by: Claude Code:claude-opus-5

* test: enable the Windows cp932 codec case

The shared CJK engine now satisfies the Windows code-page adapter, so keeping this test marked as an expected failure turns its success into a CI failure.

Assisted-by: Codex:GPT-5
Bumps the system-configuration group with 1 update in the / directory: [system-configuration](https://github.com/mullvad/system-configuration-rs).


Updates `system-configuration` from 0.7.0 to 0.8.0
- [Changelog](https://github.com/mullvad/system-configuration-rs/blob/main/CHANGELOG.md)
- [Commits](mullvad/system-configuration-rs@v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: system-configuration
  dependency-version: 0.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: system-configuration
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ustPython#8626)

Bumps [github/gh-aw/actions/setup](https://github.com/github/gh-aw) from 0.86.2 to 0.87.3.
- [Release notes](https://github.com/github/gh-aw/releases)
- [Changelog](https://github.com/github/gh-aw/blob/main/CHANGELOG.md)
- [Commits](github/gh-aw@48e5fa3...466b8ad)

---
updated-dependencies:
- dependency-name: github/gh-aw/actions/setup
  dependency-version: 0.87.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ython#8625)

Bumps [https://github.com/astral-sh/ruff-pre-commit](https://github.com/astral-sh/ruff-pre-commit) from v0.16.3 to 0.16.4.
- [Release notes](https://github.com/astral-sh/ruff-pre-commit/releases)
- [Commits](astral-sh/ruff-pre-commit@v0.16.3...v0.16.4)

---
updated-dependencies:
- dependency-name: https://github.com/astral-sh/ruff-pre-commit
  dependency-version: 0.16.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…on#8630)

* Name the function a failed argument binding was binding for

A built-in call that binds badly reported neither the function nor, for a
method, the right numbers: `self` was prepended before binding, so it was
counted as both an expected parameter and a given argument. `[].append()`
read `expected at least 2 arguments, got 1`.

`Callee` carries what a message says about the function: the name, and
whether the leading argument fills the instance parameter. `PyNativeFn`
takes one, `PyNativeFunction` and `PyMethodDescriptor` supply it, and the
four method-descriptor call specializations in `frame.rs` do the same.
`bind_for` subtracts the instance from both numbers.

The messages now read as their counterparts do:

- `_PyArg_CheckPositional`: `find expected at least 1 argument, got 0`,
  with the singular when the count is one.
- `_PyArg_UnpackKeywords`: `split() got an unexpected keyword argument
  'bogus'`, and `cast() missing required argument 'format' (pos 1)` for a
  parameter a call may pass by name. A positional-only parameter is only
  ever counted, so it keeps the first form.

A binding that happens where the name isn't known leaves it off, the way
`_PyArg_Parser.fname` is NULL.

`test_find_etc_raise_correct_error_messages` now passes in both
`test_bytes` and `string_tests`.

Assisted-by: Claude

* Name the type whose constructor or initializer bound the arguments

`Constructor::slot_new` and `Initializer::slot_init` bound their arguments
without a name, and so did the slots that override them, so a constructor
that was called wrongly said only `expected 1 argument, got 0`.

Both defaults and every override now name the type the slot was written
for, not the subclass being constructed and not the module `tp_name`
carries: `float expected at most 1 argument, got 2`, and `deque expected
at most 2 arguments, got 3` for a `deque` subclass.

`slice()` and `range()` counted their own arguments and raised messages of
their own; they now raise the one `_PyArg_CheckPositional` raises, which
makes `test_range_constructor_error_messages` pass.

Assisted-by: Claude
Preserve logical memoryview descriptors across interpreter transfers, make channel-end release idempotent, and scope runtime-owned interpreter cleanup to the creating top-level runtime. Re-read owned interpreters during shutdown so finalizers cannot leave newly-created children behind.

Add regression coverage for sliced and cast memoryviews, repeated channel release, and multiple embedded runtimes.

Assisted-by: Codex:GPT-5
Related: RustPython#8599

My original parser was monolithic and inflexible. It worked as intended
for the derived data files, but anything more complicated required hacky
code. For example, the original parser always expected to build a vector
of values, yet sometimes we required other data structures such as
BTreeMaps.

I split up the parser into helper functions that are both cleaner and
more flexible. The actual parser should still yield the same results -
this doesn't "fix" anything yet. However, this is preliminary work for
fixing more of UCD since we will need a more flexible parser for fixes
like the linked PR.
Add dependency-free CPU-bound workloads adapted from pyperformance 1.14.0 to cover interpreter, standard-library serialization, regex, numeric, and garbage-collection paths under the existing Criterion and CodSpeed harnesses. Keep inputs deterministic and scale expensive cases for simulation while checking that benchmark execution does not leak interpreter state.

Assisted-by: Codex:gpt-5.6-sol
* common: share the zlib stream engine

Move the VM-independent zlib-rs stream owner and state machine into a feature-gated rustpython-common module. Make rustpython-stdlib adapt that engine instead of carrying its own flate2 implementation, including native stream copy/deepcopy, Z_BLOCK flushes, and wbits=0 decompression.

Assisted-by: Codex:GPT-5

* common: share the lzma stream engine

Move the VM-independent liblzma stream owner, filter properties, and compression state into a target-gated rustpython-common module. Keep rustpython-stdlib as the Python object and exception adapter, and remove its direct xz dependencies and lzma-only generic compressor machinery.

Assisted-by: Codex:GPT-5

* common: address compression engine reviews
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.22.1 to 0.23.1.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](marshallpierce/rust-base64@v0.22.1...v0.23.1)

---
updated-dependencies:
- dependency-name: base64
  dependency-version: 0.23.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [libsqlite3-sys](https://github.com/rusqlite/rusqlite) from 0.38.1 to 0.38.2.
- [Release notes](https://github.com/rusqlite/rusqlite/releases)
- [Changelog](https://github.com/rusqlite/rusqlite/blob/master/Changelog.md)
- [Commits](https://github.com/rusqlite/rusqlite/commits)

---
updated-dependencies:
- dependency-name: libsqlite3-sys
  dependency-version: 0.38.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…r themselves (RustPython#8634)

* Raise the shared argument-binding errors from the slots that count for themselves

Sites all over the tree checked their own argument counts and keywords and
wrote the message by hand, so they drifted: `attrgetter expected 1 argument,
got 0.` carried a full stop, `TypeVar() got unexpected keyword argument(s):
bogus` named neither the keyword the way its counterpart does nor the
function the way `typevar()` is named, and `_csv`'s dialect parser handed a
whole formatted sentence to `InvalidKeywordArgument`, which then wrapped it
in another one.

`Callee::arity_error` and `Callee::unexpected_keyword` are now public, and
these sites raise through them, so there is one place the wording lives.
`FrameLocalsProxy` counts its arguments before reading its keywords, and
`min`/`max` do too, which is the order `framelocalsproxy_new` and `min_max`
check in. `weakref.ref`, `GenericAlias`, `frozenset`, `attrgetter`,
`itemgetter`, `islice`, `start_new_thread`, `TextIOWrapper`, `AttributeError`,
`NameError`, `TypeVar`, `ParamSpec`, `TypeVarTuple` and `TypeAliasType` all
follow.

Of 25 calls measured against CPython 3.14.6, 21 now produce the identical
string. The four left over need what `Callee` cannot carry: `weakref.proxy`
is a function of its own there rather than a type, `frozenset` names the
subclass being constructed, and `TextIOWrapper` names its first parameter
and counts before converting.

Assisted-by: Claude

* Give the argument-binding errors the constructors the rest of them have

Raising one of these meant writing `Callee::of::<Self>(vm).arity_error(1..=3,
0, vm)` — the vm twice, and a type to know about before you could say what
you meant. Every other error in the tree is a `vm.new_*_error`.

So the message texts move out of `Callee` into functions the vm can call, and
the vm gets `new_arity_type_error` and `new_unexpected_keyword_type_error`
alongside `new_unsupported_bin_op_error` and the rest. A slot names itself
with the `NAME` its class already carries:

    return Err(vm.new_arity_type_error(Self::NAME, 1..=3, 0));
    return Err(vm.new_unexpected_keyword_type_error(Some("typevar"), &key));

The keyword one takes the name as an `Option` because the parser sometimes
has none of its own, which is what `_PyArg_Parser.fname` being NULL means and
what `_csv`'s dialect parser wants.

`bind_for` and `check_kwargs_empty_for` take anything a `Callee` converts
from, so the slots pass `Self::NAME` there too. `Callee` itself is left to the
binding machinery, which is the only place that needs to carry a name around
rather than use one.

No message changes: 22 of 26 calls still match CPython 3.14.6 exactly, and the
43-call set still stands at 23.

Assisted-by: Claude
- math floor/ceil/trunc: skip the __floor__/__ceil__/__trunc__ dunder
  lookup for exact float/int, avoiding a bound-method allocation
- str.translate: use a non-raising lookup instead of catching a
  KeyError exception for every unmapped character
- bytes.join: copy exact bytes/bytearray elements directly instead of
  going through the buffer protocol
- str(): return the interned empty-string singleton instead of
  allocating a fresh empty string
- dict merge (dict(pairs)/dict.update): use an interned "keys"
  identifier and a non-raising attribute lookup

Assisted-by: choco-pi:anthropic/claude-sonnet-4-5
Signed-off-by: Lee Dogeon <[email protected]>
@moreal moreal closed this Sep 3, 2026
@moreal moreal reopened this Sep 3, 2026
@codspeed

codspeed Bot commented Sep 3, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 15.75%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 65 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ gc_collect.py[rustpython] 157.3 ms 135.9 ms +15.75%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing perf/quick-wins (55fdc94) with main (287dcd9)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.