Skip to content

Add canonical cross-interpreter serialization tests and fixture bridges - #9408

Merged
youknowone merged 2 commits into
RustPython:mainfrom
youknowdot:split/canonical-crossinterp-fixtures
Oct 28, 2026
Merged

youknowone merged 2 commits into
RustPython:mainfrom
youknowdot:split/canonical-crossinterp-fixtures

Conversation

@youknowdot

@youknowdot youknowdot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Split the canonical cross-interpreter serialization test cohort out of #8954:

  • Import the complete test_crossinterp module from CPython d220409a13b77de7a6c30abe16ea90db01528edc using scripts/update_lib migrate, without changed expectations or new markers.
  • Add the two _testinternalcapi capsule bridges consumed by that module, reusing the existing cross-interpreter serialization and validation implementation.
  • Keep pickle/marshal/script fixture modes in a private CrossinterpData wrapper instead of extending the production SharedValue representation or public API.

The supporting _code_definitions and _crossinterp_definitions modules already match the same CPython revision. No lifecycle, interpreter creation/destruction, C API exports, parser or vendored dependency changes are included.

This is test infrastructure and canonical coverage, not a new production cross-interpreter API. Static source inventory finds 102 discoverable test methods (93 definitions plus inherited tests); this is not an execution result.

CPython comparison and simplification

Compared directly with CPython 3.15.0 (d220409a), including Modules/_testinternalcapi.c and Python/crossinterp.c. Follow-up 90bb550c2d52a156627ae391ca8a48fdfdcafb14 consolidates six constructor entry points into one mode dispatcher (19 fewer Rust lines), retaining the distinction between generic marshal and code transport, capsule ownership checks, and pure-script validation order. The canonical tests are unchanged; the four adjacent exception-capture tests cover existing behavior rather than justify new bridge code.

Validation

  • Canonical module and supporting definitions: byte-for-byte comparisons passed.
  • Normal check-merge-conflict, Ruff format/check, redundant-test-patches and rustfmt hooks passed; prettier had no applicable files. git diff --check passed. Only cspell was omitted locally with explicit user authorization. Lint CI, including cspell, passed for simplified head 90bb550c2d52a156627ae391ca8a48fdfdcafb14.
  • Focused default VM library cargo check: passed on the simplified source.
  • Strict default VM library Clippy: fails at unchanged _ctypes/base.rs:2246, arc_with_non_send_sync on Arc<BufferWrite> in non-threading mode. The entire file matches main df0d1b74; a separate baseline Clippy run was not performed. No lint suppression or unrelated ctypes changes are included.
  • Focused threading VM library check and strict Clippy: passed on the simplified source; bound input closure verified before and after.
  • Local canonical runtime module, full interpreter build and aggregate repository test suites were not run. Linux, macOS and Windows CI passed their interpreter builds, snippets and canonical suites, with test_crossinterp passed explicitly recorded on all three for simplified head 90bb550c2d52a156627ae391ca8a48fdfdcafb14. macOS reached its final success after the test runner handled an environment-changed result. Lib dependency checks also passed. The complete 24-job correctness workflow and Lib dependency workflow passed for this exact simplified head. This PR remains a draft for review.

The focused checks are source/cache/toolchain-bound, offline and resource-bounded. Independent review covered capsule ownership, private fixture boundaries and script-validation ordering. Local runtime testing remains separate: the canonical module creates a temporary import fixture and reads supporting definition files, so it is not a read-only test.

Import the complete CPython test_crossinterp module and provide its capsule bridges using existing serialization and validation logic. Keep fixture modes private instead of expanding SharedValue or interpreter lifecycle APIs.

Assisted-by: OpenAI Codex: model version unavailable
@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] lib: cpython/Lib/xml
[ ] test: cpython/Lib/test/test_xml_etree.py (TODO: 27)
[ ] test: cpython/Lib/test/test_xml_etree_c.py
[ ] test: cpython/Lib/test/test_minidom.py (TODO: 4)
[x] test: cpython/Lib/test/test_pulldom.py (TODO: 3)
[ ] test: cpython/Lib/test/test_pyexpat.py (TODO: 49)
[ ] test: cpython/Lib/test/test_sax.py (TODO: 28)
[x] test: cpython/Lib/test/test_xml_dom_minicompat.py
[x] test: cpython/Lib/test/test_xml_dom_xmlbuilder.py

dependencies:

  • xml (native: _elementtree, _exceptions, _weakref, collections.abc, domreg, handler, pyexpat, sys, urllib.parse, urllib.request, utils, xml.dom, xml.dom.NodeFilter, xml.dom.expatbuilder, xml.dom.minicompat, xml.dom.minidom, xml.dom.xmlbuilder, xml.etree.ElementTree, xml.parsers, xml.sax, xml.sax._exceptions, xml.sax.expatreader, xml.sax.handler, xml.sax.saxutils, xmlreader)
    • io (native: _io, _thread, errno, msvcrt, sys)
    • codecs, collections, contextlib, copy, os, posixpath, re, warnings, weakref

dependent tests: (38 tests)

  • xml: test_doctest test_minidom test_pulldom test_pydoc test_pyexpat test_regrtest test_sax test_typing test_xml test_xml_dom_minicompat test_xml_dom_xmlbuilder test_xml_etree
    • plistlib: test_plistlib
      • platform: test__locale test__osx_support test_android test_asyncio test_baseexception test_builtin test_calendar test_cmath test_ctypes test_fcntl test_math test_mimetypes test_os test_platform test_shutil test_socket test_ssl test_strptime test_sysconfig test_time test_tkinter test_winreg test_wsgiref
      • webbrowser: test_webbrowser
    • xmlrpc.client: test_xmlrpc

[x] lib: cpython/Lib/concurrent
[x] test: cpython/Lib/test/test_concurrent_futures (TODO: 1)
[x] test: cpython/Lib/test/test_interpreters
[ ] test: cpython/Lib/test/test__interpreters.py
[ ] test: cpython/Lib/test/test__interpchannels.py
[x] test: cpython/Lib/test/test_crossinterp.py

dependencies:

  • concurrent

dependent tests: (19 tests)

  • concurrent: test_asyncio test_compileall test_concurrent_futures test_context test_external_inspection test_genericalias test_inspect test_interpreters test_struct test_sys test_threading test_types test_wmi
    • asyncio: test_asyncio test_logging test_os test_pdb test_unittest
      • pdb: test_remote_pdb

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

Match CPython 3.15.0 mode dispatch with one private constructor instead of six forwarding entry points. Keep generic marshal data distinct from code transport, preserve capsule ownership checks and pure-script validation, and leave the canonical tests unchanged.

Assisted-by: OpenAI Codex: model version unavailable
@codspeed

codspeed Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks1


Comparing youknowdot:split/canonical-crossinterp-fixtures (90bb550) with main (a463449)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

youknowdot added a commit to youknowdot/RustPython that referenced this pull request Oct 27, 2026
Reconcile the mega bridge with reviewed RustPython#9408 source at 90bb550.
Keep unrelated interpreter lifecycle, fixtures and canonical Lib unchanged.
This reduces production API exposure rather than total lines.

AI assistance: dot prepared the source reconciliation, consumer audit and
validation report. Exact model version is not exposed.

Assisted-by: dot: model version not exposed
@youknowone
youknowone marked this pull request as ready for review October 28, 2026 01:11
@youknowone
youknowone merged commit 058bebf into RustPython:main Oct 28, 2026
30 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-28T01:13:24.760659Z
We couldn’t complete this review. Comment @codex review to try again.
90bb550 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

youknowdot added a commit to youknowdot/RustPython that referenced this pull request Oct 28, 2026
Merge main 058bebf after RustPython#9408 landed. Keep the landed standalone
CrossinterpData import and remove its duplicate grouped import from the
combined tree. Preserve incoming main changes and prior mega cleanup.

AI assistance: dot diagnosed the synthetic-merge compile failure, prepared
and reviewed the reconciliation and validation report. Exact model
version is not exposed.

Assisted-by: dot: model version not exposed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants