Skip to content

Sync canonical stdlib modules and offline stack formatting - #9315

Merged
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:sync-canonical-stdlib-final-20261022
Oct 23, 2026
Merged

youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:sync-canonical-stdlib-final-20261022

Conversation

@youknowdot

Copy link
Copy Markdown
Contributor

Summary

Split nine coherent standard-library files from #8954 and synchronize them with CPython v3.15.0 (d220409a13b77de7a6c30abe16ea90db01528edc) using scripts/update_lib. Existing RustPython adaptations are preserved and independently replayed against canonical sources.

  • Update asyncio's console fallback, environment handling, prompt formatting and --retries CLI options; synchronize hostname and shutdown-race regressions.
  • Synchronize marshal frozen-dictionary/reference/version regressions, macholib version deprecation, and color-theme copying coverage.
  • Add the canonical offline stack-dump formatter and its 17 mock-based tests. This formats supplied records only; it does not enable live process inspection or change the broader live sampling frontend/test-support helpers.

The existing native types/status constants needed by offline formatting are exported on main without Linux-only conditions. No new native capability or platform-skip adaptation is introduced.

Validation

Tested base: 0b0a3607648b5b01b3adfc1c06f3bec1d87f8a13.

  • All nine sources compile with CPython 3.15.0 and RustPython.
  • 21 exact canonical cases passed: 17 offline formatting, 2 memory-BIO hostname, 1 macholib deprecation and 1 color-theme case.
  • Four marshal cases passed in a separate bounded process, including the fixed-size immutable-reference regression inputs.
  • Three help-only asyncio invocations passed (--help, ps --help, pstree --help). No PID command or interactive session was invoked.
  • No skips, expected failures or unexpected successes in those selected tests.
  • Normal repository commit hooks passed. All migrated-file hashes remain unchanged after hooks.

The runtime used an existing sealed main-base binary with only the unrelated #9303 AST-Single warning change, not an exact-main binary. Selected compilation and cases do not enter that changed route; relevant native source bindings were verified. No new native build or full workspace/CAPI suite was run. Local results are Linux-only; macOS/Windows execution remains for CI.

The separately guarded canonical loopback TLS shutdown-race case also passed, with no skip or expected failure. It used only two owned 127.0.0.1 server instances and repository test certificates; the pass does not prove the early-return race branch was taken.

AI assistance: Codex was used for source comparison, official-updater preparation, bounded validation and this description.

Migrate nine substantial stdlib and canonical test modules from CPython
v3.15.0 through scripts/update_lib, preserving RustPython adaptations.
Include offline stack formatting without live process inspection or
changes to the held profiling frontend and test-support closure.

Assisted-by: Codex:unknown
@coderabbitai

coderabbitai Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] lib: cpython/Lib/profiling
[ ] test: cpython/Lib/test/test_profiling

dependencies:

  • profiling (native: main, _child_monitor, _css_utils, _format_utils, _lsprof, _remote_debugging, binary_collector, binary_reader, cli, collector, constants, display, dump, errors, gecko_collector, heatmap_collector, importlib.machinery, importlib.resources, importlib.util, itertools, jsonl_collector, live_collector, marshal, math, module_utils, opcode_utils, profiling.tracing, profiling.tracing._utils, pstats_collector, sample, stack_collector, string_table, sys, time, trend_tracker, widgets)
    • curses (native: _curses, _curses_panel, curses.ascii, has_key, sys)
    • io (native: _io, _thread, errno, msvcrt, sys)
    • opcode (native: _opcode, builtins)
    • platform (native: _wmi, itertools, sys, vms_lib, winreg)
    • site (native: _io, _pyrepl.main, _pyrepl.pager, _pyrepl.readline, _pyrepl.unix_console, _pyrepl.windows_console, atexit, builtins, errno, readline, sitecustomize, sys, usercustomize)
    • sysconfig (native: _sysconfig, _winapi, importlib.machinery, importlib.util, os.path, sys)
    • _colorize, abc, argparse, base64, collections, contextlib, dataclasses, functools, html, json, linecache, locale, optparse, os, pathlib, pstats, re, runpy, selectors, socket, statistics, subprocess, tempfile, threading, traceback, types, typing, uuid, webbrowser

dependent tests: (3 tests)

  • profiling: test_profiling
    • cProfile: test_cprofile test_pstats

[x] lib: cpython/Lib/ctypes
[ ] test: cpython/Lib/test/test_ctypes (TODO: 8)
[ ] test: cpython/Lib/test/test_stable_abi_ctypes.py

dependencies:

  • ctypes

dependent tests: (37 tests)

  • ctypes: test_android test_buffer test_bytes test_code test_codecs test_ctypes test_genericalias test_ntpath test_os test_ssl test_venv
    • platform: test__locale test__osx_support test_asyncio test_baseexception test_builtin test_calendar test_cmath test_fcntl test_math test_mimetypes test_os test_platform test_regrtest test_shutil test_socket test_strptime test_sysconfig test_time test_tkinter test_winreg test_wsgiref
      • profiling.sampling.gecko_collector: test_profiling
      • profiling.sampling.heatmap_collector: test_profiling
      • pydoc: test_enum test_pydoc
    • webbrowser: test_webbrowser

[x] test: cpython/Lib/test/test_marshal.py (TODO: 3)

dependencies:

dependent tests: (32 tests)

  • marshal: test_bool test_exceptions test_importlib test_inspect test_marshal test_profiling test_zipimport
    • importlib._bootstrap_external: test_importlib test_unittest
      • modulefinder: test_importlib test_modulefinder
      • py_compile: test_cmd_line_script test_compileall test_importlib test_multiprocessing_main_handling test_py_compile test_pydoc test_runpy
      • pydoc: test_enum
      • zipfile: test_pdb test_pkgutil test_shutil test_zipapp test_zipfile test_zipfile64 test_zipimport_support
    • pkgutil: test_pyrepl
    • profile: test_profile
    • profiling.tracing: test_profiling
      • cProfile: test_cprofile test_pstats
    • zipimport: test_importlib

[ ] lib: cpython/Lib/asyncio
[ ] test: cpython/Lib/test/test_asyncio (TODO: 3)

dependencies:

  • asyncio (native: _asyncio, _overlapped, _pyrepl.console, _pyrepl.main, _pyrepl.simple_interact, _remote_debugging, _winapi, asyncio.tools, base_events, collections.abc, concurrent.futures, coroutines, errno, events, exceptions, futures, graph, itertools, locks, log, math, msvcrt, protocols, queues, readline, runners, streams, sys, taskgroups, tasks, threads, time, timeouts, transports, unix_events, windows_events)
    • io, site
    • _colorize, argparse, ast, code, collections, contextlib, contextvars, dataclasses, enum, functools, heapq, inspect, linecache, logging, os, reprlib, rlcompleter, selectors, signal, socket, ssl, stat, struct, subprocess, threading, tokenize, traceback, types, warnings, weakref

dependent tests: (8 tests)

  • asyncio: test_asyncio test_concurrent_futures test_external_inspection test_inspect test_logging test_os test_pdb test_unittest

[x] lib: cpython/Lib/_colorize.py
[x] test: cpython/Lib/test/test__colorize.py

dependencies:

  • _colorize

dependent tests: (267 tests)

  • _colorize: test__colorize test_argparse test_httpservers test_json test_pdb test_profiling test_pyrepl test_regrtest test_timeit test_traceback
    • argparse:
      • ast: test_ast test_builtin test_codeop test_compile test_compiler_codegen test_coroutines test_dis test_fstring test_future_stmt test_peepholer test_peg_generator test_site test_ssl test_type_comments test_ucn test_unparse
      • calendar: test_calendar test_imaplib test_strftime
      • compileall: test_compileall
      • dis: test__opcode test_code test_compiler_assemble test_dtrace test_inspect test_monitoring test_opcache test_patma test_positional_only_arg test_type_cache
      • ensurepip: test_ensurepip test_venv
      • gzip: test_fileinput test_tarfile test_xmlrpc
      • http.server: test_logging test_robotparser test_urllib2_localnet
      • inspect: test_abc test_asyncgen test_buffer test_clinic test_collections test_decimal test_enum test_functools test_generators test_grammar test_ntpath test_operator test_posixpath test_pydoc test_signal test_sqlite3 test_turtle test_type_annotations test_type_params test_types test_typing test_unittest test_yield_from test_zipimport test_zipimport_support test_zoneinfo
      • mimetypes: test_mimetypes
      • pickle: test_annotationlib test_array test_asyncio test_bool test_bytes test_bz2 test_codecs test_concurrent_futures test_configparser test_csv test_ctypes test_defaultdict test_deque test_descr test_dict test_dictviews test_email test_enumerate test_exceptions test_fractions test_genericalias test_genericpath test_http_cookies test_importlib test_io test_ipaddress test_iter test_itertools test_json test_list test_lzma test_memoryview test_minidom test_ordered_dict test_os test_pathlib test_pickle test_picklebuffer test_pickletools test_platform test_plistlib test_random test_range test_re test_set test_shelve test_slice test_socket test_statistics test_str test_string test_structseq test_super test_time test_trace test_tuple test_type_aliases test_unittest test_urlparse test_uuid test_xml_dom_minicompat test_xml_etree test_xpickle test_zipfile test_zlib test_zoneinfo
      • platform: test__locale test__osx_support test_asyncio test_baseexception test_cmath test_ctypes test_fcntl test_math test_shutil test_strptime test_sysconfig test_tkinter test_winreg test_wsgiref
      • py_compile: test_cmd_line_script test_importlib test_modulefinder test_multiprocessing_main_handling test_py_compile test_runpy
      • random: test_asyncio test_bisect test_complex test_context test_dbm_dumb test_devpoll test_dummy_thread test_email test_float test_grp test_heapq test_hmac test_importlib test_int test_long test_mmap test_numeric_tower test_poll test_pow test_pprint test_profiling test_pwd test_queue test_richcmp test_selectors test_sort test_strtod test_struct test_sys test_thread test_threading test_tokenize test_weakref test_zipfile test_zstd
      • sqlite3.main: test_sqlite3
      • tokenize: test_linecache test_tabnanny
      • uuid: test_os
      • webbrowser: test_webbrowser
      • zipapp: test_zipapp
      • zipfile: test_pkgutil test_zipfile test_zipfile64
    • ast:
      • annotationlib: test_reprlib
      • pyclbr: test_pyclbr
      • traceback: test_asyncio test_code_module test_concurrent_futures test_contextlib test_contextlib_async test_dictcomps test_http_cookiejar test_importlib test_listcomps test_pyexpat test_setcomps test_subprocess test_threadedtempfile test_unittest test_with
    • calendar:
      • mailbox: test_mailbox
      • ssl: test_asyncio test_ftplib test_httplib test_poplib test_urllib
    • difflib: test_difflib test_profile test_sys_settrace
    • http.server:
      • xmlrpc.server: test_docxmlrpc
    • profiling.sampling.pstats_collector: test_profiling
    • tokenize:
      • idlelib: test_idle
      • importlib._bootstrap_external: test_importlib
      • linecache: test_bdb
    • traceback:
      • concurrent.futures.interpreter: test_concurrent_futures
      • concurrent.futures.process: test_concurrent_futures
      • logging: test_hashlib test_support test_urllib2net
      • multiprocessing: test_asyncio test_concurrent_futures test_importlib test_multiprocessing_forkserver test_profiling
      • socketserver: test_socketserver
      • threading: test_android test_asyncio test_concurrent_futures test_ctypes test_external_inspection test_fork1 test_frame test_gc test_importlib test_io test_ioctl test_pathlib test_sched test_smtplib test_syslog test_termios test_threading_local
      • tkinter: test_tkinter test_ttk test_ttk_textonly

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@youknowone
youknowone marked this pull request as ready for review October 23, 2026 08:56
@youknowone
youknowone merged commit d8b8e47 into RustPython:main Oct 23, 2026
22 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T09:03:52.965188Z 5963efc Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5963efcdcf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Lib/asyncio/__main__.py
from _colorize import get_theme
from _pyrepl.console import InteractiveColoredConsole as InteractiveConsole
except ModuleNotFoundError:
from code import InteractiveConsole

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Define the failure sentinel for the basic-console fallback

When _pyrepl.console is unavailable and this fallback is selected, any evaluated statement that raises reaches AsyncIOInteractiveConsole.runcode(), which returns self.STATEMENT_FAILED; code.InteractiveConsole does not define that attribute (it exists only on InteractiveColoredConsole). The resulting AttributeError escapes the basic console's interact() loop and REPLThread stops the event loop, so the first normal user error terminates the fallback REPL. Define the sentinel on AsyncIOInteractiveConsole or avoid returning it on this path.

Useful? React with 👍 / 👎.

Comment thread Lib/asyncio/__main__.py
ps.add_argument("pid", type=int, help="Process ID to inspect")
ps.add_argument(
"--retries",
type=int,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject negative retry counts before invoking the tools

When either new --retries option is given a negative integer (for example, ps 42 --retries -1), argparse accepts it because type=int; asyncio.tools._get_awaited_by_tasks() then iterates range(retries + 1) zero times and returns None, after which the table/tree builder raises TypeError and the CLI emits a traceback. The generated help advertises --retries RETRIES as “Number of retries on transient attach errors”, so use a non-negative parser validator while preserving 0 as one initial attempt.

Useful? React with 👍 / 👎.

def _display_filename(filename):
if not filename or filename == "~":
return filename
with contextlib.suppress(ValueError):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep dumps working when the current directory is gone

When format_stack_dump() runs after the profiler's current directory has been removed, os.path.relpath() calls getcwd() and raises FileNotFoundError; this suppresses only ValueError, so filename prettification aborts the entire dump even though the original filename can still be emitted. Suppress OSError as well and fall back to filename.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants