Skip to content

Align XML backend and canonical libraries with Python 3.15 - #9197

Merged
youknowone merged 5 commits into
RustPython:mainfrom
youknowdot:split/xml-backend-315
Oct 27, 2026
Merged

youknowone merged 5 commits into
RustPython:mainfrom
youknowdot:split/xml-backend-315

Conversation

@youknowdot

@youknowdot youknowdot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Replace the earlier vendored implementation with an independent, non-vendored XML compatibility split from #8954.

  • Keep the existing registry xml 1.4.0 dependency and its existing expansion limits. No Cargo changes, vendored files or updater-tool changes remain in this PR's diff.
  • Improve namespace callbacks, parser-error propagation, handler-return validation and ElementTree error reporting through the published backend's APIs.
  • Update 12 XML library/test files from CPython 3.15.0, commit d220409a13b77de7a6c30abe16ea90db01528edc, using the existing scripts/update_lib tool.
  • Remove 25 existing expected-failure decorators demonstrated stale by canonical tests. Test bodies/assertions are unchanged; unproven markers remain.

The previous custom backend supplied cumulative amplification-accounting APIs absent from the published crate. This replacement excludes those APIs instead of vendoring the backend. Its pre-existing fixed expansion protections are unchanged.

Fresh validation

The local validation below was run on replacement commit dcff68be451106be0de358b266152b424c837eda against pinned main 75df7a8c2e47f981163792fe0154d698c28bcf93. The subsequent one-decorator follow-up is described separately below:

  • Locked, offline, single-job build with --no-default-features --features stdlib,stdio,importlib,host_env,threading: passed.
  • Targeted rustpython-stdlib library Clippy with -D warnings: passed.
  • Exactly 52 canonical observations in five bounded processes: 49 successes and 3 expected failures, with no unexpected successes, failures, errors or skips. Coverage includes XML validators, SAX/DOM consumers, pure/native ElementTree cases, handler-return errors, and three isolated parent-parser constructor/reference-drop methods.
  • The three retained expected failures document incomplete reparse-enabled callback timing and two malformed-input error-message differences.
  • A fresh build initially reached its 40-minute execution limit. A separately reviewed same-source continuation preserved completed dependencies, rebuilt unfinished VM/stdlib/root components, and completed build and Clippy. The timed-out attempt is not counted as a successful build.

The 52 local observations are not a complete XML-suite or workspace validation. On replacement commit dcff68be, Linux, macOS and Windows canonical CI found one stale expected-failure marker: ExpatReaderTest.test_expat_external_dtd_enabled passed unexpectedly in each initial run and rerun. This follow-up removes that decorator through update_lib, leaving assertions unchanged. The test validates external resolver dispatch and the expected unsupported-scheme URLError, not successful external DTD parsing. Correctness CI for dcff68be has finished: all other jobs passed; native canonical jobs failed only on this stale marker. On follow-up commit 4eb7a6889f08c93c6b459f7daeba7bae82364aac, all three native canonical jobs and the other 20 correctness jobs passed; Lib Dependencies also passed. The remaining Linux Rust job reached its 45-minute execution limit during sandbox-mode compilation and was cancelled, so full CI is not green. A targeted rerun request using the PR author account was denied because repository admin rights are required; a maintainer rerun is needed. Earlier broad results in this PR applied to the superseded vendored implementation, not this replacement.

Remaining limitations

  • Cumulative amplification-limit setters remain unsupported; existing published-backend fixed limits are not weakened.
  • External child-parser construction/inherited state does not implement parsing external DTDs or entity fragments; feeding those inputs still raises NotImplementedError.
  • Namespace maps cannot preserve redundant same-URI declarations or arbitrary declaration order.
  • Reparse settings do not provide full Expat deferred-callback timing or buffered-WASM performance/security parity.
  • The bounded local validation did not run broad lifecycle/GC, allocation-failure, entity-expansion, external-resolution or network suites. The isolated constructor methods create at most three parsers without feeding XML or explicitly invoking GC.
  • Reviewed reads use three exact local canonical fixtures. Audit hooks provide defense in depth, not complete audit-event coverage or kernel isolation.
  • The shared XMLGenerator method passed in four memory-backed variants in local validation; its file-writing variant was deliberately unrun locally. CI subsequently exercised the full SAX module; this is not a claim that all unselected markers are resolved.

Commit hooks

cspell alone was skipped for the initial XML correction commit under explicit user authorization. The normal Git commit ran conflict checking, Ruff format/check, redundant-test-patches and rustfmt successfully. Prettier reported no applicable files under normal merge-hook selection. The separate exception for #9342 was not reused; no broader hook waiver applies.

For the one-line Python-only follow-up, the normal Git commit ran with SKIP unset: conflict checking, Ruff format/check and redundant-test-patches passed. Rustfmt, cspell and Prettier naturally reported no applicable files. No further hook exception was used.

AI assistance: dot assisted with extraction, canonical migration, source review and bounded validation. The exact model version is not exposed.

@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] lib: cpython/Lib/xml
[x] test: cpython/Lib/test/test_xml_etree.py (TODO: 18)
[ ] test: cpython/Lib/test/test_xml_etree_c.py
[x] test: cpython/Lib/test/test_minidom.py (TODO: 2)
[x] test: cpython/Lib/test/test_pulldom.py
[ ] test: cpython/Lib/test/test_pyexpat.py (TODO: 41)
[x] test: cpython/Lib/test/test_sax.py (TODO: 25)
[x] test: cpython/Lib/test/test_xml_dom_minicompat.py
[x] test: cpython/Lib/test/test_xml_dom_xmlbuilder.py

dependencies:

  • xml

dependent tests: (38 tests)

  • xml: test_doctest test_minidom test_pulldom test_pydoc test_pyexpat test_regrtest test_sax test_typing test_xml test_xml_dom_minicompat test_xml_dom_xmlbuilder test_xml_etree
    • plistlib: test_plistlib
      • platform: test__locale test__osx_support test_android test_asyncio test_baseexception test_builtin test_calendar test_cmath test_ctypes test_fcntl test_math test_mimetypes test_os test_platform test_shutil test_socket test_ssl test_strptime test_sysconfig test_time test_tkinter test_winreg test_wsgiref
      • webbrowser: test_webbrowser
    • xmlrpc.client: test_xmlrpc

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@codspeed

codspeed Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks1


Comparing youknowdot:split/xml-backend-315 (4eb7a68) with main (75df7a8)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Comment thread vendor/xml/patches/amplification.patch Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, vendoring is strictly not allowed. tell me why this is required

youknowdot and others added 3 commits October 21, 2026 22:01
Retain the existing XML backend accounting changes and their explicit registry publication limitation. Preserve canonical XML tests and existing RustPython guards, including multiline marker explanations through update_lib.

Assisted-by: OpenAI Codex:model version unavailable
Assisted-by: OpenAI Codex:model version unavailable
Replace the vendored crate sources with the published xml 1.4.0 archive
and reapply the amplification accounting patch on top of it. The two
lexer hunks that conflicted with the new line-ending normalization are
merged into the 1.4.0 read loop. The patch no longer normalizes CR in
entity values or collapses CRLF in attribute values, because the 1.4.0
lexer and attribute reader already do so.

Update the checksums, recorded upstream commit, archive hash and
provenance notes, and require version 1.4 for the path dependency.

Assisted-by: Claude Code:claude-opus-5-5
@youknowone

Copy link
Copy Markdown
Member

kornelski/xml-rs#93

Merge pinned main 75df7a8 into the existing XML branch and replace the
vendored implementation with the reviewed 14-file compatibility split.
Keep registry xml 1.4.0 and its existing expansion limits; exclude the
unsupported cumulative-amplification APIs and updater-tool changes.

Migrate canonical CPython 3.15 XML files through update_lib and retire
24 proven stale expected-failure decorators without changing assertions.
Fresh locked/offline build and targeted stdlib Clippy passed. The exact
52 bounded canonical observations produced 49 successes and 3 retained
expected failures; this is not a full XML-suite or cross-platform claim.
The first guarded build timed out; a reviewed same-source continuation
completed successfully without treating the timed-out attempt as a pass.

Skip cspell only for this one correction commit under the user's explicit
exception. All other applicable normal hooks remain enabled.

AI assistance covered extraction, migration, source review and bounded
validation. The actual model identifier is not exposed.

Assisted-by: dot: model version not exposed
Linux and macOS canonical CI both demonstrate that the existing
unsupported-scheme resolver test passes. Retire only its expected-failure
decorator through update_lib, preserving its canonical assertions.
This does not implement external DTD parsing.

AI assistance: dot diagnosed CI, migrated the marker, and reviewed the
source delta and normal-hook validation; the model version is not exposed.

Assisted-by: dot: model version not exposed
@youknowone
youknowone marked this pull request as ready for review October 27, 2026 12:02
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-27T12:03:31.253079Z
We couldn’t complete this review. Comment @codex review to try again.
4eb7a68 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@youknowone
youknowone merged commit b25ab5e into RustPython:main Oct 27, 2026
53 of 54 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants