Skip to content

Preserve numeric warning error precedence in AST single mode - #9303

Merged
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:split/ast-single-warning-recovery-20261022
Oct 22, 2026
Merged

youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:split/ast-single-warning-recovery-20261022

Conversation

@youknowdot

Copy link
Copy Markdown
Contributor

Summary

Follow up #9296 by allowing numeric-warning recovery in compile(..., 'single', ast.PyCF_ONLY_AST).

When a parser-origin escape warning becomes an error, CPython still scans for later numeric tokenizer warnings raised as errors. The ordinary compilation path already preserves that precedence, but the AST path only enabled it for file/eval input. Include Single in that existing allowlist. This also matters when the numeric warning is on a later logical line.

The patch does not change normal warning-emission boundaries. In particular, the remaining cases that emit warnings from a second statement are separate work. Prompt-interactive input and FuncType retain their existing paths.

Validation

Compared against final CPython 3.15.0, pinned at d220409a13b77de7a6c30abe16ea90db01528edc, and the source-bound historical #9296 implementation at eb79fef279d646aedf5fca6598d330b54a401138. Candidate is based on main 0b0a3607648b5b01b3adfc1c06f3bec1d87f8a13; the historical baseline is not presented as current main.

  • 32 fixed Single cases cover ordinary/AST compilation, always/error filters, one expression, semicolon-separated statements, later logical lines, compound suites and controls. Exactly four AST error-selection results change to the expected numeric error. Warning/error-selection agreement improves from 18/32 to 22/32. Full raw metadata equality remains 12/32 because existing numeric diagnostic metadata differences are deliberately retained.
  • 16 parser-invalid/tokenizer-warning regression cases are unchanged from the historical baseline, including its existing CPython differences.
  • The unchanged 24-case mixed warning-order corpus is unchanged from the historical baseline.
  • Every observation only calls compile; returned code and AST objects are never executed. No custom warning callbacks were exercised.
  • Four unchanged canonical test_compile.TestSpecifics methods passed: test_single_statement, test_bad_single_statement, test_compile_ast, and test_compile_warnings. Four tests ran with no skips, expected failures, unexpected successes, errors or failures.
  • Native default+capi build and normal commit hooks passed. Parser-disabled, nonthreading and threading VM checks, test-target compilation, and strict all-target threading Clippy passed.

No full Python test modules, Rust test execution, workspace/CAPI suites, Windows/macOS/WASM/Miri execution or performance benchmark was run for this patch. The scoped test-target gate compiles tests; it does not execute them.

Limits

Parser-invalid collection and tokenizer-origin f-string warning guards remain in place. AST emission does not itself gate on has_valid_syntax, so this is not a claim about every malformed-source case. Pre-parse safety paths were source-reviewed, not exercised with deep/hazardous inputs.

Non-SyntaxWarning callback exceptions still return directly. A callback-raised SyntaxWarning or subclass qualifies for the same existing type-based recovery as a filter-escalated warning, so AST Single can now proceed to later numeric-warning handling in that case. Custom-callback parity with CPython is not established by these observations.

This does not fix the ten observed statement-boundary cases, numeric offset/text metadata, exception args/repr, or the separate invalid-source recovery gaps. No Lib assertions or expected-failure markers were changed.

AI assistance

This change was prepared and validated with OpenAI Codex assistance.

Match ordinary single compilation by allowing later numeric warnings raised as errors to take precedence after a parser-origin escape warning. Preserve tokenizer-origin guards and the separate prompt path.

Assisted-by: Codex:unknown
@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
📥 Commits

Reviewing files that changed from the base of the PR and between 0b0a360 and 567e3ec.

📒 Files selected for processing: 1
  • crates/vm/src/vm/compile.rs
+---------------------------------------------------------+
| Nuke the bugs from orbit. It's the only way to be sure. |
+---------------------------------------------------------+
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@youknowone
youknowone marked this pull request as ready for review October 22, 2026 14:23
@youknowone
youknowone merged commit 36f8e89 into RustPython:main Oct 22, 2026
21 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T14:26:49.523993Z 567e3ec Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants