Skip to content

Fix interpolated string warning diagnostic locations - #9314

Merged
youknowone merged 4 commits into
RustPython:mainfrom
youknowdot:fix-interpolated-warning-locations-20261022
Oct 24, 2026
Merged

youknowone merged 4 commits into
RustPython:mainfrom
youknowdot:fix-interpolated-warning-locations-20261022

Conversation

@youknowdot

@youknowdot youknowdot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Rebased #9314 onto main fd3c3fca644b93d0bec719efb3f864056b5d3b68 and reviewed its implementation against CPython 3.15.0 source d220409a13b77de7a6c30abe16ea90db01528edc.

  • Reconstruct CPython decoder fragments, including named Unicode boundaries and the different doubled-brace rules for body versus format-spec constants.
  • Keep source offsets for deduplication and Single-input boundaries, with separate END/MIDDLE token anchors for diagnostic locations and parser action ordering.
  • Match the Unicode decoder's protection of non-ASCII characters after backslashes; independently collect tokenizer-owned brace warnings.
  • Preserve parser/tokenizer phase order across nested f/t strings, format specs, numeric events, and warning escalation/recovery.
  • Preserve pending Single and rejected-type-comment errors while allowing the appropriate tokenizer recovery events. TYPE_COMMENTS uses the shared warning collector instead of prematurely emitting format-spec callbacks.
  • Match numeric zero-width error metadata and physical-line text, and ordinary Unicode-token final-line text and universal-newline positions, including exec bare CR.

No canonical tests or Python library files were modified, no new RustPython behavior tests were added, and no diagnostic filtering or expected-output special cases were introduced.

Validation

The exact production source passed:

  • Default-feature VM cargo check --offline --locked -p rustpython-vm --lib.
  • Strict VM cargo clippy --offline --locked -p rustpython-vm --lib -- -D warnings.
  • Default CLI cargo build --offline --locked --bin rustpython.
  • 832 fixed compile-only observations against the pinned CPython oracle, with complete raw warnings and exception objects retained. Returned code objects/ASTs were not executed.
  • Five unchanged canonical methods through normal unittest fixtures/decorators: f-string string-part backslashes, backslash-before-double-bracket, warns-once, raw t-strings, and triple-quoted t-strings. All passed with no skips or expected failures.

An independent readonly candidate binary was sealed and its actual native and Python-module origins verified. Every gate checked source/tool/Lib bindings. Candidate source patch SHA256: 2a3f8ba850233eee80a0a2df4972363697d1707ae57e42a40d3cd4a1f958a7df (final follow-up delta against validated v8); binary SHA256: eb75c0a58986a48142d3ae7986823b89c520c6c8224da8ba925841cbb565b9a5.

All 832 observations agree on warning lists and exception selection/chaining, including f/t body, format-spec, brace-tokenizer locations and the added numeric/ordinary-text corrections. The final follow-up corrected exactly four ordinary exec bare-CR rows; the other 828 rows were unchanged.

Exact results and remaining boundaries

Full raw equality is 382/832. A further 382 rows differ only in existing global SyntaxError args/repr construction. The remaining 68 rows retain other diagnostic differences: rejected TYPE_COMMENT validator ranges (40), Single generic-error LF text (18), the explicit malformed-source fallback control (10). These are recorded, not normalized away. General ordinary-string diagnostics outside the reviewed final-line scope are not claimed globally equivalent.

Arbitrary custom warning-callback exception conversion, complete malformed-source parser recovery, global SyntaxError argument tuples, and real-file filename lookup are not rewritten or claimed fully equivalent. Full workspace/CAPI suites and entire Python test modules were not run in this bounded validation.

AI assistance: OpenAI Codex (model version unavailable); commits contain Assisted-by trailers.

@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@youknowdot
youknowdot force-pushed the fix-interpolated-warning-locations-20261022 branch from 657e8a1 to 6f606cd Compare October 23, 2026 12:30
Track CPython string-parser fragment anchors separately from escape source offsets. Preserve scoped error-line text and mode-specific final newlines with lazy sparse location indexing.

Assisted-by: OpenAI Codex:model-version-unavailable
Assisted-by: OpenAI Codex (model version unavailable)
Reconstruct body and format-spec decoder fragments, preserve tokenizer/parser action order and pending Single/type-comment errors, and match Unicode protection plus scoped numeric and ordinary-token diagnostics.

Assisted-by: OpenAI Codex:model-version-unavailable
Keep warning and error coordinates consistent across bare CR, CRLF, and LF while retaining file-input synthetic newlines.

Assisted-by: OpenAI Codex:model-version-unavailable
@youknowdot
youknowdot force-pushed the fix-interpolated-warning-locations-20261022 branch from f1ef908 to d02a4f1 Compare October 24, 2026 00:49
@youknowone
youknowone marked this pull request as ready for review October 24, 2026 02:28
@youknowone
youknowone merged commit a788de2 into RustPython:main Oct 24, 2026
20 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-24T02:30:01.482704Z
We couldn’t complete this review. Comment @codex review to try again.
d02a4f1 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants