Skip to content

Fix format field parsing and lookup order - #9438

Draft
youknowdot wants to merge 2 commits into
RustPython:mainfrom
youknowdot:youknowdot/lazy-format-fields-315-20261028
Draft

Fix format field parsing and lookup order#9438
youknowdot wants to merge 2 commits into
RustPython:mainfrom
youknowdot:youknowdot/lazy-format-fields-315-20261028

Conversation

@youknowdot

@youknowdot youknowdot commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Fix the native prerequisite for CPython 3.15's full StrTest.test_format rather than retaining its expected-failure marker.

  • Resolve each format field's root before parsing its suffix, then interleave each suffix parse with its attribute/item lookup. _string.formatter_field_name_split now returns a self-iterator with CPython's resumable cursor behavior after parsing errors.
  • Scan field names, conversions and format specifications separately. Braces, ! and : inside item keys remain key data; malformed outer fields are rejected before lookup. Conversion happens before nested specification expansion, with an explicit two-level expansion budget rather than a syntax-parser nesting restriction.
  • Regenerate Lib/test/test_str.py from pinned CPython 3.15.0 using scripts/update_lib migrate, retiring only the now-passing test_format marker. The other two existing markers are preserved exactly. The same updater also retires the now-passing FormatterTest.test_format_validate marker in Lib/test/test_logging.py, restoring exact pinned canonical bytes. No assertions or new skips were added.

Validation

  • Scoped cargo check and strict cargo clippy -- -D warnings: passed.
  • Bounded executable build: passed; validated binary built from base 95544a5b82c7f40c45b169801a45701547762a2c plus this native patch.
  • Six selected complete canonical methods: passed, no skips (StrTest.test_format, test_format_map, test_format_auto_numbering, test_format_huge_item_number, StringModuleTest.test_formatter_field_name_split, and ModuleTest.test_basic_formatter). The regenerated repository's full test_format also passes independently. The complete regenerated FormatterTest.test_format_validate passes separately (one test, no skips or expected failures); this retires the obsolete marker responsible for the initial hosted Linux unexpected-success failure.
  • Finite comparisons against pinned CPython d220409a13b77de7a6c30abe16ea90db01528edc: seven suffix-iterator cases and eight lookup cases match, and fourteen outer parser/format cases match. Three additional phase cases match; the known remaining case below is explicitly not fixed.
  • Updater unit tests and applicable hooks passed. No whole-suite or concurrency compatibility claim.

Remaining limits

This does not establish complete CPython formatting parity. FormatString still parses the whole string eagerly: for '{0}{'.format() RustPython reports the later syntax error before the missing first argument, whereas CPython reports IndexError. Existing numeric-field policy differences (Unicode decimal digits, plus signs and overflow prefixes) remain outside this patch. The local validation base predates the subsequently merged source-decoder and string-interning updates; neither unrelated update is part of the tested binary.

Assisted by OpenAI Codex; model version unavailable.

@codex
Fix format field parsing and lookup order
adc4d7e 
Assisted-by: OpenAI Codex: model version unavailable
@coderabbitai

coderabbitai Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] lib: cpython/Lib/logging
[x] test: cpython/Lib/test/test_logging.py

dependencies:

  • logging

dependent tests: (27 tests)

  • logging: test_asyncio test_concurrent_futures test_decimal test_genericalias test_hashlib test_logging test_pkgutil test_support test_unittest test_urllib2net
    • asyncio.futures: test_asyncio
    • concurrent.futures._base: test_concurrent_futures
    • http.cookiejar: test_http_cookiejar test_urllib2
      • urllib.request: test_httpservers test_pathlib test_pydoc test_sax test_site test_ssl test_urllib test_urllib2_localnet test_urllibnet
    • multiprocessing.util: test_asyncio test_compileall test_concurrent_futures
    • venv: test_venv

[x] test: cpython/Lib/test/test_str.py (TODO: 2)
[ ] test: cpython/Lib/test/test_fstring.py (TODO: 1)
[x] test: cpython/Lib/test/test_string_literals.py

dependencies:

dependent tests: (no tests depend on str)

[x] test: cpython/Lib/test/test_source_encoding.py

dependencies:

dependent tests: (no tests depend on source_encoding)

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@codex
Retire passing logging formatter validation marker
f1a1d99 
Regenerate the canonical logging test via update_lib after the native formatter fix. The complete formatter validation method passes without its obsolete expected-failure marker.

Assisted-by: OpenAI Codex: model version unavailable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

See More

Successfully merging this pull request may close these issues.

2 participants

@youknowdot @codex