Skip to content

Keep deque contents and bounds in coherent shared state - #9457

Merged
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:youknowdot/deque-shared-state-315-20261029
Oct 29, 2026
Merged

youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:youknowdot/deque-shared-state-315-20261029

Conversation

@youknowdot

@youknowdot youknowdot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Keep deque contents and maxlen in one synchronized payload, removing the shared maxlen mutation through a cast.
  • Use the existing creator-bound PyOperation for writers and index/copy, preserving reentrant callbacks outside short payload locks.
  • Match CPython's ordering for index-bound conversion, successful/raising comparisons, iterator mutation checks, initialization, extension, and subclass copying.
  • Import the whole test_free_threading/test_collections.py through scripts/update_lib, pinned to CPython d220409a13b77de7a6c30abe16ea90db01528edc.

This is independent of #9439. Its existing operation-guard prerequisite from #9449 is already on main.

CPython basis

  • _collectionsmodule.c: index lines 1262–1323, extend lines 479–574, copy lines 593–660, initialization lines 1755–1777, and iterator traversal lines 1969–2005 / 2120–2156.
  • Clinic argument conversion: start and stop conversion precedes entering the index operation.

Validation

The exact two changed files were validated on main b85582d57063c892fdf1839a495e47e5bf8e78be before the execution environment was replaced:

  • Scoped cargo check and strict Clippy passed.
  • A CLI build passed.
  • Eight ordinary canonical test_deque methods passed: copy/copy_method, index regression 24913, maxlen attribute/zero, initialization, altered iterator data, and mutation of an initially empty iterator.
  • Ten bounded serial callback/argument-conversion cases matched pinned CPython 3.15 exactly, including index mutation and subclass-copy behavior.

Both changed files are restored byte-for-byte from that validated patch. The later main changes through 698c196b2e188c46772f8a3cf76164227282a645 affect SQLite, regex, Windows overlapped cleanup, and an asyncio snippet; none overlap the deque implementation or operation guard. The binary was lost when the environment was replaced, so these are recorded earlier runs, not fresh execution on the final main base. Hosted CI must validate the published commit.

The new whole free-threading module was not run locally. No full workspace/C-API suites, concurrent/GC/destructor/stress tests, or broad thread-safety claim are included. Remaining count/contains/richcompare, integer-width/error-text, GC/finalizer and cross-interpreter behavior is outside this change's scope.

AI assistance: OpenAI Codex (model version unavailable).

Summary by CodeRabbit

  • Bug Fixes
    • Improved deque handling of maximum lengths during initialization, insertion, and repetition.
    • Corrected index conversion for deque operations, including support for objects implementing __index__ and safe handling of out-of-range values.
    • Improved mutation detection during comparisons and iteration, and ensured iterator errors behave consistently.
    • Fixed deque state handling in copying, concatenation, multiplication, and pickling.
    • Improved error reporting when allocation fails during in-place multiplication.

Use the existing creator-bound Python operation guard for deque mutations and index/copy, synchronize contents with maxlen, and preserve CPython callback ordering. Import the whole canonical free-threading collections test through update_lib.

Assisted-by: OpenAI Codex: model version unavailable
@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 698c196 and 0dc1683.

Files ignored due to path filters: 1
  • Lib/test/test_free_threading/test_collections.py is excluded by !Lib/**
Files selected for processing: 1
  • crates/vm/src/stdlib/_collections.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

Deque storage and maxlen now share one lock-protected data structure. Deque operations use operation guards, and iteration, mutation detection, copying, and repetition use updated state handling.

Changes: Deque updates

Layer / File(s) Summary
Deque state and mutations
crates/vm/src/stdlib/_collections.rs
Deque storage and accessors group items with maxlen. Extension and core mutation methods use operation guards and release removed objects after unlocking.
Repetition, concatenation, and copying
crates/vm/src/stdlib/_collections.rs
Repetition, concatenation, and copying use deque snapshots and preserve maxlen. Allocation failures during repetition map to MemoryError.
Search and indexed insertion
crates/vm/src/stdlib/_collections.rs
Index bounds use explicit conversion and saturation. Search and removal detect mutation during comparisons, and insert checks maxlen before changing the deque.
Snapshots and iterator state
crates/vm/src/stdlib/_collections.rs
Representation and iterator setup read consistent deque state. Forward and reverse iterators share mutation and exhaustion handling while preserving their respective mutation-error behavior.

Priority: Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix

Possibly related issues

  • RustPython/RustPython#6839 — This PR imports test_free_threading/test_collections.py as one module-level step in the tracked library and test migration.

Possibly related PRs

Suggested reviewers: youknowone, 1ndahous3

Merge Risk: Minimal · up to 0dc16

The deque synchronization changes have no established merge-blocking defect. Merge after normal checks; concurrent and stress behavior has not been independently validated.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: synchronizing deque contents and bounds in one coherent shared state.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 13 functions across 1 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] test: cpython/Lib/test/test_free_threading

dependencies:

dependent tests: (no tests depend on free_threading)

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@youknowone
youknowone marked this pull request as ready for review October 29, 2026 11:19
@youknowone
youknowone enabled auto-merge (squash) October 29, 2026 11:19
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-29T11:20:40.610494Z
We couldn’t complete this review. Comment @codex review to try again.
0dc1683 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@youknowone
youknowone merged commit bcaa07a into RustPython:main Oct 29, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants