Share ctypes backing-owner snapshots between aggregate copies by 1ndahous3 · Pull Request #9392 · RustPython/RustPython

1ndahous3 · 2026-10-26T09:04:35Z

Addresses the quadratic private ownership metadata identified while reviewing #9251.

Summary

Share a flat snapshot of hidden backing owners while the source owner set is unchanged. Destination slots retain group IDs instead of copying every owner ID.
Keep owner counts and snapshot invalidation local to distinct groups, preserving alias lifetimes and releasing replaced values after publication.
Track internal snapshot tuples without running collection while their ownership state is being published. Expose their ownership edges to cyclic GC.

Performance

Repeated aggregate copies with an unchanged owner set:

Platform	Elements	Before	After
Windows 11 x64	1,024	11.50 ms	0.99 ms
Debian 12 amd64	4,096	257.37 ms	5.52 ms

Peak process RSS in the Debian scenario: 236.2 MiB → 70.9 MiB.

AI assistance

This PR follows our AI policy.

Written with Codex (GPT-6), reviewed by a human before submission.

Summary by CodeRabbit

Bug Fixes
- Improved reference handling for ctypes objects when retained references are replaced or shared, helping ensure related objects remain available while needed.
- Improved memory-management behavior during object creation and reference updates. Some new objects can now be tracked without triggering collection immediately; subsequent tracked allocations continue to check collection thresholds as usual. These changes help reduce unintended collection during allocation and improve the reliability of object lifetimes.

chatgpt-codex-connector · 2026-10-26T09:04:46Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	🔄 Running since 2026-10-26T17:39:32.933102Z	`f4df593`	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

coderabbitai · 2026-10-26T09:05:28Z

No actionable comments were generated in the recent review.

Recent review info

Run configuration

Source mode: Auto
Repository: RustPython/RustPython
Branch: main
File: .coderabbit.yml
Commit: b811715
Layers: Repository YAML
Review profile: CHILL
Plan: Advanced
Run ID: 97ed01c2-e46a-4524-b869-894daa91c4ef

Commits

Reviewing files that changed from the base of the PR and between 332101a and f4df593.

Files selected for processing: 2

crates/vm/src/gc_state.rs
crates/vm/src/object/core.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

Walkthrough

Object tracking can now suppress collection during allocation. ctypes hidden references use owner groups and cached tuple snapshots for groups with multiple owners.

Changes: ctypes hidden-reference retention

Layer / File(s)	Summary
Optional collection during allocation `crates/vm/src/gc_state.rs`, `crates/vm/src/object/core.rs`	`track_new_object_impl` and a shared `PyRef` allocation path accept a collection flag. `new_ref` enables collection; `new_ref_without_collection` disables it on supported host builds.
Owner groups and snapshots `crates/vm/src/stdlib/_ctypes/base.rs`	`PrivateRefs` tracks owner groups and reference counts, traverses and clears retained references, and caches a tuple snapshot for multiple owners until the owner set changes.
CData hidden-reference integration `crates/vm/src/stdlib/_ctypes/base.rs`	`CDataValue` and retention helpers use optional `HiddenRefs` groups in conversions, assignments, and keep operations.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Possibly related PRs

RustPython/RustPython#9391: Changes PrivateRefs owner structure and traversal paths used by this PR.
RustPython/RustPython#9390: Adds GC traversal and clearing for ctypes state that this PR extends.
RustPython/RustPython#9251: Introduces the reference-counted ownership and snapshot model that this PR changes.

Suggested reviewers: youknowone

Merge Risk: Minimal · up to f4df5

No actionable merge-blocking defect is established by the reviewed changes.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 21.05% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 19 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: sharing ctypes backing-owner snapshots across aggregate copies.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Assisted-by: Codex:GPT-6

youknowone · 2026-10-26T11:42:00Z

+
+    /// Allocate and track internal storage without running collection during
+    /// publication. The next ordinary tracked allocation still checks thresholds.
+    pub(crate) fn new_ref_without_collection(


the naming is confusing. new_ref_without_collection is default false call of new_ref_with_collection. those names are looking symmetric.

youknowone · 2026-10-26T15:53:02Z

+/// # Safety
+/// obj must be a valid pointer to an untracked PyObject with freshly zeroed `gc_bits`.
+#[inline]
+pub(crate) unsafe fn track_new_object_with_collection(obj: NonNull<PyObject>, collect: bool) {


let's give a good name here. I will think a bit more.

Assisted-by: Codex:GPT-6

youknowone

thanks!

1ndahous3 added 2 commits October 26, 2026 15:32

Share ctypes backing-owner snapshots between aggregate copies

986e6a3

Assisted-by: Codex:GPT-6

Limit ctypes snapshot helper to supported targets

332101a

Assisted-by: Codex:GPT-6

1ndahous3 force-pushed the ctypes_shared_owners branch from b23664e to 332101a Compare October 26, 2026 12:45

youknowone requested changes Oct 26, 2026

View reviewed changes

Clarify ctypes snapshot allocation helper names

f4df593

Assisted-by: Codex:GPT-6

youknowone approved these changes Oct 27, 2026

View reviewed changes

youknowone merged commit 415e472 into RustPython:main Oct 27, 2026
28 checks passed

youknowdot mentioned this pull request Oct 27, 2026

Integrate remaining Python 3.15 final runtime and libraries #8954

Draft

1ndahous3 deleted the ctypes_shared_owners branch October 28, 2026 07:34

Navigation Menu

Repository navigation

Share ctypes backing-owner snapshots between aggregate copies - #9392

Share ctypes backing-owner snapshots between aggregate copies#9392
youknowone merged 3 commits into
RustPython:mainfrom
1ndahous3:ctypes_shared_owners

1ndahous3 commented Oct 26, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

chatgpt-codex-connector Bot commented Oct 26, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 26, 2026 •

edited

Loading

Uh oh!

youknowone Oct 26, 2026

Uh oh!

youknowone Oct 26, 2026

Uh oh!

youknowone left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

1ndahous3 commented Oct 26, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Performance

AI assistance

Summary by CodeRabbit

Uh oh!

chatgpt-codex-connector Bot commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

coderabbitai Bot commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

youknowone Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

youknowone Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

youknowone left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

1ndahous3 commented Oct 26, 2026 •

edited by coderabbitai Bot

Loading

chatgpt-codex-connector Bot commented Oct 26, 2026 •

edited

Loading

coderabbitai Bot commented Oct 26, 2026 •

edited

Loading