Skip to content

Make release and debug ABI compatible #80646

Closed
Closed
Make release and debug ABI compatible#80646
Labels
3.8 (EOL)end of lifebuildThe build process and cross-build

Description

@vstinner
vstinner
opened on Mar 28, 2019
BPO 36465
Nosy @pitrou, @vstinner
PRs
  • WIP: bpo-36465: Remove Py_TRACE_REFS special build #12614
  • bpo-36465: Make release and debug ABI compatible #12615
  • bpo-36465: Fix test_regrtest on Windows #12945
  • bpo-39647: Update doc of init_config.rst #18520
  • bpo-43688: Support the limited C API in debug mode #25131
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2019-04-24.22:58:10.873>
created_at = <Date 2019-03-28.23:01:42.956>
labels = ['build', '3.8']
title = 'Make release and debug ABI compatible'
updated_at = <Date 2021-04-02.13:45:45.078>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-04-02.13:45:45.078>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-04-24.22:58:10.873>
closer = 'vstinner'
components = ['Build']
creation = <Date 2019-03-28.23:01:42.956>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 36465
keywords = ['patch']
message_count = 9.0
messages = ['339079', '339080', '339081', '340746', '340750', '340812', '340813', '362126', '390064']
nosy_count = 2.0
nosy_names = ['pitrou', 'vstinner']
pr_nums = ['12614', '12615', '12945', '18520', '25131']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue36465'
versions = ['Python 3.8']

    Activity

    1. vstinner commented on Mar 28, 2019

      @vstinner
      vstinner
      on Mar 28, 2019
      MemberAuthor

      When Python is built in debug mode, PyObject gets 2 new fields: _ob_prev and _ob_next. These fields change the offset of following fields in the PyObject structure and so breaks the ABI.

      I propose to modify the debug build (Py_DEBUG) to not imply Py_TRACE_REFS anymore. Antoine Pitrou proposed this idea when the C API was discussed to get a stable ABI.

      Another more radical idea is to completely remove Py_TRACE_REFS special build.

    2. vstinner
      added
      3.8 (EOL)end of life
      buildThe build process and cross-build
      on Mar 28, 2019

    3. vstinner commented on Mar 28, 2019

      @vstinner
      vstinner
      on Mar 28, 2019
      MemberAuthor

      Another more radical idea is to completely remove Py_TRACE_REFS special build.

      I wrote PR 12614 to implement this idea. I wrote it to see which code depends on it:

      commit 6350949 (HEAD -> remove_trace_refs, origin/remove_trace_refs)
      Author: Victor Stinner <[email protected]>
      Date: Thu Mar 28 23:26:58 2019 +0100

      WIP: bpo-36465: Remove Py_TRACE_REFS special build

Remove _ob_prev and _ob_next fields of PyObject when Python is
compiled in debug mode to make debug ABI closer to the release ABI.

Remove:

* sys.getobjects()
* PYTHONDUMPREFS environment variable
* _PyCoreConfig.dump_refs
* PyObject._ob_prev and PyObject._ob_next fields
* _PyObject_HEAD_EXTRA and _PyObject_EXTRA_INIT macros
* _Py_AddToAllObjects()
* _Py_PrintReferenceAddresses()
* _Py_PrintReferences()
* _Py_ForgetReference(op) is replaced with _Py_INC_TPFREES(op)

      I never used PYTHONDUMPREFS. I just tried in Python 3.7: Python does crash with this option... so this option doesn't sound popuplar. Otherwise, many users would complain.

      $ PYTHONDUMPREFS=1 python3.7-dbg -c pass
...
0x7f7eae14aa90 [1] 'Thread-local dummy'
0x7f7eae19b448 [1] (<class '_thread._localdummy'>, <class 'object'>)
0x7f7eae14aa30 [1] {'__doc__': 'Thread-local dummy'}
0x7f7eae1356d8 [1] (<class 'object'>,)
0x7d0940 [2] <class '_thread._localdummy'>
0x7f7eae120d58 [1] Segmentation fault (core dumped)

      I never used sys.getobjects() neither, but I can imagine that someone might want to use for very specific needs. So maybe it's safer to not immediately remove the feature. At least, a deprecation period would be needed.

      I suggest to reject PR 12614 and not remove Py_TRACE_REFS.

    4. vstinner commented on Mar 28, 2019

      @vstinner
      vstinner
      on Mar 28, 2019
      MemberAuthor

      PR 12615 changes Py_DEBUG to no longer imply Py_TRACE_REFS. IMHO it's a more reasonable approach.

      I'm not sure if it's enough to magically get exactly the same ABI than Python built in release mode.

    5. vstinner commented on Apr 23, 2019

      @vstinner
      vstinner
      on Apr 23, 2019
      MemberAuthor

      Since my goal here is to support a C extension compiled in release mode on a Python compiled in debug mode, Py_TRACE_REFS should be a new separated ABI since it modify Py_INCREF/Py_DECREF. Maybe a new configure option should be added to opt-in for Py_TRACE_REFS and add "t" (T stands for Trace references) to SOABI.

    6. vstinner commented on Apr 24, 2019

      @vstinner
      vstinner
      on Apr 24, 2019
      MemberAuthor

      I started a thread on python-dev to discuss this issue:
      "[Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode"
      https://mail.python.org/pipermail/python-dev/2019-April/157178.html

      It's a follow-up of my previous thread:
      "[Python-Dev] No longer enable Py_TRACE_REFS by default in debug build"
      https://mail.python.org/pipermail/python-dev/2019-April/157015.html

    7. vstinner
      changed the title [-]No longer enable Py_TRACE_REFS by default in debug build[/-] [+]Make release and debug ABI compatible[/+] on Apr 24, 2019

    8. vstinner commented on Apr 24, 2019

      @vstinner
      vstinner
      on Apr 24, 2019
      MemberAuthor

      New changeset f4e4703 by Victor Stinner in branch 'master':
      bpo-36465: Make release and debug ABI compatible (GH-12615)
      f4e4703

    9. vstinner
      closed this as completedon Apr 24, 2019

    10. vstinner commented on Apr 24, 2019

      @vstinner
      vstinner
      on Apr 24, 2019
      MemberAuthor

      New changeset 235e7b2 by Victor Stinner in branch 'master':
      bpo-36465: Fix test_regrtest on Windows (GH-12945)
      235e7b2

    11. vstinner commented on Feb 17, 2020

      @vstinner
      vstinner
      on Feb 17, 2020
      MemberAuthor

      New changeset a784759 by Hai Shi in branch 'master':
      bpo-36465: Update doc of init_config.rst (GH-18520)
      a784759

    12. vstinner commented on Apr 2, 2021

      @vstinner
      vstinner
      on Apr 2, 2021
      MemberAuthor

      New changeset 3359cab by Victor Stinner in branch 'master':
      bpo-43688: Support the limited C API in debug mode (GH-25131)
      3359cab

    13. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.8 (EOL)end of lifebuildThe build process and cross-build

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions