Skip to content

Fix possible SIGSGV when asyncio.Future is created in __del__ #77804

New issue
New issue
Closed
Closed
Fix possible SIGSGV when asyncio.Future is created in __del__#77804
Labels
3.7 (EOL)end of life3.8 (EOL)end of liferelease-blockertopic-asynciotype-crashA hard crash of the interpreter, possibly with a core dump

Description

@1st1
1st1
opened on May 23, 2018
BPO 33623
Nosy @vstinner, @ned-deily, @asvetlov, @serhiy-storchaka, @1st1, @miss-islington, @pbasista
PRs
  • bpo-33623: Fix possible SIGSGV when asyncio.Future is created in __del__ #7080
  • [3.7] bpo-33623: Fix possible SIGSGV when asyncio.Future is created in __del__ (GH-7080) #7151
  • [3.6] bpo-33623: Fix possible SIGSGV when asyncio.Future is created in __del__ (GH-7080) #7152
    		•
    Files
  • uvloop-test.gdb-backtrace.txt: GDB backtrace showing the assertion failure while using uvloop
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-05-29.15:16:35.878>
created_at = <Date 2018-05-23.18:55:46.468>
labels = ['3.7', '3.8', 'release-blocker', 'type-crash', 'expert-asyncio']
title = 'Fix possible SIGSGV when asyncio.Future is created in __del__'
updated_at = <Date 2018-05-29.15:16:35.877>
user = 'https://github.com/1st1'

    bugs.python.org fields:

    activity = <Date 2018-05-29.15:16:35.877>
actor = 'yselivanov'
assignee = 'none'
closed = True
closed_date = <Date 2018-05-29.15:16:35.878>
closer = 'yselivanov'
components = ['asyncio']
creation = <Date 2018-05-23.18:55:46.468>
creator = 'yselivanov'
dependencies = []
files = ['47619']
hgrepos = []
issue_num = 33623
keywords = ['patch']
message_count = 12.0
messages = ['317437', '317440', '317441', '317444', '317602', '317658', '317858', '317863', '317864', '317868', '317930', '318026']
nosy_count = 7.0
nosy_names = ['vstinner', 'ned.deily', 'asvetlov', 'serhiy.storchaka', 'yselivanov', 'miss-islington', 'pbasista']
pr_nums = ['7080', '7151', '7152']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue33623'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

    Activity

    1. 1st1 commented on May 23, 2018

      @1st1
      1st1
      on May 23, 2018
      MemberAuthor

      Originally reported in MagicStack/uvloop#143

      Future.__init__ shouldn't try to capture the current traceback if the interpreter is being finalized.

    2. 1st1
      added
      3.7 (EOL)end of life
      3.8 (EOL)end of life
      topic-asyncio
      type-crashA hard crash of the interpreter, possibly with a core dump
      on May 23, 2018

    3. serhiy-storchaka commented on May 23, 2018

      @serhiy-storchaka
      serhiy-storchaka
      on May 23, 2018
      Member

      Is the problem because traceback_extract_stack is NULL?

      But this will not fix the problem completely. For example calling repr() on a future can crash because of asyncio_future_repr_info_func == NULL. And many other operations use globals cleared in module_free().

    4. 1st1 commented on May 23, 2018

      @1st1
      1st1
      on May 23, 2018
      MemberAuthor

      Is the problem because traceback_extract_stack is NULL?

      It's not NULL, it's just a broken reference at that point.

    5. serhiy-storchaka commented on May 23, 2018

      @serhiy-storchaka
      serhiy-storchaka
      on May 23, 2018
      Member

      Then we should find what callable is NULL and fix the place where it is called. _PyObject_FastCallDict() should never be called with NULL.

    6. 1st1 commented on May 24, 2018

      @1st1
      1st1
      on May 24, 2018
      MemberAuthor

      Then we should find what callable is NULL and fix the place where it is called. _PyObject_FastCallDict() should never be called with NULL.

      My understanding is that the interpreter is being shutdown and half of the objects are freed. We're still holding a reference to *something* in Python space and try calling it. The obvious fix for that is simply avoid capturing tracebacks if a Future object is created during finalization of the interpreter (it's pointless to capture it anyways at that point).

    7. serhiy-storchaka commented on May 25, 2018

      @serhiy-storchaka
      serhiy-storchaka
      on May 25, 2018
      Member

      It is okay if the fact that half of the objects are freed leads to raising exceptions. But an assertion failure is sign of bugs in Python core or the _asyncio module. PR 7080 removes one way of exposing these bugs, but bugs itself still are here, and there may be other ways of triggering a crash.

      Can this crash be reproduced without uvloop? What is the backtrace of the assertion failure?

    8. pbasista commented on May 28, 2018

      @pbasista
      pbasistamannequin
      on May 28, 2018
      Mannequin

      Can this crash be reproduced without uvloop?
      I am not aware of a way to reproduce this without uvloop.

      What is the backtrace of the assertion failure?
      Without uvloop, the following exception is raised:

      Exception ignored in: <bound method LoopWrapper.__del__ of <__main__.LoopWrapper object at 0x7fe14ec569b0>>
Traceback (most recent call last):
  File "uvloop_test.py", line 13, in __del__
  File "/usr/lib/python3.6/asyncio/base_events.py", line 276, in create_future
AttributeError: 'NoneType' object has no attribute 'Future'

      With uvloop and with Python compiled with --with-pydebug configure option, the failed assertion is mentioned in the original uvloop bug report. In particular, this assertion fails:

      python3.6-dbg: ../Objects/abstract.c:2300: _PyObject_FastCallDict: Assertion `func != NULL' failed.

      A full GDB backtrace is attached.

    9. 1st1 commented on May 28, 2018

      @1st1
      1st1
      on May 28, 2018
      MemberAuthor

      There's nothing specific here to uvloop except the fact that it was compiled with Cython, so module finalization/GC pattern is slightly different to that of vanilla asyncio.

      Serhiy, I'm merging this PR since I'd like to avoid having segfaults in 3.7.0. Feel free to debug this further.

    10. 1st1 commented on May 28, 2018

      @1st1
      1st1
      on May 28, 2018
      MemberAuthor

      New changeset 35230d0 by Yury Selivanov in branch 'master':
      bpo-33623: Fix possible SIGSGV when asyncio.Future is created in __del__ (bpo-7080)
      35230d0

    11. miss-islington commented on May 28, 2018

      @miss-islington
      miss-islington
      on May 28, 2018
      Contributor

      New changeset 51d0a2c by Miss Islington (bot) in branch '3.7':
      bpo-33623: Fix possible SIGSGV when asyncio.Future is created in __del__ (GH-7080)
      51d0a2c

    12. 1st1 commented on May 28, 2018

      @1st1
      1st1
      on May 28, 2018
      MemberAuthor

      Would be great to merge this in 3.7.0. The change is super safe to merge.

    13. 1st1
      added
      release-blocker
      on May 28, 2018

    14. vstinner commented on May 29, 2018

      @vstinner
      vstinner
      on May 29, 2018
      Member

      Would be great to merge this in 3.7.0. The change is super safe to merge.

      The 3.7rc1 is not supposed to be tagged on the current 3.7 branch?

    15. 1st1
      closed this as completedon May 29, 2018
    16. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.7 (EOL)end of life3.8 (EOL)end of liferelease-blockertopic-asynciotype-crashA hard crash of the interpreter, possibly with a core dump

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions