Skip to content

Recursive calls crash interpreter when checking exceptions #86675

Closed
Closed
Recursive calls crash interpreter when checking exceptions#86675
Labels
3.10 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)type-crashA hard crash of the interpreter, possibly with a core dump

Description

@xxm
xxm
mannequin
opened on Nov 30, 2020
BPO 42509
Nosy @ronaldoussoren, @markshannon, @sweeneyde

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-11-30.14:24:29.400>
created_at = <Date 2020-11-30.06:01:25.952>
labels = ['interpreter-core', '3.10', 'type-crash']
title = 'Recursive calls crash interpreter when checking exceptions'
updated_at = <Date 2020-11-30.14:24:29.398>
user = 'https://bugs.python.org/xxm'

bugs.python.org fields:

activity = <Date 2020-11-30.14:24:29.398>
actor = 'Mark.Shannon'
assignee = 'none'
closed = True
closed_date = <Date 2020-11-30.14:24:29.400>
closer = 'Mark.Shannon'
components = ['Interpreter Core']
creation = <Date 2020-11-30.06:01:25.952>
creator = 'xxm'
dependencies = []
files = []
hgrepos = []
issue_num = 42509
keywords = []
message_count = 6.0
messages = ['382111', '382112', '382114', '382117', '382128', '382151']
nosy_count = 4.0
nosy_names = ['ronaldoussoren', 'Mark.Shannon', 'Dennis Sweeney', 'xxm']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue42509'
versions = ['Python 3.10']

Activity

  1. xxm commented on Nov 30, 2020

    @xxm
    xxmmannequin
    on Nov 30, 2020
    MannequinAuthor

    The following program 1 can crash in Python 3. We have reproduce it in the Python version 3.5, 3.6, 3.7, 3.8, 3.9, 3.10. This bug seems to be similar to bpo-36272, however, the tracking system shows bpo-36272 has been fixed and the program 2, which triggers bpo-36272, will not cause crash (“core dump”) in Python 3.8, 3.9, 3.10.
    We have attached the stack trace in the end of this report.

    Program 1:
    ========================

    import logging

def rec():
	try:
		logging.error("foo")
	except:
		pass
	rec()
rec()

    ========================

    Program 2:
    ========================

    import logging

def rec():
	logging.error("foo")
	rec()
rec()

    ======================
    The error message is like following:
    “
    ERROR:root:foo
    ERROR:root:foo
    ERROR:root:foo
    ERROR:root:foo
    …
    ERROR:root:foo
    ERROR:root:foo
    ERROR:root:foo
    Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow.
    Python runtime state: initialized

    Current thread 0x00007f0fa440b700 (most recent call first):
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 420 in usesTime
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 643 in usesTime
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 675 in format
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 938 in format
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 1094 in emit
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 963 in handle
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 1673 in callHandlers
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 1611 in handle
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 1601 in _log
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 1483 in error
    File "/usr/local/python310/lib/python3.10/logging/init.py", line 2080 in error
    File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 8 in rec
    File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec
    File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec
    File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec
    ….
    File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec
    File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec
    File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec
    File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec
    File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec
    ...
    Aborted (core dumped)”

  2. xxm
    added
    3.10 (EOL)end of life
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Nov 30, 2020

  3. sweeneyde commented on Nov 30, 2020

    @sweeneyde
    sweeneyde
    on Nov 30, 2020
    Member

    This might be the expected behavior. See https://bugs.python.org/issue25222

    If you already caught a RecursionError and you keep recursing anyway, once you go 50 levels beyond sys.getrecursionlimit(), the interpreter crashes regardless of what is excepted. In /Python/ceval.c, there's this:

        if (tstate->overflowed) {
        if (tstate->recursion_depth > recursion_limit + 50) {
            /* Overflowing while handling an overflow. Give up. */
            Py_FatalError("Cannot recover from stack overflow.");
        }
        return 0;
    }

    In your Program 2, when the interpreter raises a RecursionError, it is raised normally and everything is fine.

    In your Program 1, when the interpreter raises a RecursionError, it is excepted, so the interpreter thinks it's okay to keep going, and when it does, it raises more RecursionErrors, which it keeps excepting, until it finally can't go any farther ( > 50 + sys.getrecursionlimit()), and has no option but to crash.

    "Cannot recover from stack overflow." seems to make sense to me: when the interpreter tries to recover, the code won't let it.

  4. ronaldoussoren commented on Nov 30, 2020

    @ronaldoussoren
    ronaldoussoren
    on Nov 30, 2020
    Contributor

    See also bpo-42500

  5. xxm commented on Nov 30, 2020

    @xxm
    xxmmannequin
    on Nov 30, 2020
    MannequinAuthor

    But program like following program 3 will not cause any core dump, RecursionError is also being caught in this Recursion.
    program 3
    def rec():
    try:
    rec()
    except:
    pass
    rec()

    Beside,I use sys.setrecursionlimit(80), and the program 1 still cause core dump.I print sys.getrecursionlimit(),the value is 1000. 80 is << 50 +1000, it shouldn't cause core dump.

  6. sweeneyde commented on Nov 30, 2020

    @sweeneyde
    sweeneyde
    on Nov 30, 2020
    Member

    sys.getrecursionlimit() returns whatever was passed to the most recent call of sys.setrecursionlimit(...), with some system default (here 1000).

    Catching a RecursionError might be fine sometimes, but the issue is that Program 1 catches a RecursionError *and then keeps recursing more* rather than stopping.

    I think it might have to be the responsibility of the Python user to make sure that if a RecursionError is to be caught, that the program can recover without making things much worse. It's my understanding that the extra buffer of +50 is to make sure that the programmer has room to stop the overflow and do any necessary cleanup [1].

    If no attempt is made to clean up, then it seems reasonable to me that Python should crash, unless there's some idea of what could happen that I'm missing. The interpreter could allow arbitrary recursion during the cleanup until the C stack overflows, but that sort of defeats the point of the recursion checker. It could raise some new ExtraSuperRecurionError, but that doesn't fix anything: what if *that* error is caught ;) ?

    [1] to get back to a recursion depth lower than some lower threshold: https://github.com/python/cpython/blob/master/Include/internal/pycore_ceval.h#L98

  7. markshannon commented on Nov 30, 2020

    @markshannon
    markshannon
    on Nov 30, 2020
    Member

    Duplicate of 42500

  8. markshannon
    closed this as completedon Nov 30, 2020
  9. markshannon
    closed this as completedon Nov 30, 2020
  10. ezio-melotti
    transferred this issue fromon Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.10 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)type-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions