Regression: nested exceptions crash (Cannot recover from stack overflow) #47805

Activity

1. devdanzin commented on Aug 14, 2008

   

   devdanzinmannequin

   on Aug 14, 2008

   MannequinAuthor

   More actions

   The following code works[1] on trunk and 2.5.1, but crashes with "Fatal
   Python error: Cannot recover from stack overflow," on py3k as of rev65676:

   # Python 3.0b2+ (py3k:65676, Aug 14 2008, 14:37:38)
   # [GCC 4.1.3 2007092 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2

   import sys
   
   def overflower():
       try:
           return overflower()
       except:
           return sys.exc_info()
   
   
   def f():
         try:
             return f()
         except:
             return overflower()
   
   f()
   ######

   Catching RuntimeError crashes, letting it be raised avoids the crash.
   Adding "finally: return overflower()" along with a non
   RuntimeError-catching except also gives a Fatal Python error.

   A smaller test case for hitting the overflow in py3k would be "def f():
   [...] except: return f()", but that hangs in an (desirable?) infinite
   loop in 2.5 and trunk.

   [1] "Works" as in "doesn't crash", but both the code above and the
   infinite loop hit bpo-2548 when run on a debug build of trunk. Calling
   overflower() alone in trunk hits the "undetected error" discussed in
   that issue, but works fine in py3k.
2. devdanzin

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   on Aug 14, 2008
3. pitrou commented on Aug 14, 2008

   

   pitrou

   on Aug 14, 2008

   Member

   More actions

   I'm no expert in recursion checking inside the Python interpreter, but
   looking at the code for _Py_CheckRecursiveCall(), I don't think it is a
   bug but a feature.

   Here how I understand it. When the recursion level exceeds the normal
   recursion limit (let's call the latter N), a RuntimeError is raised and
   the normal recursion check is temporarily disabled (by setting
   tstate->overflowed) so that Python can run some recovery code (e.g. an
   except statement with a function call to log the problem), and another
   recursion check is put in place that is triggered at N+50. When the
   latter check triggers, the interpreter prints the aforementioned
   Py_FatalError and bails out.

   This is actually what happens in your example: when the normal recursion
   limit is hit and a RuntimeError is raised, you immediately catch the
   exception and run into a second infinite loop while the normal recursion
   check is temporarily disabled: the N+50 check then does its job.

   Here is a simpler way to showcase this behaviour, without any nested
   exceptions:

   def f():
       try:
           return f()
       except:
           pass
       f()
   
   f()

   Can someone else comment on this?
4. devdanzin commented on Aug 14, 2008

   

   devdanzinmannequin

   on Aug 14, 2008

   MannequinAuthor

   More actions

   Antoine,

   Thanks for your analysis. I still believe this is a regression for the
   case described, but take my opinion with a grain of salt :)

   > looking at the code for _Py_CheckRecursiveCall(), I don't think it
   > is a bug but a feature.

   It does seem to be working as designed, if that is a desirable behavior
   then this issue should be closed.

   This is actually what happens in your example: when the normal
   recursion limit is hit and a RuntimeError is raised, you
   immediately catch the exception and run into a second infinite
   loop while the normal recursion check is temporarily disabled:
   the N+50 check then does its job.

   Except that it wasn't an infinite loop in 2.5 and isn't in trunk: it
   terminates on overflower's except. That's why I think this is a
   regression. Besides being different behavior, it seems weird to bail out
   on a recursion issue instead of dealing with it.

   Your showcase is a better way of getting an infinite loop in trunk than
   the one I mentioned, but AFAIK we are more comfortable with infinite
   loops than with fatal errors.
5. pitrou commented on Aug 14, 2008

   

   pitrou

   on Aug 14, 2008

   Member

   More actions

   Except that it wasn't an infinite loop in 2.5 and isn't in trunk: it
   terminates on overflower's except.

   That's because the stated behaviour is only implemented in 3.0 and not
   in 2.x. I'm not sure what motivated it, but you are the first one to
   complain about it. If you think it is a regression, I think you should
   open a thread on the python-dev mailing-list about it.
6. devdanzin commented on Aug 22, 2008

   

   devdanzinmannequin

   on Aug 22, 2008

   MannequinAuthor

   More actions

   Antoine,
   All the cases I could find would be more "test" than "use" cases. Given
   that most ways to abort I find in 3.0 are related to "undetected error"s
   in trunk, I'm almost convinced that 3.0 is right here :)

   My last worry is that it'd be kinda easy to get Fatal errors from
   sane-ish functions and deeply nested input:
   ============

   class rec:
       def __str__(self):
           return str(self)
   
   def overflower(x):
       try:
           return overflower(x)
       except:
           print (x)
   
   list_rec = [100000000001]
   for _ in range(12): list_rec = [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[
   [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[list_rec]]]]]]]]]]]]]]]]]]]]]]]]]
   ]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]
   
   str_rec = rec()
   
   overflower(1) # OK
   overflower(list_rec) # Aborts
   overflower(str_rec) # Aborts

   ============

   Thanks for the feedback!
   Attached is a file that shows how trunk is doing something weird when it
   works (besides the other reported issues that arise from that).
7. kaizhu commented on Nov 17, 2009

   

   kaizhumannequin

   on Nov 17, 2009

   Mannequin

   More actions

   just submitted a nearly identical bug (bpo-7338) b4 checking for this one.
   so u think it works correctly up to the 2nd N+50 check. can someone
   give a reason y we should crash after that instead of throwing a
   fallback RuntimeError?
8. terryjreedy commented on Dec 24, 2011

   

   terryjreedy

   on Dec 24, 2011

   Member

   More actions

   This is one of four essentially duplicate issues bpo-6028, bpo-7338, bpo-13644.
   bpo-6028 has a proposed patch.
9. terryjreedy

   closed this as completedon Dec 24, 2011
10. ezio-melotti

    transferred this issue fromon Apr 10, 2022