Display exceptions' subclasses in help()

@birkenfeld

BPO	8525
Nosy	@birkenfeld, @terryjreedy, @ncoghlan, @abalkin, @merwok, @briancurtin, @serhiy-storchaka, @eric-wieser, @CuriousLearner, @tirkarthi
PRs	bpo-8525: help() on a type now shows builtin subclasses #5066
Files	pydoc.py: pydoc.py with enhancement to help() help_8525.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/CuriousLearner'
closed_at = <Date 2018-10-21.07:27:06.464>
created_at = <Date 2010-04-24.23:10:00.456>
labels = ['3.8', 'type-feature', 'library']
title = "Display exceptions' subclasses in help()"
updated_at = <Date 2018-12-29.15:22:31.762>
user = 'https://bugs.python.org/robcliffe'

bugs.python.org fields:

activity = <Date 2018-12-29.15:22:31.762>
actor = 'serhiy.storchaka'
assignee = 'CuriousLearner'
closed = True
closed_date = <Date 2018-10-21.07:27:06.464>
closer = 'ncoghlan'
components = ['Library (Lib)']
creation = <Date 2010-04-24.23:10:00.456>
creator = 'robcliffe'
dependencies = []
files = ['17075', '19748']
hgrepos = []
issue_num = 8525
keywords = ['patch']
message_count = 26.0
messages = ['104134', '104656', '104657', '104658', '121940', '122044', '122057', '122115', '122116', '122118', '122193', '123167', '123169', '123170', '123228', '288243', '309296', '321651', '321675', '322226', '322240', '322287', '322288', '328194', '328195', '332721']
nosy_count = 14.0
nosy_names = ['georg.brandl', 'terry.reedy', 'ncoghlan', 'belopolsky', 'ron_adam', 'eric.araujo', 'brian.curtin', 'robcliffe', 'henriquebastos', 'Rodolpho.Eckhardt', 'serhiy.storchaka', 'Eric.Wieser', 'CuriousLearner', 'xtreak']
pr_nums = ['5066']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue8525'
versions = ['Python 3.8']

help() on an exception class lists the method resolution order, which is in effect the class inheritance hierarchy. E.g. help(ArithmeticError) lists ArithmeticError, StandardError, Exception, BaseException, __builtin__.object. It struck me it would help to find my way around if it also listed the builtin SUBclasses (if any). Something like:
Built-in subclasses:
FloatingPointError
OverflowError
ZeroDivisionError
In fact why not do it for any class, not just exceptions?
I attach a patched version of pydoc.py - tested but only on my PC which is running Python 2.5 under Windows XP. I have added lines 1129-1148 to the docclass method of the TextDoc class (and flagged them # [RAC] ).
(I don't pretend to understand the magic where __builtins__ is a dictionary when pydoc.py is run but becomes a module later on. Never mind - the patch works (I believe).)
For consistency, a similar patch would also have to be made to the docclass nethod of the HTMLDoc class (which outputs HTML rather than plain text). I have not attempted this as I don't know how it is called and hence how to test any patch, but it should be straightforward for anyone with the know-how.

2.5 is frozen. 2.6 and 3.1 are in bug-fix mode only, and 2.7 is in beta so new features are unlikely there. So I recommend doing a patch against 3.1 or even the 3.2 trunk. And of course, make sure this proposal makes sense for 3.x.

Thanks for your work, Rob. To get reviews and comments, you’ll need to submit a patch (a diff) instead of the whole file. This makes it easier to see your changes than looking for a special comment :)

This short doc should contain all the information you’ll need: http://www.python.org/dev/patches

As Terry said, 2.6 and 3.1 only get bug fixes, 2.7 and 3.2 are in beta stage and don’t get new features, so you’ll want to port your changes to the py3k branch (the trunk for the 3.x series).

Regards

Minor correction to the last comment: 3.2 is not in beta nor feature freeze.

Migrated the patch to the 3.x trunk and added three tests.

Thank you. I uploaded your patch to Rietveld and reviewed it: http://codereview.appspot.com/3169042/

The following passes tests in elp_8525.patch, but is much simpler:

===================================================================

--- Lib/pydoc.py	(revision 86600)
+++ Lib/pydoc.py	(working copy)
@@ -1139,6 +1139,15 @@
                 push('    ' + makename(base))
             push('')
 
+        # List the built-in subclasses, if any:
+        subclasses = [cls.__name__ for cls in object.__subclasses__()
+                      if cls.__module__ == 'builtins']
+        if subclasses:
+            push("Built-in subclasses:")
+            for subclassname in sorted(subclasses):
+                push('    ' + subclassname)
+            push('')
+
         # Cute little class to pump out a horizontal rule between sections.
         class HorizontalRule:
             def __init__(self):

Thanks for your work. Glad if I have made a contribution to Python,
however small.
Rob Cliffe

On 22/11/2010 00:26, Éric Araujo wrote:

Éric Araujo<[email protected]> added the comment:

Thank you. I uploaded your patch to Rietveld and reviewed it: http://codereview.appspot.com/3169042/

----------

Python tracker<[email protected]>
<http://bugs.python.org/issue8525\>

I would not be at all surprised if my patch could be simplified (in fact
I'd be surprised if it couldn't).
However, I did try out your version on Python 2.5 specifically, and it
did not work for me.
Trying it out on help(Exception), the relevant members of
object.__subclasses__() viz.
<type 'exceptions.StandardError'>, <type 'exceptions.StopIteration'> etc.
had a __module__attribute which equalled 'exceptions', not 'builtins'.
Best wishes
Rob Cliffe

On 22/11/2010 01:33, Alexander Belopolsky wrote:

Alexander Belopolsky<[email protected]> added the comment:

The following passes tests in elp_8525.patch, but is much simpler:

===================================================================
--- Lib/pydoc.py (revision 86600)
+++ Lib/pydoc.py (working copy)
@@ -1139,6 +1139,15 @@
push(' ' + makename(base))
push('')
   # List the built-in subclasses, if any:
   subclasses = [cls.\_\_name__ for cls in object.\_\_subclasses__()
                 if cls.\_\_module__ == 'builtins']
   if subclasses:
       push("Built-in subclasses:")
       for subclassname in sorted(subclasses):
           push('    ' + subclassname)
       push('')
    # Cute little class to pump out a horizontal rule between sections.
    class HorizontalRule:
        def \_\_init__(self):
----------
nosy: +belopolsky

Python tracker<[email protected]>
<http://bugs.python.org/issue8525\>

New features can only go into 3.2, so you have to test with an updated checkout of the Subversion branch named py3k. See the link I have in a previous message.

(P.S. Would you be so kind as to edit quoted text out of your replies? It’s unnecessary noise. Thanks in advance.)

The proposal is to display builtin subclasses as for example:

>>> help(ArithmeticError)
class ArithmeticError(Exception)
 |  Base class for arithmetic errors.
 |  
 |  Method resolution order:
 |      ArithmeticError
 |      Exception
 |      BaseException
 |      object
 |  
 |  Built-in subclasses:
 |      FloatingPointError
 |      OverflowError
 |      ZeroDivisionError

Note that this really affects only exceptions because no other builtin class has builtin subclasses. (dict has subclasses in collections, but not in builtins.)

Exception hierarchy is presented in the reference manual at

http://docs.python.org/dev/library/exceptions.html?#exception-hierarchy

I wonder if rather than having MRO and subclasses sections, we should just present a portion of the exception hierarchy including the given exception, all its bases and direct subclasses:

object
|
BaseException
|
Exception
|
*ArithmeticError*
|
+-- FloatingPointError
+-- OverflowError
+-- ZeroDivisionError

Reviewing the builtins in 3.7, I get the following results for builtin objects that have defined subclasses immediately after interpreter startup:

=============

>>> for name, obj in vars(builtins).items():
...     if isinstance(obj, type) and name in str(obj):
...         subclasses = type(obj).__subclasses__(obj)
...         if subclasses:
...             print(f"{obj}: {len(subclasses)}")
... 
<class 'classmethod'>: 1
<class 'dict'>: 1
<class 'property'>: 1
<class 'int'>: 1
<class 'object'>: 132
<class 'staticmethod'>: 1
<class 'tuple'>: 16
<class 'type'>: 1
<class 'BaseException'>: 4
<class 'Exception'>: 19
<class 'ImportError'>: 2
<class 'OSError'>: 13
<class 'RuntimeError'>: 3
<class 'NameError'>: 1
<class 'SyntaxError'>: 1
<class 'IndentationError'>: 1
<class 'LookupError'>: 3
<class 'ValueError'>: 2
<class 'UnicodeError'>: 3
<class 'ArithmeticError'>: 3
<class 'SystemError'>: 1
<class 'Warning'>: 10
<class 'ConnectionError'>: 4

=============

So rather than special-casing exceptions or builtins in general, my inclination would be to include a section that lists up to 4 subclasses inline, and then adds a "... and NNN additional subclasses" trailing when there are more than 4 (or when there are less than 4 subclasses with public names, but additional private subclasses).

So in a class like "int", for example, we'd see:

Currently imported subclasses:
    bool

While in a class like OSError we'd see:

Currently imported subclasses:
    BlockingIOError
    ChildProcessError
    ConnectionError
    FileExistsError
    ... and 9 other subclasses

And in "help(object)" we'd see something like:

Currently imported subclasses:
    async_generator
    BaseException
    builtin_function_or_method
    bytearray
    ... and 215 other subclasses

The initial list of subclasses to show would be restricted to public builtins: sorted((str(cls) for cls in object.__subclasses__() if not cls.__name__.startswith("_") and cls.__module__ == "builtins"), key=str.lower)

If that gives less then four names, then the list of names would be expanded to subclasses with public names in public modules (i.e. neither __qualname__ nor __module__ starts with "_", neither of those contains "._", and qualname doesn't contain ".<locals>.").

The full count of currently imported subclasses would ignore whether the subclass names were public or not.

On 14/07/2018 13:44, Nick Coghlan wrote:

Nick Coghlan <[email protected]> added the comment:

Reviewing the builtins in 3.7, I get the following results for builtin objects that have defined subclasses immediately after interpreter startup:

=============
>>> for name, obj in vars(builtins).items():
.. if isinstance(obj, type) and name in str(obj):
.. subclasses = type(obj).__subclasses__(obj)
.. if subclasses:
.. print(f"{obj}: {len(subclasses)}")
..
<class 'classmethod'>: 1
<class 'dict'>: 1
<class 'property'>: 1
<class 'int'>: 1
<class 'object'>: 132
<class 'staticmethod'>: 1
<class 'tuple'>: 16
<class 'type'>: 1
<class 'BaseException'>: 4
<class 'Exception'>: 19
<class 'ImportError'>: 2
<class 'OSError'>: 13
<class 'RuntimeError'>: 3
<class 'NameError'>: 1
<class 'SyntaxError'>: 1
<class 'IndentationError'>: 1
<class 'LookupError'>: 3
<class 'ValueError'>: 2
<class 'UnicodeError'>: 3
<class 'ArithmeticError'>: 3
<class 'SystemError'>: 1
<class 'Warning'>: 10
<class 'ConnectionError'>: 4
=============

So rather than special-casing exceptions or builtins in general, my inclination would be to include a section that lists up to 4 subclasses inline, and then adds a "... and NNN additional subclasses" trailing when there are more than 4 (or when there are less than 4 subclasses with public names, but additional private subclasses).

My 2 cents:
To use Exceptions optimally (e.g. to make the errors you trap
neither too specific nor too general), you often need to know (and
understand) the relevant part of the Exception hierarchy. In particular
you may know the name of an Exception that covers a particular use case,
but not the names of its subclasses, one of which might be more
appropriate. Exceptions are exceptional* in that the "issubclass"
relationship is vital to the way that they work. So it is USEFUL to
know ALL subclasses of a given Exception class (not just 4; in practice
there won't be more than a dozen or two). I have found myself in just
this position; in fact it impelled me to adding a "show subclasses"
feature to help(<anyExceptionClass>) in my then current version of
Python 2. (An alternative might be a handy way of showing the complete
built-in Exception hierarchy.)

I question whether it is really useful to know all subclasses of ANY
class, or whether YAGNI. I think that, for non-Exception classes,
typically when you look at a class you may want to know its inheritance
(to understand its functionality better), but it is rare that you will
want to know what subclasses it has, if any. No doubt there are
exceptions* (perhaps you monkey-patch a class and want to know what
subclasses might be affected).
Regards
Rob Cliffe

Pun not intended

I get the following results for builtin objects that have defined subclasses

From that list, the only unhelpful ones with > 4 items, in my opinion, appear to be object, since that just tells you every type that exists, and tuple, because that lists every single namedtuple.

So it is USEFUL to know ALL subclasses of a given Exception class

I agree with this - most of the value here comes from showing the full set of exceptions. If we don't do that, we should probably point the user to calling cls.__subclasses__() so they can inspect the full list

Hi,

My perception with all the discussion and WIP patch is that we can ideally limit the no. of subclasses shown only for object, and not for any other class.

From that list, the only unhelpful ones with > 4 items, in my opinion, appear to be object, since that just tells you every type that exists, and tuple, because that lists every single namedtuple.

So it is USEFUL to know ALL subclasses of a given Exception class

+1

I agree with this - most of the value here comes from showing the full set of exceptions. If we don't do that, we should probably point the user to calling cls.__subclasses__() so they can inspect the full list

I agree with this. I would propose to only limit to 4 classes for subclasses of object and for everything else displaying all the subclasses. We can optionally display the message to call cls.__subclasses() in the case when help(object) is called.

How does it sound?

This is a UX problem folks: dynamic unbounded lists are inherently problematic in a line based information display.

For methods on a class, there's the natural constraint that broad class APIs are a problem for a variety of reasons, but there's no such design constraint for exception hierarchies.

In a Fedora Python 3.6 shell (with the Fedora abrt exception handling hook loaded), Exception has 31 subclasses, and OSError has 17.

In a 3.6 shell venv (with no Fedora abrt hook), those numbers are 20 and 13.

But if you import assorted Django modules then the number of Exception subclasses grows significantly, while the number of OSError subclasses grows slightly:

>>> import django
>>> len(list(pkgutil.iter_modules(django.__path__)))
17
>>> len(Exception.__subclasses__())
76
>>> len(OSError.__subclasses__())
22

Having "help(Exception)" list 50+ different Django exceptions is unlikely to be informative for anyone.

In many ways, Python 2 was actually better in this regard, since you could just do "import exceptions; help(exceptions)" to get a nice overview of the full exception hierarchy using the standard logic for displaying.

The closest Python 3 currently gets to that is "import builtins; help(builtins)", which is significantly harder to discover.

I think the way modules display this information may actually provide a solid precedent that addresses the dynamic state problem I'm concerned about: filter the subclass list to only include subclasses that come from the same module as the object being displayed, and use the existing showtree/getclasstree interfaces to display the nested hierarchy nicely (the same way modules do).

Then the "... and NNN other subclasses in other modules" would only be appended when there was cross-module inheritance involved.

New changeset a323cdc by Nick Coghlan (Sanyam Khurana) in branch 'master':
bpo-8525: help() on a type now shows builtin subclasses (GH-5066)
a323cdc

I've merged the version that displays up to 4 builtin subclasses in a flat list when help() is called on a type. Thanks for the patch Sanyam, and for the comments and suggestions everyone else.

While I'm closing out this feature request as implemented, if anyone's interested in pursuing the more sophisticated showtree/getclasstree approach that would better show position in the class hierarchy (both parents *and* children), feel free to file a new enhancement issue that refers back to this one.

This caused a regression. See bpo-35614 for details.

Navigation Menu

Repository navigation

Display exceptions' subclasses in help() #52771

8 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Display exceptions' subclasses in help() #52771

Description

Activity

robcliffe commented on Apr 24, 2010

terryjreedy commented on Apr 30, 2010

merwok commented on Apr 30, 2010

briancurtin commented on Apr 30, 2010

RodolphoEckhardt commented on Nov 21, 2010

merwok commented on Nov 22, 2010

abalkin commented on Nov 22, 2010

robcliffe commented on Nov 22, 2010

robcliffe commented on Nov 22, 2010

merwok commented on Nov 22, 2010

abalkin commented on Nov 23, 2010

8 remaining items

ncoghlan commented on Jul 14, 2018

robcliffe commented on Jul 15, 2018

eric-wieser commented on Jul 23, 2018

CuriousLearner commented on Jul 23, 2018

ncoghlan commented on Jul 24, 2018

ncoghlan commented on Jul 24, 2018

ncoghlan commented on Oct 21, 2018

ncoghlan commented on Oct 21, 2018

serhiy-storchaka commented on Dec 29, 2018

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions