importlib does not properly remove frames when invoking external import hooks

@brettcannon

BPO	23773
Nosy	@brettcannon, @ncoghlan, @ericsnowcurrently, @raulcd
Files	cpython-3ac58de829ef-fix-external-module-loader-call-traceback.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2015-03-25.03:28:58.344>
labels = ['type-bug', 'library']
title = 'importlib does not properly remove frames when invoking external import hooks'
updated_at = <Date 2016-09-08.17:28:36.728>
user = 'https://bugs.python.org/shiz'

bugs.python.org fields:

activity = <Date 2016-09-08.17:28:36.728>
actor = 'brett.cannon'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2015-03-25.03:28:58.344>
creator = 'shiz'
dependencies = []
files = ['38681']
hgrepos = []
issue_num = 23773
keywords = ['patch']
message_count = 3.0
messages = ['239219', '240764', '240781']
nosy_count = 5.0
nosy_names = ['brett.cannon', 'ncoghlan', 'eric.snow', 'shiz', 'raulcd']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue23773'
versions = ['Python 3.4', 'Python 3.5']

When adding a custom module loader to sys.meta_path, importlib does not properly remove its frames before invoking it. This results in a weird traceback with importlib._bootstrap frames in if an error occurs during load_module(), like such:

Traceback (most recent call last):
  File "/Users/mark/Development/Projects/rave/rave/bootstrap/__init__.py", line 102, in bootstrap_game
    __import__(MODULE_PACKAGE + '.' + mod.replace('.py', ''))
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/__init__.py", line 1, in <module>
    from . import common, core3
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/__init__.py", line 11, in <module>
    from .texture import Texture, Image
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/texture.py", line 8, in <module>
    raise ValueError

Attached is a patch against the current hg head that makes sure module loaders are invoked through _call_with_frames_removed, so that the output becomes a lot more readable:

Traceback (most recent call last):
  File "/Users/mark/Development/Projects/rave/rave/bootstrap/__init__.py", line 102, in bootstrap_game
    __import__(MODULE_PACKAGE + '.' + mod.replace('.py', ''))
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/__init__.py", line 1, in <module>
    from . import common, core3
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/__init__.py", line 11, in <module>
    from .texture import Texture, Image
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/texture.py", line 8, in <module>
    raise ValueError
ValueError

Ideally it would remove the calls /within/ the module loader itself too, but I do not see an easy way to do this without changing the entire _call_with_frames_removed semantics. This fixes most of the issues, though.

Did you happen to run the test suite with this change, Shiz? There's actually a test to make sure that the part of the traceback you're avoiding isn't lost in case there's a bug in importlib itself.

Can you show that your change won't actually affect finding errors in importlib itself?

Ah, that is a very good point. I'll run the suite as soon as I find the time and will report back.

Navigation Menu

Repository navigation

importlib does not properly remove frames when invoking external import hooks #67961

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

importlib does not properly remove frames when invoking external import hooks #67961

Description

Activity

shiz commented on Mar 25, 2015

brettcannon commented on Apr 13, 2015

shiz commented on Apr 13, 2015

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions