Repository navigation
Py_NewInterpreter does not work #47973
Description
| BPO | 3723 |
|---|---|
| Nosy | @loewis, @warsaw, @amauryfa, @tiran, @benjaminp, @djc |
| Dependencies | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/warsaw'
closed_at = <Date 2008-10-30.21:48:53.355>
created_at = <Date 2008-08-29.12:11:09.887>
labels = ['extension-modules', 'type-bug', 'release-blocker']
title = 'Py_NewInterpreter does not work'
updated_at = <Date 2008-10-30.21:48:53.353>
user = 'https://github.com/amauryfa'bugs.python.org fields:
activity = <Date 2008-10-30.21:48:53.353>
actor = 'christian.heimes'
assignee = 'barry'
closed = True
closed_date = <Date 2008-10-30.21:48:53.355>
closer = 'christian.heimes'
components = ['Extension Modules']
creation = <Date 2008-08-29.12:11:09.887>
creator = 'amaury.forgeotdarc'
dependencies = ['4213']
files = ['11524', '11661', '11820', '11895']
hgrepos = []
issue_num = 3723
keywords = ['patch', 'needs review']
message_count = 17.0
messages = ['72127', '73403', '73530', '73531', '74069', '74070', '74071', '74072', '74919', '74938', '74939', '74940', '74943', '75248', '75277', '75338', '75385']
nosy_count = 7.0
nosy_names = ['loewis', 'barry', 'amaury.forgeotdarc', 'christian.heimes', 'benjamin.peterson', 'djc', 'grahamd']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue3723'
versions = ['Python 3.0']
Activity
amauryfa commentedon Aug 29, 2008
The example Demo/embed/importexc.c crashes, because Py_NewInterpreter
cannot reimport builtins and sys modules. This problem seems important
for embedding applications like mod_python, for example.
(the "import exceptions" statement does not work with python 3.0, but
replacing with e.g. "import types" does not change anything: the
interpreter is not correctly renewed)
I tried to put "-1" in the m_size structure of these modules, and they
seem to import correctly. However, "builtins" comes with its original
dictionary - without the standard exceptions.
I think that these modules should be made re-importable, with specific
functions.
Maybe two related problems:
once you've done
del sys.modules['sys']
it's not possible to get it back, "import sys" raises an error...
the usual trick to call sys.setdefaultencoding will not work, since it
needs to "imp.reload(sys)"
benjaminp commentedon Sep 18, 2008
Maybe, I'm not seeing the whole problem, but can't we just add
_PySys_Init and _PyBuiltin_Init to config.c like in the attached patch?
Obviously, we will eventually want to make a separate state to store
module globals in, but I think this will work for 3.0 final.
amauryfa commentedon Sep 21, 2008
I applied the patch to PC/config.c, but this did not change anything.
benjaminp commentedon Sep 21, 2008
Interesting, here it lets import.c's init_builtin reinitalize modules...
grahamd commentedon Sep 30, 2008
Adding the functions as initfunc in module init table is of no use as
they aren't invoked when creating a sub interpreter.
One thing that does appear to work, although no idea of whether it is
correct way to solve problem, is to duplicate the builtin/sys
initialisation that occurs in Py_InitializeEx() function.
Attached diff shows nature of changes. Diff is bit messy as have left
existing code in there but #ifdef'd out.
Maybe this will give someone who knows how overall interpreter
initialisation is supposed to work a head start on coming up with proper
fix. But then it could be totally wrong as well.
At least with change as is, mod_wsgi works for sub interpreters now.
I'll do more work later on whether it is correct way to solve it.
amauryfa commentedon Sep 30, 2008
Your patch may go in the right direction, but please provide only
context diff or unified diff files.
Use "diff -du", or "svn diff" to generate the file.
grahamd commentedon Sep 30, 2008
Argh. Personally I like to provide context diff's but more often than not
get abused for providing them over a unified diff. Was in a hurry this
time as had only a couple of minutes of battery life left on the laptop,
so quickly did it without thinking and then ran off to find a power point.
:-)
1 remaining item
loewis commentedon Oct 17, 2008
Here is a patch that fixed importexc.c. It consists of the following parts:
that these modules don't support repeated initialization. This should be
reviewed; perhaps it's better (and necessary) to record the init
function not only for dynamically-loaded modules in m_init, but also for
statically linked ones, so that the reinit code can call them again
(whether or not it is safe to call them again for sys and builtins
should then be studied).
not ready. when trying to use the file system encoding in this state,
use ASCII instead.
amauryfa commentedon Oct 17, 2008
I think the patch goes in the right direction.
But in addition, Py_NewInterpreter() has to call initstdio() between initmain() and initsite() (the same sequence as in Py_InitializeEx)
Found by using the following command string in importexc.c:
"import types; print(types.XXX)"
benjaminp commentedon Oct 17, 2008
Wouldn't it make more sense to move interpreter initialization things to
Py_NewInterpreter and call it from Py_InitializeEx?
tiran commentedon Oct 17, 2008
Sounds like a good plan, Benjamin
loewis commentedon Oct 17, 2008
Can you propose a specific patch? I'm worried that doing so blindly
introduces other bugs.
tiran commentedon Oct 26, 2008
The patch "subinterpreter.patch" is based on Martin's patch
"importexc.diff". The patch contains additional code to setup a
preliminary stderr object and a call to initstdio(). Amaury is right. I
had to add initstdio() to initialize the standard streams. But I can't
get it to work.
tiran commentedon Oct 28, 2008
In combination with the patch in bpo-4213, "subinterpreter.patch" fixes the
problem.
I'm assigning the bug to Barry for his final decision.
grahamd commentedon Oct 30, 2008
In conjunction with bpo-4213, the attached subinterpreter.patch appears to
fix issue for mod_wsgi.
tiran commentedon Oct 30, 2008
Applied in r67057