Repository navigation
Allowing multiple instances of IDLE with sub-processes #43724
Description
| BPO | 1529142 |
|---|---|
| Nosy | @rhettinger, @kbkaiser, @taleinat |
| Superseder | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/kbkaiser'
closed_at = <Date 2009-04-04.07:35:44.342>
created_at = <Date 2006-07-26.17:45:06.000>
labels = ['expert-IDLE']
title = 'Allowing multiple instances of IDLE with sub-processes'
updated_at = <Date 2009-04-26.03:13:53.485>
user = 'https://github.com/taleinat'bugs.python.org fields:
activity = <Date 2009-04-26.03:13:53.485>
actor = 'kbk'
assignee = 'kbk'
closed = True
closed_date = <Date 2009-04-04.07:35:44.342>
closer = 'kbk'
components = ['IDLE']
creation = <Date 2006-07-26.17:45:06.000>
creator = 'taleinat'
dependencies = []
files = ['7427', '12180']
hgrepos = []
issue_num = 1529142
keywords = ['patch']
message_count = 16.0
messages = ['50763', '50764', '65460', '65547', '65560', '65563', '76517', '76586', '76716', '80470', '80583', '80610', '83738', '85013', '85379', '86597']
nosy_count = 6.0
nosy_names = ['rhettinger', 'kbk', 'ggenellina', 'taleinat', 'gpolo', 'weeble']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = None
status = 'closed'
superseder = '5487'
type = None
url = 'https://bugs.python.org/issue1529142'
versions = ['Python 3.1', 'Python 2.7']
Activity
taleinat commentedon Jul 26, 2006
(See patch 1201569 "allow running multiple instances of
IDLE" for previous discussion on this topic)
To summarize discussion up to this point: There is a
problem doing this on Windows since more than one
listening socket can be openned on a single port.
I suggest we "dodge" the Windows problem for now by
having IDLE try a random port every time (like in the
IDLEfork patch 661363) - thus collisions will be kept
to a minimum. If we choose among over 10,000 ports,
most users will never encounter a port collision. And
even when a collision does happen, it will probably be
detected and properly dealt with - collision
non-detection errors are, as Kurt mentioned, erratic.
This patch chooses a port from range(49152, 65536),
which are the 'dynamic' ports as described by the DCCP
(see http://www.iana.org/assignments/port-numbers), and
avoids known 'dangerous' ports (used by trojans, worms,
etc.) which I gathered by Googling for half an hour.
I replaced socket.timeout with select(), since I find
its timeout mechanism is more reliable. (Currently,
Python's socket's timeout mechanism raises unexpected,
unexplained errors on Windows.)
I also changed the flow a bit, so if a timeout occurs
after a sub-process is spawned, the subprocess is killed.
IMO This requires thorough testing. I tested it for
half an hour on my WinXP Python2.4.3, and after fixing
typos and such, everything seems to work.
taleinat commentedon Feb 7, 2007
After removing the SO_REUSEADDR flag, this has been working without any pauses, delays or hitches on my WinXP Python2.5 install. I sometimes have as many as 8 separate instances of IDLE running in parallel, with no side effects.
About the SO_REUSEADDR flag, I'm wondering if this is still required on Posix systems now that we're choosing random ports. If so, we should choose whether to use this flag based on the OS.
Is there any good reason -not- to choose a random port? I can't think of any.
taleinat commentedon Apr 14, 2008
I really, really wish we could get this in for Python2.6 - this issue is
a major drawback for beginners, for whom IDLE is mostly intended.
Perhaps not this specific patch; I am willing to work on cleaning up the
code and getting it tested by users on different platforms. Is there any
chance of this happening? What can I do to get this moving again?
gpolo commentedon Apr 16, 2008
Works fine for me, I just dislike that Port binding error message
because it is a bit misleading. It says "IDLE can't bind any TCP/IP
port, ..." which is not entirely true.
taleinat commentedon Apr 16, 2008
Thanks for taking the time to review this patch, Guilherme. For the
record, on which OS+Python have you tested this?
gpolo commentedon Apr 16, 2008
I tested under Linux (Ubuntu distro), Python trunk
weeble commentedon Nov 27, 2008
Is this ever likely to make it into IDLE? Running without a subprocess
in Windows appears to interact badly with the multiprocessing library
(attempting to spawn a process just creates a new IDLE window). I
couldn't figure out how to apply the patch on Windows - I couldn't tell
if the errors were because I was using the patch program wrong or
because the file is different now. Nevertheless, I made the changes
manually and they seemed to work. What needs to happen for this issue to
be fixed, whether by this patch or another solution?
taleinat commentedon Nov 29, 2008
First of all, the multiprocessing module doesn't work fully in an
interactive shell.
From the Python2.6 docs:
"Note Functionality within this package requires that the __main__
method be importable by the children. This is covered in Programming
guidelines however it is worth pointing out here. This means that some
examples, such as the multiprocessing.Pool examples will not work in
the interactive interpreter."
And later on that same page (under Programming Guidelines -> Windows):
"Make sure that the main module can be safely imported by a new Python
interpreter without causing unintended side effects (such a starting a
new process)."
This is probably why IDLE is acting up for you. I would be surprised
if using the multiprocessing module will work well in IDLE either with
or without a sub-process.
As per the patch, someone should just work up an updated one with all
of the insight gained in the last several years (!) regarding this.
It should just do two things:
code -- see http://bugs.python.org/issue1201569
underlying socket library to select an available port, and later
retrieve the selected port from the socket object -- see
http://mail.python.org/pipermail/idle-dev/2008-June/002687.html
The other changes in my original patch, while they do work, are
unnecessarily complex and make it much harder to get the patch
accepted. They should also not be significant if the spawning delays
are fixed.
If you'd like to work up such a patch and get it reviewed and
committed, please be my guest! I will help you in any way I can if you
take this up. Otherwise I guess it will wait until I find the time and
energy...
weeble commentedon Dec 1, 2008
Okay, I've uploaded a patch. As suggested, it passes 0 for the port
number and the port is automatically assigned. It then extracts the port
from the socket and passes it to the subprocess. Note that this means
the subprocess inherits the listening socket, which I gather is less
than ideal. However, as far as I can tell, this happens anyway if the
subprocess is restarted, since the socket remains bound and listening
the whole time. The patch also removes the use of SO_REUSEADDR because
it is no longer needed.
This works without a problem for me on Windows XP, but I can't test it
on anything more Unixy. I removed the three retries when it fails to
bind a socket - it's not clear when this might fail and yet still be a
good idea to retry, but I can put that back if that's something that
shouldn't be messed with.
weeble commentedon Jan 24, 2009
I installed Ubuntu on a laptop at the beginning of January and have
applied the patch I submitted above to IDLE. I've been using it
regularly for the last few weeks and have had no problems with it.
weeble commentedon Jan 26, 2009
A thought occurs to me: would this patch make it harder to cope with
awkward firewalls that block the connection? Are they more or less
likely to intervene when passing a port of 0 and letting it pick a port
automatically? And if they do intervene, would users be able to create
an exception if there is no small list of ports that might be used?
Apart from that, what do I need to do to move this forward? Go hunt for
more testers willing to try out the patch? Find an expert to review it?
ggenellina commentedon Jan 27, 2009
If this patch were accepted, bpo-5065 would be a non-issue then.
rhettinger commentedon Mar 18, 2009
Is there anyone following this thread that thinks the weeble's patch is
not ready to commit?
kbkaiser commentedon Apr 1, 2009
The hooks for the IP address will be needed if we add remote debugging
to IDLE, so I'd like to keep them. Otherwise, David Scherer's idea re
using an ephemeral port assignment looks very promising. I'll check
something in.
kbkaiser commentedon Apr 4, 2009
r71126. Thanks to everyone for all the work on this!
Note: I still need to patch the Windows install to
bind .py files to a different command string. The
-e option now only opens an editor window, but I need
to remove the -n option.
kbkaiser commentedon Apr 26, 2009
Opened a bug to get the -n switch removed from
the Windows Installer "Edit with IDLE" function.