Skip to content

subprocess leaves open fds on construction error #49429

Closed

Description

@birkenfeld
BPO 5179
Nosy @birkenfeld, @pitrou
Files
  • sp-patch.py
  • fix_subprocess_leak_on_failure_on_windows.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2009-03-03.22:56:40.218>
    created_at = <Date 2009-02-07.20:04:37.082>
    labels = ['library']
    title = 'subprocess leaves open fds on construction error'
    updated_at = <Date 2009-03-03.22:56:40.199>
    user = 'https://github.com/birkenfeld'

    bugs.python.org fields:

    activity = <Date 2009-03-03.22:56:40.199>
    actor = 'ocean-city'
    assignee = 'none'
    closed = True
    closed_date = <Date 2009-03-03.22:56:40.218>
    closer = 'ocean-city'
    components = ['Library (Lib)']
    creation = <Date 2009-02-07.20:04:37.082>
    creator = 'georg.brandl'
    dependencies = []
    files = ['12972', '13236']
    hgrepos = []
    issue_num = 5179
    keywords = ['patch']
    message_count = 12.0
    messages = ['81347', '81348', '81387', '82001', '82028', '82082', '82083', '82084', '82092', '83097', '83099', '83108']
    nosy_count = 3.0
    nosy_names = ['georg.brandl', 'pitrou', 'ocean-city']
    pr_nums = []
    priority = 'high'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue5179'
    versions = ['Python 2.6', 'Python 3.0', 'Python 3.1', 'Python 2.7']

    Activity

    1. birkenfeld commented on Feb 7, 2009

      @birkenfeld
      MemberAuthor

      The test case below will (on Linux here) eventually quit with "OSError:
      [Errno 24] Too many open files". I assume that some additional cleaning
      up is in order.

      -------------------------------------------------------------------

      from subprocess import Popen, PIPE
      
      while 1:
          try:
              Popen(['nonexisting'], stdout=PIPE, stderr=PIPE)
          except OSError, err:
              if err.errno != 2:  # ignore "no such file"
                  raise
    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      on Feb 7, 2009
    3. birkenfeld commented on Feb 7, 2009

      @birkenfeld
      MemberAuthor

      Proposed patch attached.

    4. pitrou commented on Feb 8, 2009

      @pitrou
      Member

      Perhaps you could add a test?

    5. birkenfeld commented on Feb 14, 2009

      @birkenfeld
      MemberAuthor

      I would, but how this fails is likely to be highly platform-specific.
      Can you try it on Windows and tell me what the resulting exception is?

    6. pitrou commented on Feb 14, 2009

      @pitrou
      Member

      Well, I'm not under Windows. I'll try to launch a VM if nobody beats me
      to it...

    7. birkenfeld commented on Feb 14, 2009

      @birkenfeld
      MemberAuthor

      On second thought, if the patch is applied, there *shouldn't* be an
      exception. And simply running the loop for a fixed number of
      repetitions isn't a good test either, since different platforms may have
      a different maximum for open file descriptors.

      Hmm, is there a way to get the current number of open file descriptors?

    8. pitrou commented on Feb 14, 2009

      @pitrou
      Member

      Hmm, is there a way to get the current number of open file descriptors?

      Under Unix, you can use resource.getrlimit():

      (1024, 1024)

      But 1024 is a very common value, so you could simply loop 1024 times if
      it's not too slow.
      Besides, you definitely don't want to loop 2**31 times if 2**31 happens
      to be the current limit.

    9. pitrou commented on Feb 14, 2009

      @pitrou
      Member

      Wow, the Roundup email gateway borks code snippets:

      >>> import resource
      >>> resource.getrlimit(resource.RLIMIT_NOFILE)
      (1024, 1024)
    10. birkenfeld commented on Feb 14, 2009

      @birkenfeld
      MemberAuthor

      OK, Windows has 2048. Added test and fixed in r69620.

    11. ocean-city commented on Mar 3, 2009

      ocean-citymannequin
      Mannequin

      This issue is not fixed on windows yet. (test_leaking_fds_on_error fails)
      I think attached patch will fix this.

      ======================================================================
      ERROR: test_writes_before_communicate (main.ProcessTestCase)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
        File "test_subprocess.py", line 398, in test_writes_before_communicate
        File "e:\python-dev\py3k\lib\subprocess.py", line 632, in __init__
        File "e:\python-dev\py3k\lib\subprocess.py", line 746, in _get_handles
      IOError: [Errno 24] Too many open files
    12. birkenfeld commented on Mar 3, 2009

      @birkenfeld
      MemberAuthor

      Since I can't test on Windows, I'll leave that in your hands :)

    13. ocean-city commented on Mar 3, 2009

      ocean-citymannequin
      Mannequin

      Thanks, fixed in r70137(trunk), r70142(py3k), r70146(release30-maint),
      r70147(release26-maint)

    14. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      stdlibStandard Library Python modules in the Lib/ directory

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions