Skip to content

test_venv: test_with_pip() failed on "AMD64 Fedora without threads 3.x" buildbot: urllib3 dependency requires the threading module #63965

Closed

Description

@vstinner
BPO 19766
Nosy @vsajip, @ncoghlan, @vstinner, @tiran, @dstufft
Files
  • urllib3_no_thread.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/dstufft'
    closed_at = <Date 2013-12-21.15:21:19.460>
    created_at = <Date 2013-11-25.09:41:00.194>
    labels = ['type-bug', 'tests']
    title = 'test_venv: test_with_pip() failed on "AMD64 Fedora without threads 3.x" buildbot: urllib3 dependency requires the threading module'
    updated_at = <Date 2013-12-21.15:21:29.702>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2013-12-21.15:21:29.702>
    actor = 'ncoghlan'
    assignee = 'dstufft'
    closed = True
    closed_date = <Date 2013-12-21.15:21:19.460>
    closer = 'ncoghlan'
    components = ['Tests']
    creation = <Date 2013-11-25.09:41:00.194>
    creator = 'vstinner'
    dependencies = []
    files = ['32857']
    hgrepos = []
    issue_num = 19766
    keywords = ['patch', 'buildbot']
    message_count = 15.0
    messages = ['204316', '204413', '204475', '204495', '205111', '205506', '205512', '205535', '205542', '205557', '205590', '205813', '206225', '206227', '206753']
    nosy_count = 5.0
    nosy_names = ['vinay.sajip', 'ncoghlan', 'vstinner', 'christian.heimes', 'dstufft']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue19766'
    versions = ['Python 3.4']

    Activity

    1. vstinner commented on Nov 25, 2013

      @vstinner
      MemberAuthor

      The line "from . import exceptions" of urllib3 failed:
      https://github.com/shazow/urllib3/blob/master/urllib3/__init__.py#L22

      It is strange because urllib3/exceptions.py is part of the urllib3 module.

      http://buildbot.python.org/all/builders/AMD64%20Fedora%20without%20threads%203.x/builds/5674/steps/test/logs/stdio

      ======================================================================
      FAIL: test_with_pip (test.test_venv.EnsurePipTest)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
        File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/test/test_venv.py", line 299, in test_with_pip
          self.run_with_capture(venv.create, self.env_dir, with_pip=True)
      subprocess.CalledProcessError: Command '['/tmp/tmpd8r2w88y/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/test/test_venv.py", line 305, in test_with_pip
          self.fail(msg)
      AssertionError: Command '['/tmp/tmpd8r2w88y/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1
      
      **Subprocess Output**
      Traceback (most recent call last):
        File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/runpy.py", line 160, in _run_module_as_main
          "__main__", fname, loader, pkg_name)
        File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/runpy.py", line 73, in _run_code
          exec(code, run_globals)
        File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/ensurepip/__main__.py", line 66, in <module>
          main()
        File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/ensurepip/__main__.py", line 61, in main
          default_pip=args.default_pip,
        File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/ensurepip/__init__.py", line 92, in bootstrap
          _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
        File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/ensurepip/__init__.py", line 28, in _run_pip
          import pip
        File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/__init__.py", line 11, in <module>
        File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/vcs/mercurial.py", line 9, in <module>
        File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/download.py", line 22, in <module>
        File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/__init__.py", line 58, in <module>
        File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/utils.py", line 24, in <module>
        File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/compat.py", line 7, in <module>
        File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/packages/__init__.py", line 3, in <module>
        File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/packages/urllib3/__init__.py", line 22, in <module>
      ImportError: cannot import name 'exceptions'
    2. ncoghlan commented on Nov 25, 2013

      @ncoghlan
      Contributor

      Potentially another platform compatibility issue for pip's dependencies.

    3. vstinner commented on Nov 26, 2013

      @vstinner
      MemberAuthor

      In fact, it's an issue in the urllib3 library which require threads.

      $ cd /home/haypo/pip/PIP/pip/_vendor/requests/packages
      $ /home/haypo/prog/python/default/venv/bin/python -c 'import urllib3'
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "/home/haypo/pip/PIP/pip/_vendor/requests/packages/urllib3/__init__.py", line 24, in <module>
          from .poolmanager import PoolManager, ProxyManager, proxy_from_url
        File "/home/haypo/pip/PIP/pip/_vendor/requests/packages/urllib3/poolmanager.py", line 14, in <module>
          from ._collections import RecentlyUsedContainer
        File "/home/haypo/pip/PIP/pip/_vendor/requests/packages/urllib3/_collections.py", line 8, in <module>
          from threading import RLock
        File "/home/haypo/prog/python/default/Lib/threading.py", line 4, in <module>
          import _thread
      ImportError: No module named '_thread'

      Here is a patch for urllib3.

      I reported the issue upstream:
      urllib3/urllib3#289

    4. vstinner commented on Nov 26, 2013

      @vstinner
      MemberAuthor

      Cool, my patch was already applied upstream:

      urllib3/urllib3@929f158

      @nick: can you try to package a new wheel package for pip?

    5. changed the title [-]test_venv: test_with_pip() failed on "AMD64 Fedora without threads 3.x" buildbot[/-] [+]test_venv: test_with_pip() failed on "AMD64 Fedora without threads 3.x" buildbot: urllib3 dependency requires the threading module[/+] on Nov 28, 2013
    6. dstufft commented on Dec 3, 2013

      @dstufft
      Member

      The urllib3 in requests VCS was updated, I just need to bother Kenneth to make a new release of requests or update pip to an unreleased requests.

    7. ncoghlan commented on Dec 8, 2013

      @ncoghlan
      Contributor

      bpo-19924 suggests we may need a new distlib as well

    8. dstufft commented on Dec 8, 2013

      @dstufft
      Member

      Requests was released and pip updated it, I can release a new pip but it appears that perhaps distlib needs fixed before the without threads case is taken care of?

    9. vsajip commented on Dec 8, 2013

      @vsajip
      Member

      I will look at doing a distlib update shortly - but there's another issue (bpo-19913) that might also require an update - it' not clear yet.

    10. vsajip commented on Dec 8, 2013

      @vsajip
      Member

      I've updated distlib to use dummy_threading where threading isn't available - see

      https://bitbucket.org/pypa/distlib/commits/029fee573900765729402203e39b2171d7ae0784

      Can someone please test with this version vendored into pip to check that the failures no longer occur in a no-thread environment, before I do a distlib release?

    11. tiran commented on Dec 8, 2013

      @tiran
      Member

      Vinay, thanks for your fast response! :) bpo-19913 should be resolved, too. A couple of months ago several people complained about a new file that looked like a ZIP bomb. This virus warnings looks even more severe although it's probably a false positive. Could you try a new set of binaries without UPX? https://www.virustotal.com/ lets you scan the files with more than 40 programs at once and for free.

    12. vsajip commented on Dec 8, 2013

      @vsajip
      Member

      This commit in distlib uses uncompressed launcher executables which pass the virustotal.com checks:

      https://bitbucket.org/pypa/distlib/commits/e23c9e4fd3125fa88063de4dec80367b1ac82aff

    13. dstufft commented on Dec 10, 2013

      @dstufft
      Member

      Vinay, I've verified that the current default branch of distlib works without threading when vendored in pip and these tests pass.

    14. vsajip commented on Dec 15, 2013

      @vsajip
      Member

      I've released distlib 0.1.5 on PyPI. This release uses dummy_threading when threading isn't available.

    15. ncoghlan commented on Dec 15, 2013

      @ncoghlan
      Contributor

      Not quite fixed yet - Donald, I think all the vendored dependencies have been updated now, would it be possible to spin and incorporate a pip 1.5rc2 somewhat soonish so we know what's still left to be addressed on the ensurepip side?

    16. ncoghlan commented on Dec 21, 2013

      @ncoghlan
      Contributor

      Donald updated CPython to pip 1.5rc2, so test_venv is now passing without threading support:

      http://buildbot.python.org/all/builders/AMD64%20Fedora%20without%20threads%203.x/builds/5874/steps/test/logs/stdio

    17. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions