Skip to content

test.test_urllib2net.TimeoutTest ftp related tests fail due to ftp://www.pythontest.net/ being unavailable #80312

Closed
Closed
test.test_urllib2net.TimeoutTest ftp related tests fail due to ftp://www.pythontest.net/ being unavailable#80312
Labels
3.7 (EOL)end of life3.8 (EOL)end of lifetestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

Description

@tirkarthi
tirkarthi
opened on Feb 27, 2019
BPO 36131
Nosy @vstinner, @benjaminp, @pablogsal, @tirkarthi

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-02-28.17:42:14.817>
created_at = <Date 2019-02-27.05:06:02.870>
labels = ['3.7', '3.8', 'type-bug', 'tests']
title = 'test.test_urllib2net.TimeoutTest ftp related tests fail due to ftp://www.pythontest.net/ being unavailable'
updated_at = <Date 2019-03-19.02:27:43.947>
user = 'https://github.com/tirkarthi'

bugs.python.org fields:

activity = <Date 2019-03-19.02:27:43.947>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2019-02-28.17:42:14.817>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2019-02-27.05:06:02.870>
creator = 'xtreak'
dependencies = []
files = []
hgrepos = []
issue_num = 36131
keywords = []
message_count = 12.0
messages = ['336729', '336730', '336731', '336739', '336740', '336741', '336746', '336773', '336801', '336802', '336857', '338311']
nosy_count = 4.0
nosy_names = ['vstinner', 'benjamin.peterson', 'pablogsal', 'xtreak']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue36131'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7', 'Python 3.8']

Activity

  1. tirkarthi commented on Feb 27, 2019

    @tirkarthi
    tirkarthi
    on Feb 27, 2019
    MemberAuthor

    I am seeing this error on Windows and Mac CI builds where FTP related tests in test.test_urllib2net.TimeoutTest are failing. It's reproducible locally too where the tests are skipped on Mac and Ubuntu. Not sure if it's random since several PRs in the last few hours fail with this.

    Appveyor :

    VSTS builds :

    $ ./python.exe -m unittest -vv test.test_urllib2net.TimeoutTest
test_ftp_basic (test.test_urllib2net.TimeoutTest) ... skipped "Resource 'ftp://www.pythontest.net/' is not available"
test_ftp_default_timeout (test.test_urllib2net.TimeoutTest) ... skipped "Resource 'ftp://www.pythontest.net/' is not available"
test_ftp_no_timeout (test.test_urllib2net.TimeoutTest) ... /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: ResourceWarning: unclosed <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.3.111', 49633), raddr=('159.89.235.38', 21)>
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: ResourceWarning: unclosed <socket.socket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.3.111', 49635), raddr=('159.89.235.38', 21)>
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: ResourceWarning: unclosed <socket.socket fd=10, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.3.111', 49639), raddr=('159.89.235.38', 21)>
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: ResourceWarning: unclosed <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.3.111', 49641), raddr=('159.89.235.38', 21)>
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.3.111', 49643), raddr=('159.89.235.38', 21)>
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
skipped "Resource 'ftp://www.pythontest.net/' is not available"
test_ftp_timeout (test.test_urllib2net.TimeoutTest) ... skipped "Resource 'ftp://www.pythontest.net/' is not available"
test_http_basic (test.test_urllib2net.TimeoutTest) ... ok
test_http_default_timeout (test.test_urllib2net.TimeoutTest) ... ok
test_http_no_timeout (test.test_urllib2net.TimeoutTest) ... ok
test_http_timeout (test.test_urllib2net.TimeoutTest) ... ok

    Ran 8 tests in 33.617s

    OK (skipped=4)

  2. tirkarthi
    added
    3.8 (EOL)end of life
    testsTests in the Lib/test dir
    type-bugAn unexpected behavior, bug, or error
    on Feb 27, 2019

  3. pablogsal commented on Feb 27, 2019

    @pablogsal
    pablogsal
    on Feb 27, 2019
    Member

    Several buildbots are failing due to this problem. Some examples:

    https://buildbot.python.org/all/#builders/33/builds/520
    https://buildbot.python.org/all/#builders/72/builds/503
    https://buildbot.python.org/all/#builders/78/builds/419

  4. pablogsal
    added
    3.7 (EOL)end of life
    on Feb 27, 2019

  5. tirkarthi commented on Feb 27, 2019

    @tirkarthi
    tirkarthi
    on Feb 27, 2019
    MemberAuthor

    These tests were skipped in Travis with bpo-35411

  6. vstinner commented on Feb 27, 2019

    @vstinner
    vstinner
    on Feb 27, 2019
    Member

    Right now, ftp://www.pythontest.net/ fails with "500 OOPS: vsf_sysutil_bind" according to Firefox. It's not the first time that I see this error.

  7. vstinner commented on Feb 27, 2019

    @vstinner
    vstinner
    on Feb 27, 2019
    Member

    https://buildbot.python.org/all/#builders/33/builds/520

    ERROR: test_ftp_basic (test.test_urllib2net.TimeoutTest)
    ERROR: test_ftp_default_timeout (test.test_urllib2net.TimeoutTest)
    ERROR: test_ftp_no_timeout (test.test_urllib2net.TimeoutTest)
    ERROR: test_ftp_timeout (test.test_urllib2net.TimeoutTest)

    Example:

    ERROR: test_ftp_basic (test.test_urllib2net.TimeoutTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "D:\buildarea\2.7.ware-win81-release.refleak\build\lib\test\test_urllib2net.py", line 300, in test_ftp_basic
    u = _urlopen_with_retry(self.FTP_HOST)
  File "D:\buildarea\2.7.ware-win81-release.refleak\build\lib\test\test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "D:\buildarea\2.7.ware-win81-release.refleak\build\lib\test\test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
URLError: <urlopen error ftp error: [Errno 10061] No connection could be made because the target machine actively refused it>

  8. vstinner commented on Feb 27, 2019

    @vstinner
    vstinner
    on Feb 27, 2019
    Member
    $ ./python -m test -v test_urllib2net -u all

    ...
    test_ftp (test.test_urllib2net.OtherNetworkTests) ... skipped "Resource 'ftp://www.pythontest.net/README' is not available"
    test_redirect_url_withfrag (test.test_urllib2net.OtherNetworkTests) ... ok
    test_sites_no_connection_close (test.test_urllib2net.OtherNetworkTests) ... skipped 'XXX: http://www.imdb.com is gone'
    test_urlwithfrag (test.test_urllib2net.OtherNetworkTests) ... ok
    test_ftp_basic (test.test_urllib2net.TimeoutTest) ... skipped "Resource 'ftp://www.pythontest.net/' is not available"
    test_ftp_default_timeout (test.test_urllib2net.TimeoutTest) ... skipped "Resource 'ftp://www.pythontest.net/' is not available"
    test_ftp_no_timeout (test.test_urllib2net.TimeoutTest) ... ERROR
    test_ftp_timeout (test.test_urllib2net.TimeoutTest) ... ERROR
    ...

    ERROR: test_ftp_no_timeout (test.test_urllib2net.TimeoutTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/urllib/request.py", line 1540, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
  File "/home/vstinner/prog/python/master/Lib/urllib/request.py", line 2425, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
  File "/home/vstinner/prog/python/master/Lib/ftplib.py", line 359, in ntransfercmd
    host, port = self.makepasv()
  File "/home/vstinner/prog/python/master/Lib/ftplib.py", line 337, in makepasv
    host, port = parse227(self.sendcmd('PASV'))
  File "/home/vstinner/prog/python/master/Lib/ftplib.py", line 273, in sendcmd
    return self.getresp()
  File "/home/vstinner/prog/python/master/Lib/ftplib.py", line 246, in getresp
    raise error_perm(resp)
ftplib.error_perm: 500 OOPS: vsf_sysutil_bind

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/test/test_urllib2net.py", line 327, in test_ftp_no_timeout
    u = _urlopen_with_retry(self.FTP_HOST, timeout=None)
  File "/home/vstinner/prog/python/master/Lib/test/test_urllib2net.py", line 27, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/vstinner/prog/python/master/Lib/test/test_urllib2net.py", line 23, in _retry_thrice
    raise last_exc
  File "/home/vstinner/prog/python/master/Lib/test/test_urllib2net.py", line 19, in _retry_thrice
    return func(*args, **kwargs)
  File "/home/vstinner/prog/python/master/Lib/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/home/vstinner/prog/python/master/Lib/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/home/vstinner/prog/python/master/Lib/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/home/vstinner/prog/python/master/Lib/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/home/vstinner/prog/python/master/Lib/urllib/request.py", line 1551, in ftp_open
    raise exc.with_traceback(sys.exc_info()[2])
  File "/home/vstinner/prog/python/master/Lib/urllib/request.py", line 1540, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
  File "/home/vstinner/prog/python/master/Lib/urllib/request.py", line 2425, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
  File "/home/vstinner/prog/python/master/Lib/ftplib.py", line 359, in ntransfercmd
    host, port = self.makepasv()
  File "/home/vstinner/prog/python/master/Lib/ftplib.py", line 337, in makepasv
    host, port = parse227(self.sendcmd('PASV'))
  File "/home/vstinner/prog/python/master/Lib/ftplib.py", line 273, in sendcmd
    return self.getresp()
  File "/home/vstinner/prog/python/master/Lib/ftplib.py", line 246, in getresp
    raise error_perm(resp)
urllib.error.URLError: <urlopen error ftp error: error_perm('500 OOPS: vsf_sysutil_bind')>

  9. tirkarthi commented on Feb 27, 2019

    @tirkarthi
    tirkarthi
    on Feb 27, 2019
    MemberAuthor

    Right now, ftp://www.pythontest.net/ fails with "500 OOPS: vsf_sysutil_bind" according to Firefox. It's not the first time that I see this error.

    Related issue in the past due to disk space bpo-35386 . When I first created the issue opening the FTP URL on Firefox used to give me result or raise 425 Bad IP error but now it's now consistently returning "500 OOPS: vsf_sysutil_bind" consistently.

  10. pablogsal commented on Feb 27, 2019

    @pablogsal
    pablogsal
    on Feb 27, 2019
    Member

    Almost all buildbots have failed due to this issue. It seems that is fixed now:

    ❯ ./python.exe -m test -v test_urllib2net -u all
    == CPython 3.8.0a2+ (heads/master:d5a551c269, Feb 27 2019, 09:17:08) [Clang 10.0.0 (clang-1000.10.44.4)]
    == macOS-10.14.3-x86_64-i386-64bit little-endian
    == cwd: /Users/pgalindo3/github/cpython/build/test_python_15573
    == CPU count: 8
    == encodings: locale=UTF-8, FS=utf-8
    Run tests sequentially
    0:00:00 load avg: 2.02 [1/1] test_urllib2net
    test_close (test.test_urllib2net.CloseSocketTest) ... ok
    test_custom_headers (test.test_urllib2net.OtherNetworkTests) ... ok
    test_file (test.test_urllib2net.OtherNetworkTests) ... ok
    test_ftp (test.test_urllib2net.OtherNetworkTests) ... ok
    test_redirect_url_withfrag (test.test_urllib2net.OtherNetworkTests) ... ok
    test_sites_no_connection_close (test.test_urllib2net.OtherNetworkTests) ... skipped 'XXX: http://www.imdb.com is gone'
    test_urlwithfrag (test.test_urllib2net.OtherNetworkTests) ... ok
    test_ftp_basic (test.test_urllib2net.TimeoutTest) ... ok
    test_ftp_default_timeout (test.test_urllib2net.TimeoutTest) ... ok
    test_ftp_no_timeout (test.test_urllib2net.TimeoutTest) ... ok
    test_ftp_timeout (test.test_urllib2net.TimeoutTest) ... ok
    test_http_basic (test.test_urllib2net.TimeoutTest) ... ok
    test_http_default_timeout (test.test_urllib2net.TimeoutTest) ... ok
    test_http_no_timeout (test.test_urllib2net.TimeoutTest) ... ok
    test_http_timeout (test.test_urllib2net.TimeoutTest) ... ok

    ----------------------------------------------------------------------

    Ran 15 tests in 9.531s

    OK (skipped=1)

    == Tests result: SUCCESS ==

    1 test OK.

    Total duration: 10 sec 17 ms
    Tests result: SUCCESS

    I am going to re-trigger a build in all affected buildbots.

  11. tirkarthi commented on Feb 28, 2019

    @tirkarthi
    tirkarthi
    on Feb 28, 2019
    MemberAuthor

    Just curious if it was some regression or infrastructure related issue in pythontest.net ?

  12. pablogsal commented on Feb 28, 2019

    @pablogsal
    pablogsal
    on Feb 28, 2019
    Member

    That was my impression.

  13. vstinner commented on Feb 28, 2019

    @vstinner
    vstinner
    on Feb 28, 2019
    Member

    I discussed with Ernest on #python-infra. It was an issue with the DNS "resolving the host back to 127.0.1.1 (...) so vsftpd is telling users to connect back to themselves".

  14. vstinner
    closed this as completedon Feb 28, 2019

  15. pablogsal commented on Mar 19, 2019

    @pablogsal
    pablogsal
    on Mar 19, 2019
    Member

    A similar issue happened again on s390x SLES 2.7:

    https://buildbot.python.org/all/#/builders/66/builds/373

  16. ezio-melotti
    transferred this issue fromon Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.7 (EOL)end of life3.8 (EOL)end of lifetestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions