test.support.find_unused_port() race condition: test_socket fails with OSError: [Errno 98] Address already in use #82878

Activity

1. vstinner commented on Nov 5, 2019

   

   vstinner

   on Nov 5, 2019

   MemberAuthor

   More actions

   When two tests using support.find_unused_port() run in parallel, it's possible to get the same port number and so the first process using it wins, and the other one fails. Example with test_socket running in parallel of multiple many other tests.

   Example of fix to avoid support.find_unused_port():

   diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
   index 1bf562a03d..4479a69943 100644
   --- a/Lib/test/test_socket.py
   +++ b/Lib/test/test_socket.py
   @@ -6202,10 +6202,11 @@ class CreateServerFunctionalTest(unittest.TestCase):
                         "dualstack_ipv6 not supported")
        @unittest.skipUnless(support.IPV6_ENABLED, 'IPv6 required for this test')
        def test_dual_stack_client_v6(self):
   -        port = support.find_unused_port()
   -        with socket.create_server(("", port), family=socket.AF_INET6,
   +        with socket.create_server(("", 0), family=socket.AF_INET6,
                                      dualstack_ipv6=True) as sock:
                self.echo_server(sock)
   +            port = sock.getsockname()[1]
   +            print("PORT", port)
                self.echo_client(("::1", port), socket.AF_INET6)

   AMD64 RHEL7 Refleaks 3.8:
   https://buildbot.python.org/all/#/builders/299/builds/48

   0:02:32 load avg: 7.37 [132/423] test_timeout passed (56.5 sec) -- running: test_regrtest (1 min 56 sec), test_logging (1 min 17 sec), test_mailbox (1 min 39 sec), test_statistics (33.8 sec), test_socket (55.8 sec)
   ...
   0:02:56 load avg: 8.33 [153/423] test_urllib2net passed -- running: test_regrtest (2 min 20 sec), test_logging (1 min 40 sec), test_mailbox (2 min 2 sec), test_statistics (57.4 sec), test_socket (1 min 19 sec)
   ...
   0:03:11 load avg: 8.30 [161/423] test_asyncore passed -- running: test_logging (1 min 55 sec), test_support (33.7 sec), test_mailbox (2 min 17 sec), test_statistics (1 min 12 sec), test_socket (1 min 34 sec)
   ...
   0:03:24 load avg: 8.01 [167/423] test_support passed (46.7 sec) -- running: test_mailbox (2 min 31 sec), test_multiprocessing_forkserver (40.2 sec), test_socket (1 min 47 sec)
   ...
   0:03:36 load avg: 7.78 [180/423] test_docxmlrpc passed -- running: test_decimal (30.7 sec), test_mailbox (2 min 42 sec), test_multiprocessing_forkserver (52.1 sec), test_socket (1 min 59 sec)
   ...
   0:03:39 load avg: 7.88 [184/423/1] test_socket failed (2 min 2 sec) -- running: test_decimal (33.9 sec), test_mailbox (2 min 46 sec), test_multiprocessing_forkserver (55.2 sec)
   ...
   0:03:39 load avg: 7.88 [184/423/1] test_socket failed (2 min 2 sec) -- running: test_decimal (33.9 sec), test_mailbox (2 min 46 sec), test_multiprocessing_forkserver (55.2 sec)
   beginning 6 repetitions
   123456
   ....

   Warning -- threading._dangling was modified by test_socket
   Before: {<weakref at 0x7f08b2b48830; to '_MainThread' at 0x7f08b73a8140>}
   After: {<weakref at 0x7f08b07258f0; to 'Thread' at 0x7f08aefb6a00>, <weakref at 0x7f08b07257d0; to 'Thread' at 0x7f08aefb6b40>, <weakref at 0x7f08b07259b0; to '_MainThread' at 0x7f08b73a8140>, <weakref at 0x7f08b0725710; to 'Thread' at 0x7f08aefb6be0>}

   test test_socket failed -- Traceback (most recent call last):
     File "/home/buildbot/buildarea/3.8.cstratak-RHEL7-x86_64.refleak/build/Lib/test/test_socket.py", line 6206, in test_dual_stack_client_v6
       with socket.create_server(("", port), family=socket.AF_INET6,
     File "/home/buildbot/buildarea/3.8.cstratak-RHEL7-x86_64.refleak/build/Lib/socket.py", line 886, in create_server
       raise error(err.errno, msg) from None
   OSError: [Errno 98] Address already in use (while attempting to bind on address ('', 45626))
2. vstinner

   added

   3.8 (EOL)end of lifeend of life

   testsTests in the Lib/test dirTests in the Lib/test dir

   on Nov 5, 2019
3. vstinner commented on Dec 6, 2019

   

   vstinner

   on Dec 6, 2019

   MemberAuthor

   More actions

   support.find_unused_port() has a race condition by design, but tests are re-run sequentially by buildbots and so the rare race condition is worked around in practice.
4. vstinner

   closed this as completedon Dec 6, 2019
5. vstinner

   added

   invalid

   on Dec 6, 2019
6. ezio-melotti

   transferred this issue fromon Apr 10, 2022