Some test_ssl and test_asyncio tests fail with OpenSSL 1.1.1 on Python 3.4 and 3.5

@vstinner

BPO	36576
Nosy	@vstinner, @larryhastings
PRs	[3.5] bpo-36576: Skip test_ssl and test_asyncio tests failing with OpenSSL 1.1.1 #12694

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-09-25.12:32:19.144>
created_at = <Date 2019-04-09.14:41:46.254>
labels = ['tests']
title = 'Some test_ssl and test_asyncio tests fail with OpenSSL 1.1.1 on Python 3.4 and 3.5'
updated_at = <Date 2019-09-25.12:32:19.137>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2019-09-25.12:32:19.137>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-09-25.12:32:19.144>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2019-04-09.14:41:46.254>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 36576
keywords = ['patch']
message_count = 4.0
messages = ['339756', '339758', '351284', '353188']
nosy_count = 2.0
nosy_names = ['vstinner', 'larry']
pr_nums = ['12694']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue36576'
versions = ['Python 3.5']

On Fedora 29, test_ssl and test_asyncio when Python 3.5 is linked with OpenSSL 1.1.1b (Fedora package openssl-1.1.1b-3.fc29.x86_64):

test_ssl:

test_options (test.test_ssl.ContextTests)
test_alpn_protocols (test.test_ssl.ThreadedTests)
test_default_ecdh_curve (test.test_ssl.ThreadedTests)
test_shared_ciphers (test.test_ssl.ThreadedTests)

test_asyncio:

test_create_server_ssl_match_failed (test.test_asyncio.test_events.EPollEventLoopTests)
test_create_server_ssl_match_failed (test.test_asyncio.test_events.PollEventLoopTests)
test_create_server_ssl_match_failed (test.test_asyncio.test_events.SelectEventLoopTests)

Fixing these tests would require to backport some ssl features, and I don't think that it's worth it.

Attached PR 12694 skip these tests on OpenSSL 1.1.1.

Note: these tests pass with OpenSSL 1.1.0.

FYI for Fedora, we also care of having the Python 3.4 test suite passing with OpenSSL 1.1.1 and so we will maintain a similar change downstream.

======================================================================
FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_ssl.py", line 866, in test_options
    self.assertEqual(default, ctx.options)
AssertionError: 2181169236 != 2182217812

======================================================================
FAIL: test_alpn_protocols (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_ssl.py", line 3205, in test_alpn_protocols
    self.assertIsInstance(stats, ssl.SSLError)
AssertionError: {'client_alpn_protocol': None, 'server_alpn_protocols': [None], 'version': 'TLSv1.2', 'client_npn_protocol': None, 'server_npn_protocols': [None], 'server_shared_ciphers': [[('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256), ('TLS_CHACHA20_POLY1305_SHA256', 'TLSv1.3', 256), ('TLS_AES_128_GCM_SHA256', 'TLSv1.3', 128), ('TLS_AES_128_CCM_SHA256', 'TLSv1.3', 128), ('ECDHE-ECDSA-AES256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-ECDSA-CHACHA20-POLY1305', 'TLSv1.2', 256), ('ECDHE-RSA-CHACHA20-POLY1305', 'TLSv1.2', 256), ('DHE-DSS-AES256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-DSS-AES128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-CHACHA20-POLY1305', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-CCM8', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-CCM', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-SHA384', 'TLSv1.2', 256), ('ECDHE-RSA-AES256-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-SHA', 'TLSv1.0', 256), ('ECDHE-RSA-AES256-SHA', 'TLSv1.0', 256), ('DHE-RSA-AES256-CCM8', 'TLSv1.2', 256), ('DHE-RSA-AES256-CCM', 'TLSv1.2', 256), ('DHE-RSA-AES256-SHA256', 'TLSv1.2', 256), ('DHE-DSS-AES256-SHA256', 'TLSv1.2', 256), ('DHE-RSA-AES256-SHA', 'SSLv3', 256), ('DHE-DSS-AES256-SHA', 'SSLv3', 256), ('ECDHE-ECDSA-AES128-CCM8', 'TLSv1.2', 128), ('ECDHE-ECDSA-AES128-CCM', 'TLSv1.2', 128), ('ECDHE-ECDSA-AES128-SHA256', 'TLSv1.2', 128), ('ECDHE-RSA-AES128-SHA256', 'TLSv1.2', 128), ('ECDHE-ECDSA-AES128-SHA', 'TLSv1.0', 128), ('ECDHE-RSA-AES128-SHA', 'TLSv1.0', 128), ('DHE-RSA-AES128-CCM8', 'TLSv1.2', 128), ('DHE-RSA-AES128-CCM', 'TLSv1.2', 128), ('DHE-RSA-AES128-SHA256', 'TLSv1.2', 128), ('DHE-DSS-AES128-SHA256', 'TLSv1.2', 128), ('DHE-RSA-AES128-SHA', 'SSLv3', 128), ('DHE-DSS-AES128-SHA', 'SSLv3', 128), ('ECDHE-ECDSA-ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-ECDSA-CAMELLIA256-SHA384', 'TLSv1.2', 256), ('ECDHE-RSA-CAMELLIA256-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('ECDHE-RSA-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('DHE-DSS-ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-RSA-ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-DSS-ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-CAMELLIA256-SHA256', 'TLSv1.2', 256), ('DHE-DSS-CAMELLIA256-SHA256', 'TLSv1.2', 256), ('DHE-RSA-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('DHE-DSS-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('DHE-RSA-CAMELLIA256-SHA', 'SSLv3', 256), ('DHE-DSS-CAMELLIA256-SHA', 'SSLv3', 256), ('DHE-RSA-CAMELLIA128-SHA', 'SSLv3', 128), ('DHE-DSS-CAMELLIA128-SHA', 'SSLv3', 128), ('AES256-GCM-SHA384', 'TLSv1.2', 256), ('AES128-GCM-SHA256', 'TLSv1.2', 128), ('AES256-CCM8', 'TLSv1.2', 256), ('AES256-CCM', 'TLSv1.2', 256), ('AES128-CCM8', 'TLSv1.2', 128), ('AES128-CCM', 'TLSv1.2', 128), ('AES256-SHA256', 'TLSv1.2', 256), ('AES128-SHA256', 'TLSv1.2', 128), ('AES256-SHA', 'SSLv3', 256), ('AES128-SHA', 'SSLv3', 128), ('ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('CAMELLIA256-SHA256', 'TLSv1.2', 256), ('CAMELLIA128-SHA256', 'TLSv1.2', 128), ('CAMELLIA256-SHA', 'SSLv3', 256), ('CAMELLIA128-SHA', 'SSLv3', 128)]], 'peercert': {}, 'cipher': ('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), 'compression': None} is not an instance of <class 'ssl.SSLError'>

======================================================================
FAIL: test_default_ecdh_curve (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_ssl.py", line 3064, in test_default_ecdh_curve
    self.assertIn("ECDH", s.cipher()[0])
AssertionError: 'ECDH' not found in 'TLS_AES_256_GCM_SHA384'

======================================================================
FAIL: test_shared_ciphers (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_ssl.py", line 3381, in test_shared_ciphers
    self.fail(name)
AssertionError: TLS_AES_256_GCM_SHA384

======================================================================
ERROR: test_create_server_ssl_match_failed (test.test_asyncio.test_events.EPollEventLoopTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_asyncio/test_events.py", line 1172, in test_create_server_ssl_match_failed
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'

======================================================================
ERROR: test_create_server_ssl_match_failed (test.test_asyncio.test_events.PollEventLoopTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_asyncio/test_events.py", line 1172, in test_create_server_ssl_match_failed
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'

======================================================================
ERROR: test_create_server_ssl_match_failed (test.test_asyncio.test_events.SelectEventLoopTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_asyncio/test_events.py", line 1172, in test_create_server_ssl_match_failed
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'

I wasn't sure if tests pass or not on OpenSSL 1.1.0. My colleague Miro checked: the full Python 3.5 test suite pass on Fedora 28 with OpenSSL 1.1.0.

python35 is linked to OpenSSL 1.1.0 on Fedora 28:
https://src.fedoraproject.org/rpms/python35/blob/f28/f/python35.spec#_121

Example of successful python35 build:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1236291

New changeset 4d1c254 by larryhastings (Victor Stinner) in branch '3.5':
bpo-36576: Skip test_ssl and test_asyncio tests failing with OpenSSL 1.1.1 (bpo-12694)
4d1c254

Python 3.4 reached end of life. Python 3.5 is fixed. I close the issue.

Navigation Menu

Repository navigation

Some test_ssl and test_asyncio tests fail with OpenSSL 1.1.1 on Python 3.4 and 3.5 #80757

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Some test_ssl and test_asyncio tests fail with OpenSSL 1.1.1 on Python 3.4 and 3.5 #80757

Description

Activity

vstinner commented on Apr 9, 2019

vstinner commented on Apr 9, 2019

larryhastings commented on Sep 7, 2019

vstinner commented on Sep 25, 2019

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions