Repository navigation
ssl: get CPU cap flags for AESNI and PCLMULQDQ #71955
Description
| BPO | 27768 |
|---|---|
| Nosy | @gpshead, @pitrou, @tiran, @alex, @dstufft |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/tiran'
closed_at = <Date 2017-09-05.23:31:22.959>
created_at = <Date 2016-08-15.13:55:07.888>
labels = ['extension-modules', 'expert-SSL', 'type-feature']
title = 'ssl: get CPU cap flags for AESNI and PCLMULQDQ'
updated_at = <Date 2017-09-05.23:37:52.051>
user = 'https://github.com/tiran'bugs.python.org fields:
activity = <Date 2017-09-05.23:37:52.051>
actor = 'pitrou'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2017-09-05.23:31:22.959>
closer = 'christian.heimes'
components = ['Extension Modules', 'SSL']
creation = <Date 2016-08-15.13:55:07.888>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 27768
keywords = []
message_count = 13.0
messages = ['272763', '272845', '273359', '273360', '273361', '273633', '273635', '273636', '273637', '273676', '273702', '301401', '301404']
nosy_count = 6.0
nosy_names = ['gregory.p.smith', 'janssen', 'pitrou', 'christian.heimes', 'alex', 'dstufft']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue27768'
versions = ['Python 3.6']
Activity
tiran commentedon Aug 15, 2016
OpenSSL has a function called OPENSSL_ia32cap_loc() to get the processor's capability vector in X86 and X86_64 systems. The information is useful to decide which cipher suite to prefer. For example on machines without AES-NI and CLMUL CPU instructions, ChaCha20 should be prefered over AES-GCM.
https://www.openssl.org/docs/man1.0.2/crypto/OPENSSL_ia32cap_loc.html
bpo-27766 just exposes the plain OPENSSL_ia32cap_loc(). A richer API should parse the bit field and expose the bits as structure.
gpshead commentedon Aug 16, 2016
This is very architecture specific and focused on low level information that you can also already get from places like /proc/cpuinfo on Linux.
Also, regardless of what capability bits a CPU exposes that has nothing to do with what accelerations the underlying library implementation itself actually supports.
For your example purpose of choosing which algorithm to use, testing the actual performance of each option in your code at startup time seems more foolproof in any application running long enough for performance to be an issue.
I expect in most common situations you can just use ctypes to call this function from openssl if you feel you must use it.
I'm not convinced it belongs in the stdlib.
pitrou commentedon Aug 22, 2016
Agreed with Gregory.
I'm also surprised you consider selecting a cipher suite based on performance considerations rather than security. I'm not sure that's something we want to promote.
(btw, AES speed is usually not a critical factor except in very specific situations)
alex commentedon Aug 22, 2016
In this case, performance is security. Both AES-GCM and ChaCha20-Poly1305 are secure. Modulo one thing: GCM in software is hard to implement in constant-time, so it's strongly preferable to use it only when there's a hardware implementation. It works out nicely that in addition to being constant-time, the hardware implementation of GCM is faster.
pitrou commentedon Aug 22, 2016
I still think it's a slippery slope to include such specialized APIs that most people don't know what to make of. This reminds me of
RAND_egd().If ChaCha20-Poly1305 is always /at least/ as secure as AES-GCM (and sometimes more), then we should simply prioritize it in the cipher list, regardless of a potentially poorer performance that probably never has an actual impact on the Python-written application (correct me if I'm wrong here).
tiran commentedon Aug 25, 2016
GPS, sure it is simple enough under Linux. But what about other operating systems? OPENSSL_ia32cap_loc() works under Windows, too.
Antoine, AES-GCM is still faster and performs better than ChaCha20 Poly1305. NSS and Mozilla's recommended cipher suite list prefers AES-GCM over ChaCha20, too. https://wiki.mozilla.org/Security/Server_Side_TLS
It's reasonable and simple to provide the best cipher suite that matches the systems' capabilities. As Alex stated, performance is security.
To provide the CPU capabilities to the ssl module and 3rd party authors (e.g. Cory asked on behalf of requests), let's keep ssl._ia32cap() a private function and just add two constants: HAVE_AESNI = True/False/None, HAVE_PCLMULQDQ = True/False/None (None: ia32cap is not available on the system). Is that ok with you?
pitrou commentedon Aug 25, 2016
The constants would have to be private, too. We really don't want to encourage other users to rely on them.
tiran commentedon Aug 25, 2016
Why? Other libraries like requests like to use the information, too.
pitrou commentedon Aug 25, 2016
If we wanted to expose such information, we would need to do it in a dedicated module (or in
os) and provide something that doesn't restrict itself to two particular x86 flags.requests can rely on a private attribute if they really want to.
gpshead commentedon Aug 25, 2016
The processor capabilities vector is not the API you want.
What you want is an API to ask the ssl library about characteristics of algorithm implementations it will be using.
Those are not the same thing.
If we want to expose processor capabilities as a library function it should not be done in the ssl module. That information belongs in the platform module. And doing so should not require a third party library such as OpenSSL. Querying capabilities is a very simple instruction and every platform architecture specific C macros to do it.
tiran commentedon Aug 26, 2016
GPS, I no longer want a general API for CPU cap vector. The ssl module and 3rd party packages are only interested in two properties of the current hardware: are AESNI and PCLMULQDQ instructions available and enabled.
I'm pretty sure ctypes won't work on Windows and OSX. The OpenSSL libraries are statically linked into the _ssl binary. The linker optimizer removes all functions that are not used. Somebody could give it a try, though.
[-]ssl: add public API for IA-32 processor capabilities vector[/-][+]ssl: get CPU cap flags for AESNI and PCLMULQDQ[/+]tiran commentedon Sep 5, 2017
I'm retracting my patch. Instead of making the cipher suite selection more complicated, we should just rely on OpenSSL to provide sensible defaults.
pitrou commentedon Sep 5, 2017
+1 !