Repository navigation
Inconsistency in cgi.FieldStorage() causes unicode/byte TypeError. #71495
Description
| BPO | 27308 |
|---|---|
| Nosy | @bertjwregeer, @berkerpeksag |
| Superseder | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2016-06-13.20:03:47.476>
created_at = <Date 2016-06-13.18:45:31.480>
labels = ['type-bug', 'library']
title = 'Inconsistency in cgi.FieldStorage() causes unicode/byte TypeError.'
updated_at = <Date 2016-09-27.03:38:23.247>
user = 'https://bugs.python.org/MarcelHellkamp'bugs.python.org fields:
activity = <Date 2016-09-27.03:38:23.247>
actor = 'X-Istence'
assignee = 'none'
closed = True
closed_date = <Date 2016-06-13.20:03:47.476>
closer = 'berker.peksag'
components = ['Library (Lib)']
creation = <Date 2016-06-13.18:45:31.480>
creator = 'Marcel Hellkamp'
dependencies = []
files = ['43378']
hgrepos = []
issue_num = 27308
keywords = ['patch']
message_count = 5.0
messages = ['268460', '268462', '268466', '268468', '277481']
nosy_count = 3.0
nosy_names = ['X-Istence', 'berker.peksag', 'Marcel Hellkamp']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '24764'
type = 'behavior'
url = 'https://bugs.python.org/issue27308'
versions = ['Python 3.5', 'Python 3.6']
Activity
MarcelHellkamp commentedon Jun 13, 2016
Discovered here: bottlepy/bottle#856
If a multipart section has a "Content-Length" header, but no "filename" attribute in the "Content-Disposition" header, cgi.FieldStorage tries to write binary data to a temporary file opened in text-mode.
The problem here is that cgi.FieldStorage tries to decide if something is a binary file-upload or a unicode form-field, but it does so based on two different headers in two different places. If the headers contradict each other (form-fields usually don't have a Content-Length, file-uploads usually have a filename), parsing breaks with a TypeError.
Unfortunately, there are some HTTP client libraries out there that trigger this bug.
Here is what happens:
A "Content-Length" header causes
cgi.FieldStorage.lengthto be is set (which is fine).https://hg.python.org/cpython/file/3.4/Lib/cgi.py#l550
If
lengthhas a value,read_binary()is used instead ofread_lines()(which is questionable).https://hg.python.org/cpython/file/3.4/Lib/cgi.py#l733
read_binary()callsmake_file()which creates the buffer file in text mode if it does not find afilenameattribute in the "Content-Disposition" Header (which is somewhat okay).https://hg.python.org/cpython/file/3.4/Lib/cgi.py#l515
https://hg.python.org/cpython/file/3.4/Lib/cgi.py#l893
The bug is triggered if the last two steps disagree on the bytes vs. text question.
[-]Inconsistency in cgi.FieldStorage() causes unicode/byte issue.[/-][+]Inconsistency in cgi.FieldStorage() causes unicode/byte TypeError.[/+]MarcelHellkamp commentedon Jun 13, 2016
This should fix the issue.
berkerpeksag commentedon Jun 13, 2016
Thanks for the patch. From a quick look, the patch looks fine but we need a test case. You can see Lib/test/test_cgi.py for existing tests.
MarcelHellkamp commentedon Jun 13, 2016
Looks like this is a duplicate to bpo-24764 and already fixed. Sorry for the noise.
bertjwregeer commentedon Sep 27, 2016
This is not a duplicate of https://bugs.python.org/issue24764