Repository navigation
transparent gzip compression in urllib #43521
Description
| BPO | 1508475 |
|---|---|
| Nosy | @rhettinger, @jcea, @orsenthil, @vstinner, @merwok, @berkerpeksag, @vadmium, @JimJJewett, @serhiy-storchaka, @demianbrecht |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/orsenthil'
closed_at = None
created_at = <Date 2006-06-19.08:59:09.000>
labels = ['type-feature', 'library']
title = 'transparent gzip compression in urllib'
updated_at = <Date 2015-03-07.02:32:39.334>
user = 'https://bugs.python.org/antialize'bugs.python.org fields:
activity = <Date 2015-03-07.02:32:39.334>
actor = 'demian.brecht'
assignee = 'orsenthil'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2006-06-19.08:59:09.000>
creator = 'antialize'
dependencies = []
files = ['7335', '19811', '34177']
hgrepos = []
issue_num = 1508475
keywords = ['patch']
message_count = 23.0
messages = ['50500', '50501', '114671', '114725', '114726', '122342', '122343', '122351', '158315', '158380', '158400', '160355', '160384', '163925', '163935', '211893', '212252', '212473', '213982', '226258', '226399', '234363', '236013']
nosy_count = 18.0
nosy_names = ['rhettinger', 'jjlee', 'jcea', 'orsenthil', 'jerub', 'vstinner', 'antialize', 'ruseel', 'nadeem.vawda', 'thomaspinckney3', 'eric.araujo', 'abacabadabacaba', 'jcon', 'berker.peksag', 'martin.panter', 'Jim.Jewett', 'serhiy.storchaka', 'demian.brecht']
pr_nums = []
priority = 'high'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue1508475'
versions = ['Python 3.5']Linked PRs
Metadata
Metadata
Assignees
Projects
- Status
- Status
Activity
antialize commentedon Jun 19, 2006
Some webservers support gzipping things before sending
them, this patch adds transparrent support for this in
urllib2 (documentation http://www.http-compression.com/)
This patach *requires* hash patch 914340 as a
prerequirement as this enabels stream support in the
gzip libary..
jjlee commentedon Jan 30, 2007
Looks good.
This needs tests and docs. As a new feature, this could not be released until Python 2.6.
It would be nice to have support for managing content negotiation in general, but that wish isn't an obstacle to this patch.
BreamoreBoy commentedon Aug 22, 2010
@jakob could you provide an updated patch for py3k that includes unit test and doc changes?
antialize commentedon Aug 23, 2010
No, I have long since moved on to other things.
orsenthil commentedon Aug 23, 2010
Its okay, Jacab, we will take it forward.
23 remaining items
vadmium commentedon Feb 15, 2015
I suggest resolving bpo-15955 first, then the GzipFile API could be used without fear of decompression bombs.
soreavis commentedon Jul 19, 2026
Serhiy, when this came up for bulk-closing you said transparent gzip was a nice-to-have "if anybody provide PRs" — though not easy. Fair enough on that — it never really stalled on code, it stalled on one design call: opt-in handler vs default-on at http.client. That call's yours; here's one concrete answer to it: #154197.
I went opt-in — a HTTPGzipHandler that's not in build_opener()'s defaults, so urlopen() is unchanged unless you ask for it. That sidesteps Martin's 2014 footgun, where default-on would silently decompress a .tar.gz and break Range requests. It decodes only when it added the Accept-Encoding itself, and strips Content-Encoding/Content-Length.
Senthil preferred default-on with a flag; I can flip it if you'd rather — the mechanism's identical. gzip only for now, deflate a follow-up.
Merge remote-tracking branch 'upstream/main' into pythongh-43521-gzip…
Merge remote-tracking branch 'upstream/main' into pythongh-43521-gzip…
pythongh-43521: Skip the HTTPGzipHandler tests when zlib is unavailable
soreavis commentedon Oct 27, 2026
@orsenthil, #154197 has been open three months without a review. It's opt-in:
build_opener()doesn't installHTTPGzipHandler, sourlopen()behaves as before. It follows your 2012 list on headers and caller intent: Content-Encoding and Content-Length are dropped after decoding, and a request that already sets Accept-Encoding is left alone. It merges cleanly with main and the tests pass. You preferred default-on over a handler; I went opt-in because of Martin's 2014 .tar.gz and Range point. Is opt-in OK, or should I flip it?orsenthil commentedon Oct 27, 2026
@soreavis , opt-in better and conservative for this long pending request. I will review this.
Merge branch 'main' into pythongh-43521-gzip-handler
pythongh-43521: Decode gzip responses with gzip.GzipFile