zlib.error with tarfile.open

@tiran

BPO	39039
Nosy	@tiran, @ethanfurman, @ambv, @jdevries3133
PRs	bpo-39039: tarfile raises descriptive exception from zlib.error #27766 [3.10] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) #28613 [3.9] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) #28614
Files	crash-c10c9839d987fa0df6912cb4084f43f3ce08ca82

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-09-29.10:58:55.030>
created_at = <Date 2019-12-13.16:00:48.662>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'zlib.error with tarfile.open'
updated_at = <Date 2021-09-29.10:58:55.028>
user = 'https://bugs.python.org/jvoisin'

bugs.python.org fields:

activity = <Date 2021-09-29.10:58:55.028>
actor = 'lukasz.langa'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-29.10:58:55.030>
closer = 'lukasz.langa'
components = ['Library (Lib)']
creation = <Date 2019-12-13.16:00:48.662>
creator = 'jvoisin'
dependencies = []
files = ['48774']
hgrepos = []
issue_num = 39039
keywords = ['patch']
message_count = 9.0
messages = ['358337', '358340', '358341', '399811', '399989', '402834', '402845', '402847', '402848']
nosy_count = 5.0
nosy_names = ['christian.heimes', 'ethan.furman', 'lukasz.langa', 'jvoisin', 'jack__d']
pr_nums = ['27766', '28613', '28614']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39039'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

The attached file produces the following stacktrace when opened via tarfile.open, on Python 3.7.5rc1:

$ cat test.py 
import sys
import tarfile

tarfile.open(sys.argv[1])
$ python3 test.py ./crash-c10c9839d987fa0df6912cb4084f43f3ce08ca82
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    tarfile.open(sys.argv[1])
  File "/usr/lib/python3.7/tarfile.py", line 1573, in open
    return func(name, "r", fileobj, **kwargs)
  File "/usr/lib/python3.7/tarfile.py", line 1645, in gzopen
    t = cls.taropen(name, mode, fileobj, **kwargs)
  File "/usr/lib/python3.7/tarfile.py", line 1621, in taropen
    return cls(name, mode, fileobj, **kwargs)
  File "/usr/lib/python3.7/tarfile.py", line 1484, in __init__
    self.firstmember = self.next()
  File "/usr/lib/python3.7/tarfile.py", line 2289, in next
    tarinfo = self.tarinfo.fromtarfile(self)
  File "/usr/lib/python3.7/tarfile.py", line 1094, in fromtarfile
    buf = tarfile.fileobj.read(BLOCKSIZE)
  File "/usr/lib/python3.7/gzip.py", line 276, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.7/_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
  File "/usr/lib/python3.7/gzip.py", line 471, in read
    uncompress = self._decompressor.decompress(buf, size)
zlib.error: Error -3 while decompressing data: invalid distances se

This file is also an invalid tar file:

$ tar xf crash-c10c9839d987fa0df6912cb4084f43f3ce08ca82

gzip: stdin: invalid compressed data--format violated
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Sure, but as a user, I would expect a better exception, like ValueError or ReadError, along with an error message, instead of an unexpected zlib exception.

@jvoisin

@jvoisin I am able to reproduce the problem when I download your script, but I am having a hard time reproducing it by passing corrupt archives to tarfile.open. How exactly was this file corrupted? I am trying to figure out if there are any similar implementation leaks / poor error messages in similar scenarios so I can do my best to patch them all.

You can see the reproduction scripts I am using here to get a better idea of what I have been trying. Be forewarned, they are pretty gnarly!

https://gist.github.com/jdevries3133/acbb5ba2a19093d3bcc214733ef85e5a

The file was created with a fuzzer, like the one described in https://dustri.org/b/fuzzing-python-in-python-and-doing-it-fast.html

New changeset b6fe857 by Jack DeVries in branch 'main':
bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766)
b6fe857

New changeset d6b69f2 by Łukasz Langa in branch '3.10':
[3.10] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) (GH-28613)
d6b69f2

New changeset 7bff4d3 by Łukasz Langa in branch '3.9':
[3.9] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) (GH-28614)
7bff4d3

Thanks for the fix, Jack! ✨ 🍰 ✨

Since the change translated zlib.error to tarfile.ReadError which already has to be handled by user code, it's strictly decreasing the surface of necessary exception handling. So, treating this as a bug fix, I backported this to 3.9 and 3.10 as well.

Navigation Menu

Repository navigation

zlib.error with tarfile.open #83220

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

zlib.error with tarfile.open #83220

Description

Activity

jvoisin commented on Dec 13, 2019

tiran commented on Dec 13, 2019

jvoisin commented on Dec 13, 2019

jdevries3133 commented on Aug 18, 2021

jvoisin commented on Aug 20, 2021

ambv commented on Sep 29, 2021

ambv commented on Sep 29, 2021

ambv commented on Sep 29, 2021

ambv commented on Sep 29, 2021

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions