Repository navigation
'tarfile.StreamError: seeking backwards is not allowed' when extract symlink #57009
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/gustaebel'
closed_at = <Date 2020-12-18.18:53:15.392>
created_at = <Date 2011-08-20.22:57:39.951>
labels = ['type-bug', '3.8', '3.9', '3.10', '3.7', 'library']
title = "'tarfile.StreamError: seeking backwards is not allowed' when extract symlink"
updated_at = <Date 2020-12-18.18:53:15.392>
user = 'https://bugs.python.org/adunand'bugs.python.org fields:
activity = <Date 2020-12-18.18:53:15.392>
actor = 'mdk'
assignee = 'lars.gustaebel'
closed = True
closed_date = <Date 2020-12-18.18:53:15.392>
closer = 'mdk'
components = ['Library (Lib)']
creation = <Date 2011-08-20.22:57:39.951>
creator = 'adunand'
dependencies = []
files = ['49168']
hgrepos = []
issue_num = 12800
keywords = ['patch']
message_count = 13.0
messages = ['142580', '221577', '221595', '223171', '341969', '369061', '369119', '369120', '373384', '377173', '381803', '381807', '381810']
nosy_count = 9.0
nosy_names = ['lars.gustaebel', 'catlee', 'taleinat', 'adunand', 'serhiy.storchaka', 'andrew.garner', 'mdk', 'miss-islington', 'Jeffrey.Kintscher']
pr_nums = ['13217', '20972', '21409', '23508', '23509']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12800'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']
Activity
adunand commentedon Aug 20, 2011
When you extractall a tarball containing a symlink in stream mode ('r|'), an Exception happens:
Traceback (most recent call last): File "./test_extractall_stream_symlink.py", line 26, in <module> tar.extractall(path=destdir) File "/usr/lib/python3.2/tarfile.py", line 2134, in extractall self.extract(tarinfo, path, set_attrs=not tarinfo.isdir()) File "/usr/lib/python3.2/tarfile.py", line 2173, in extract set_attrs=set_attrs) File "/usr/lib/python3.2/tarfile.py", line 2249, in _extract_member self.makefile(tarinfo, targetpath) File "/usr/lib/python3.2/tarfile.py", line 2289, in makefile source.seek(tarinfo.offset_data) File "/usr/lib/python3.2/tarfile.py", line 553, in seek raise StreamError("seeking backwards is not allowed") tarfile.StreamError: seeking backwards is not allowedYou can reproduce the bug with this snippet of code:
TEMPDIR='/tmp/pyton_test'
os.mkdir(TEMPDIR)
tempdir = os.path.join(TEMPDIR, "testsymlinks")
temparchive = os.path.join(TEMPDIR, "testsymlinks.tar")
destdir = os.path.join(TEMPDIR, "extract")
os.mkdir(tempdir)
try:
source_file = os.path.join(tempdir,'source')
target_file = os.path.join(tempdir,'symlink')
with open(source_file,'w') as f:
f.write('something\n')
os.symlink('source', target_file)
tar = tarfile.open(temparchive,'w')
tar.add(target_file, arcname=os.path.basename(target_file))
tar.add(source_file, arcname=os.path.basename(source_file))
tar.close()
fo = open(temparchive, 'rb')
tar = tarfile.open(fileobj=fo, mode='r|')
try:
tar.extractall(path=destdir)
finally:
tar.close()
finally:
os.unlink(temparchive)
shutil.rmtree(TEMPDIR)
If source_file is added before target_file, there is no Exception raised. But it still raised when you create the same tarball with GNU tar.
BreamoreBoy commentedon Jun 25, 2014
ping.
serhiy-storchaka commentedon Jun 26, 2014
All works to me without exception in 2.7, 3.3 and 3.4.
andrewgarner commentedon Jul 16, 2014
This seems to be a similar to bpo-10761 where symlinks are not being overwritten by TarFile.extract but is only an issue in streaming mode and only in python3. To reproduce, attempt to extract a symlink from a tarfile opened with 'r|' and overwrite an existing file.
Here's a simple scripts that demonstrates this behavior adapted from Aurélien's.
#!/usr/bin/python
On python 3.3.2 I get the following results:
On python 3.4.1 I get the following results:
websurfer5 commentedon May 9, 2019
The problem is in TarFile.makelink() in Lib/tarfile.py. It calls os.symlink() to create the link, which fails because the link already exists and triggers the exception handler. The exception handler then tries to create the linked file under the assumption (per source code comments) that the link creation failed because the system doesn't support symbolic links. The file creation then fails because it requires seeking backwards in the archive.
catlee commentedon May 16, 2020
Is there anything I can do to help get this landed? The PR in github works for me.
JulienPalard commentedon May 17, 2020
Hi Chris, which exception did you got exactly? Was it caused by the r| mode or by a symlink (or file) already existing?
catlee commentedon May 17, 2020
It's caused by the combination of the symlink existing, and having the tarfile opened in r| mode.
If I run the attached test file in a fresh directory, I get the following exception:
raceback (most recent call last):
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 2227, in makelink
os.symlink(tarinfo.linkname, targetpath)
FileExistsError: [Errno 17] File exists: 'message.txt' -> './symlink.txt'
JulienPalard commentedon Jul 9, 2020
Strange fact, this was already fixed in 011525e (which closes bpo-10761, nice spot Andrew) but was lost during a merge in 0d28a61:
Stranger fact, the test was not lost during the merge, and still lives today (test_extractall_symlinks).
Happen that the current test is passing because it's in part erroneous, instead of trying to create a symlink on an existing one, it creates a symlink far far away:
(Pdb) p targetpath
'/home/mdk/clones/python/cpython/@test_648875_tmp-tardir/testsymlinks/home/mdk/clones/python/cpython/@test_648875_tmp-tardir/testsymlinks/symlink'
Aditionally it passes anway because tar.errorlevel equals 1, which means the error is logged but not raised.
With the following small patch:
the error is raised as expected: FileExistsError: [Errno 17] File exists: '/home/mdk/clones/python/cpython/@test_649794_tmpæ-tardir/testsymlinks/source' -> '/home/mdk/clones/python/cpython/@test_649794_tmpæ-tardir/testsymlinks/symlink'
I'm opening an PR to restore this as it was intended.
taleinat commentedon Sep 19, 2020
See also another duplicate of this issue, bpo-40049.
JulienPalard commentedon Nov 25, 2020
New changeset 4fedd71 by Julien Palard in branch 'master':
bpo-12800: tarfile: Restore fix from 011525e (GH-21409)
4fedd71
miss-islington commentedon Nov 25, 2020
New changeset 9d2c2a8 by Miss Islington (bot) in branch '3.9':
bpo-12800: tarfile: Restore fix from 011525e (GH-21409)
9d2c2a8
miss-islington commentedon Nov 25, 2020
New changeset bda2e68 by Miss Islington (bot) in branch '3.8':
bpo-12800: tarfile: Restore fix from 011525e (GH-21409)
bda2e68