tarfile cannot extract from stdin #84230

Activity

1. dtamuc commented on Mar 23, 2020

   

   dtamucmannequin

   on Mar 23, 2020

   MannequinAuthor

   More actions

   Hi,

   I have the following code:

   import tarfile
   import sys
   
   tar = tarfile.open(fileobj=sys.stdin.buffer, mode='r|*')
   tar.extractall("tarout")
   tar.close()
   

   then doing the following on a debian 10 system:

   $ python -m tarfile -c git.tar /usr/share/doc/git
   $ python -V
   Python 3.8.1
   $ cat git.tar | python foo.py
   $ cat git.tar | python foo.py
   Traceback (most recent call last):
     File "foo.py", line 5, in <module>
       tar.extractall("tarout")
     File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 2026, in extractall
       self.extract(tarinfo, path, set_attrs=not tarinfo.isdir(),
     File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 2067, in extract
       self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
     File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 2139, in _extract_member
       self.makefile(tarinfo, targetpath)
     File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 2178, in makefile
       source.seek(tarinfo.offset_data)
     File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 513, in seek
       raise StreamError("seeking backwards is not allowed")
   tarfile.StreamError: seeking backwards is not allowed
   

   The second extraction trys to seek, although the mode is 'r|*'.

   For reference if I remove ".buffer" from the code above, I can run
   it with python2 without problems:

   $ cat foo2.py
   import tarfile
   import sys
   
   tar = tarfile.open(fileobj=sys.stdin, mode='r|*')
   tar.extractall("tarout")
   tar.close()
   
   $ cat git.tar | python2 foo2.py
   $ cat git.tar | python2 foo2.py
   $ cat git.tar | python2 foo2.py
   $ cat git.tar | python2 foo2.py
   $ cat git.tar | python2 foo2.py
   
2. dtamuc

   added

   3.8 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   on Mar 23, 2020
3. Zheaoli commented on Mar 26, 2020

   

   Zheaolimannequin

   on Mar 26, 2020

   Mannequin

   More actions

   Hello

   I can't reproduce this issue on my Laptop from 3.8.1 to 3.9.0a4

   I think maybe it depends on the file you use

   would you mind to upload the file with the problem?
4. dtamuc commented on Mar 26, 2020

   

   dtamucmannequin

   on Mar 26, 2020

   MannequinAuthor

   More actions

   Hi,

   well, it says entity too large. I've attached a smaller one, that throws a similar but slightly different error. (Note: only on the _second_ extraction, it looks like problems with symlinks)

   You can find larger ones here:

   https://data.rbfh.de/issue40049/

   The typescript*.txt are showing a shell session with two different python versions. (3.4.2 and 3.8.2)
5. jonnyhsu commented on Mar 27, 2020

   

   jonnyhsumannequin

   on Mar 27, 2020

   Mannequin

   More actions

   This is caused when tarfile tries to write a symlink that already exists. Any exceptions to os.symlink() as handled as if the platform doesn't support symlinks, so it scans the entire tar to try and find the linked files. When it resumes extraction, it needs to do a negative seek to pick up where it left off, which causes the exception.

   I've reproduced the error on both Windows 10 and Ubuntu running on WSL. Python 2.7 handled this situation by checking if the symlink exists, but it looks like the entire tarfile library was replaced with an alternate implementation that doesn't check if the symlink exists. I've created a pull request to address this issue.
6. dtamuc commented on Mar 27, 2020

   

   dtamucmannequin

   on Mar 27, 2020

   MannequinAuthor

   More actions

   For me, this patch solves my problems. Thank you.
7. taleinat commented on Sep 18, 2020

   

   taleinat

   on Sep 18, 2020

   Contributor

   More actions

   GNU tar (v1.30, Ubuntu 20.04) does indeed overwrite files with symlinks upon extracting, while both ln -s and os.symlink do not. Therefore I agree that the appropriate behavior would seem to be to overwrite this way, as in the attached PR.
8. taleinat commented on Sep 19, 2020

   

   taleinat

   on Sep 19, 2020

   Contributor

   More actions

   This is actually a duplicate of bpo-12800, which itself describe precisely the same issue as in bpo-10761, which was fixed but then the fix was lost in a bad merge.

   I'm closing this, as discussion should happen on the original issues.
9. taleinat

   closed this as completedon Sep 19, 2020
10. taleinat

    closed this as completedon Sep 19, 2020
11. ezio-melotti

    transferred this issue fromon Apr 10, 2022