Skip to content

Path.resolve() fails on complex symlinks #64086

Closed
Closed
Path.resolve() fails on complex symlinks#64086
Assignees
pitrou
Labels
stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Description

@serhiy-storchaka
serhiy-storchaka
opened on Dec 4, 2013
BPO 19887
Nosy @pitrou, @serhiy-storchaka, @vajrasky
Files
  • pathlib_resolve_test.py
  • pathlib_resolve.patch
  • pathlib_resolve_2.patch
  • pathlib_resolve_3.patch
  • pathlib_resolve_4.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/pitrou'
closed_at = <Date 2013-12-16.18:58:29.225>
created_at = <Date 2013-12-04.20:46:45.868>
labels = ['type-bug', 'library']
title = 'Path.resolve() fails on complex symlinks'
updated_at = <Date 2014-01-10.11:12:48.451>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2014-01-10.11:12:48.451>
actor = 'Bluebird75'
assignee = 'pitrou'
closed = True
closed_date = <Date 2013-12-16.18:58:29.225>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2013-12-04.20:46:45.868>
creator = 'serhiy.storchaka'
dependencies = []
files = ['32974', '32987', '32990', '33024', '33156']
hgrepos = []
issue_num = 19887
keywords = ['patch']
message_count = 20.0
messages = ['205245', '205262', '205294', '205303', '205307', '205311', '205420', '205442', '205455', '205456', '205458', '206268', '206269', '206310', '206313', '206345', '206346', '206348', '206349', '207849']
nosy_count = 5.0
nosy_names = ['pitrou', 'python-dev', 'Bluebird75', 'serhiy.storchaka', 'vajrasky']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19887'
versions = ['Python 3.4']

    Activity

    1. serhiy-storchaka commented on Dec 4, 2013

      @serhiy-storchaka
      serhiy-storchaka
      on Dec 4, 2013
      MemberAuthor

      Here is a script which success with os.path.realpath(), but fails with pathlib.Path.resolve(). The readlink -f also success with these examples.

    2. serhiy-storchaka
      added
      stdlibStandard Library Python modules in the Lib/ directory
      type-bugAn unexpected behavior, bug, or error
      on Dec 4, 2013

    3. pitrou commented on Dec 4, 2013

      @pitrou
      pitrou
      on Dec 4, 2013
      Member

      This is https://bitbucket.org/pitrou/pathlib/issue/9/pathresolve-fails-on-complex-symlinks

      As noted there:

      """It's a ENAMETOOLONG error when calling readlink(), because there are many "." components. This is really an edge case caused by the specific test setup, I don't think you'd have that many "." components in real-world use."""

    4. pitrou
      changed the title [-]Path.resolve() fails on deep symlinks[/-] [+]Path.resolve() ENAMETOOLONG on pathologic symlinks[/+] on Dec 4, 2013

    5. serhiy-storchaka commented on Dec 5, 2013

      @serhiy-storchaka
      serhiy-storchaka
      on Dec 5, 2013
      MemberAuthor

      Path.resolve() also fails when last link is absolute.

      mkdir testdir
      ln -s 0/0 testdir/1
      ln -s 1/1 testdir/2
      ln -s "$(readlink -f testdir)" testdir/0

      Path('testdir/2').resolve() fails:

      Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/pathlib.py", line 1017, in resolve
    s = self._flavour.resolve(self)
  File "/home/serhiy/py/cpython/Lib/pathlib.py", line 273, in resolve
    raise RuntimeError("Symlink loop from %r" % cur)
RuntimeError: Symlink loop from '/home/serhiy/py/cpython/testdir/0'

      Here is a patch which implements an algorithm similar to the algorithm used in posixpath.realpath().

    6. vajrasky commented on Dec 5, 2013

      @vajrasky
      vajraskymannequin
      on Dec 5, 2013
      Mannequin

      Whoa, Serhiy, your patch won't fly on Windows Vista.

      For starter, the patch does not use target_is_directory=True option in os.symlink. It needs that option when creating symlink to the directory on Windows Vista.

      The maximum depth that pathlib can do on Windows Vista is 4 (Remember, this line: "for depth in 0, 1, 2, 3, 10, 100"?).

      More than that:
      OSError: [WinError 1921] The name of the file cannot be resolved by the system:
      'C:\\Users\\vajrasky\\Code\\cpython\\@test_4220_tmp\\testdir\\link5'

      Not sure whether the bug is in pathlib.py or _getfinalpathname (from nt module).

    7. serhiy-storchaka commented on Dec 5, 2013

      @serhiy-storchaka
      serhiy-storchaka
      on Dec 5, 2013
      MemberAuthor

      Thank you Vajrasky. Here is a patch with fixed tests.

    8. vajrasky commented on Dec 5, 2013

      @vajrasky
      vajraskymannequin
      on Dec 5, 2013
      Mannequin

      It works. Just a coding nitpick, instead of hardcoding os.symlink(src, dst, target_is_directory=True) in the test, you can use helper function in the test itself, self.dirlink.

    9. pitrou commented on Dec 7, 2013

      @pitrou
      pitrou
      on Dec 7, 2013
      Member

      I would prefer if you made the test simpler. The current setup makes it difficult to understand and reproduce with the command line.

    10. serhiy-storchaka commented on Dec 7, 2013

      @serhiy-storchaka
      serhiy-storchaka
      on Dec 7, 2013
      MemberAuthor

      How the test can be simpler? It is already simple.

      You ca use pathlib_resolve_test.py but replace os.symlink('.', 'testdir/0') by os.symlink(os.path.abspath('testdir'), 'testdir/0'). Or use following shell commands:

      mkdir testdir
      for i in $(seq 100); do ln -s $((i-1))/$((i-1)) testdir/$i; done
      ln -s "$(pwd)/testdir" testdir/0

      ./python -c "import pathlib; print(pathlib.Path('testdir/100').resolve())"

    11. serhiy-storchaka
      changed the title [-]Path.resolve() ENAMETOOLONG on pathologic symlinks[/-] [+]Path.resolve() fails on complex symlinks[/+] on Dec 7, 2013

    12. pitrou commented on Dec 7, 2013

      @pitrou
      pitrou
      on Dec 7, 2013
      Member

      How the test can be simpler? It is already simple.

      I mean, don't use loops but a simple test setup as in test_resolve_dot.
      I am not interested in the pathologic "100 symlinks" case, just the case where the symlink is absolute.

    13. vajrasky commented on Dec 7, 2013

      @vajrasky
      vajraskymannequin
      on Dec 7, 2013
      Mannequin

      And don't forget to use self.dirlink instead of os.symlink(src, dst, target_is_directory=True).

    14. serhiy-storchaka commented on Dec 7, 2013

      @serhiy-storchaka
      serhiy-storchaka
      on Dec 7, 2013
      MemberAuthor

      Here is a patch with unrolled loops.

    15. serhiy-storchaka
      assigned
      pitrou
      on Dec 11, 2013

    16. pitrou commented on Dec 16, 2013

      @pitrou
      pitrou
      on Dec 16, 2013
      Member

      Here is a patch with unrolled loops.

      I suppose that was some kind of joke, but what I meant was that we don't need to test with 100 levels of symlinks. 2 or 3 are enough...

    17. pitrou commented on Dec 16, 2013

      @pitrou
      pitrou
      on Dec 16, 2013
      Member

      Here is a patch with refactored tests. Vajrasky, do you want to test under Windows?

    18. serhiy-storchaka commented on Dec 16, 2013

      @serhiy-storchaka
      serhiy-storchaka
      on Dec 16, 2013
      MemberAuthor

      I suppose that was some kind of joke, but what I meant was that we don't need to test with 100 levels of symlinks. 2 or 3 are enough...

      Yes, sorry for this joke. Your tests LGTM, but why you repeat similar code 3-4 times instead using loops?

    19. vajrasky commented on Dec 16, 2013

      @vajrasky
      vajraskymannequin
      on Dec 16, 2013
      Mannequin

      The patch passed on Windows Vista.

    20. pitrou commented on Dec 16, 2013

      @pitrou
      pitrou
      on Dec 16, 2013
      Member

      why you repeat similar code 3-4 times instead using loops?

      For no real reason :) I'll try with a loop.

    21. pitrou commented on Dec 16, 2013

      @pitrou
      pitrou
      on Dec 16, 2013
      Member

      Ah, I remember. Using subtests would make it more annoying to backport to 2.7.

    22. python-dev commented on Dec 16, 2013

      @python-dev
      python-devmannequin
      on Dec 16, 2013
      Mannequin

      New changeset 12a52186b4fd by Antoine Pitrou in branch 'default':
      Issue bpo-19887: Improve the Path.resolve() algorithm to support certain symlink chains.
      http://hg.python.org/cpython/rev/12a52186b4fd

    23. pitrou commented on Dec 16, 2013

      @pitrou
      pitrou
      on Dec 16, 2013
      Member

      Thanks a lot for the patch!

    24. pitrou
      closed this as completedon Dec 16, 2013

    25. bluebird75 commented on Jan 10, 2014

      @bluebird75
      bluebird75mannequin
      on Jan 10, 2014
      Mannequin

      Hi,

      This precise set of tests fails on Windows 7 on a NTFS partition (on revision c0b0e7aef360+ tip ), see below.

      The problem is probably minor (drive letter case). I won't be able to develop a fix myself, but I'll be happy to test one.

      cheers,

      Philippe

      ======================================================================
      FAIL: test_complex_symlinks_absolute (test.test_pathlib.PathTest)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1724, in test_complex_symlinks_absolute
    self._check_complex_symlinks(BASE)
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1692, in _check_complex_symlinks
    self.assertEqual(str(p), BASE)
AssertionError: 'C:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp' != 'c:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp'
- C:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^
+ c:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^

      ======================================================================
      FAIL: test_complex_symlinks_relative (test.test_pathlib.PathTest)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1728, in test_complex_symlinks_relative
    self._check_complex_symlinks('.')
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1692, in _check_complex_symlinks
    self.assertEqual(str(p), BASE)
AssertionError: 'C:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp' != 'c:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp'
- C:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^
+ c:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^

      ======================================================================
      FAIL: test_complex_symlinks_relative_dot_dot (test.test_pathlib.PathTest)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1732, in test_complex_symlinks_relative_dot_dot
    self._check_complex_symlinks(os.path.join('dirA', '..'))
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1692, in _check_complex_symlinks
    self.assertEqual(str(p), BASE)
AssertionError: 'C:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp' != 'c:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp'
- C:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^
+ c:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^

      ======================================================================
      FAIL: test_complex_symlinks_absolute (test.test_pathlib.WindowsPathTest)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1724, in test_complex_symlinks_absolute
    self._check_complex_symlinks(BASE)
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1692, in _check_complex_symlinks
    self.assertEqual(str(p), BASE)
AssertionError: 'C:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp' != 'c:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp'
- C:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^
+ c:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^

      ======================================================================
      FAIL: test_complex_symlinks_relative (test.test_pathlib.WindowsPathTest)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1728, in test_complex_symlinks_relative
    self._check_complex_symlinks('.')
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1692, in _check_complex_symlinks
    self.assertEqual(str(p), BASE)
AssertionError: 'C:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp' != 'c:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp'
- C:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^
+ c:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^

      ======================================================================
      FAIL: test_complex_symlinks_relative_dot_dot (test.test_pathlib.WindowsPathTest)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1732, in test_complex_symlinks_relative_dot_dot
    self._check_complex_symlinks(os.path.join('dirA', '..'))
  File "c:\Users\Morgane\Documents\000\Dev\CPython\cpython\lib\test\test_pathlib.py", line 1692, in _check_complex_symlinks
    self.assertEqual(str(p), BASE)
AssertionError: 'C:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp' != 'c:\\Users\\Morgane\\Documents\\000\\Dev\\[53 chars]_tmp'
- C:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^
+ c:\Users\Morgane\Documents\000\Dev\CPython\cpython\build\test_python_6060\@test_6060_tmp
? ^

      Ran 335 tests in 1.455s

      FAILED (failures=6, skipped=93)
      test test_pathlib failed
      1 test failed:
      test_pathlib

    26. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions