Skip to content

Python startup should not require passwd entry #54705

Closed
Closed
Python startup should not require passwd entry#54705
Assignees
merwok
Labels
3.7 (EOL)end of life3.8 (EOL)end of lifestdlibStandard Library Python modules in the Lib/ directorytestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

Description

@bbi5291
bbi5291
mannequin
opened on Nov 22, 2010
BPO 10496
Nosy @loewis, @birkenfeld, @arekm, @vstinner, @tiran, @tarekziade, @ned-deily, @merwok, @bitdancer, @IIIIllllIIIIllllIIIIllllIIIIllllIIIIll, @serhiy-storchaka, @matrixise, @seirl, @yan12125, @surajssd, @izbyshev, @Cinerar, @miss-islington, @pxinwr
PRs
  • bpo-10496 Runing Inside docker container from non root user raises KeyError #3403
  • bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error #10919
  • [3.7] bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) #10924
  • [3.7] bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) #10924
  • [3.7] bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) #10924
  • [3.6] bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) #10925
  • [3.6] bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) #10925
  • [2.7] bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) #10930
  • [2.7] bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) #10930
  • [2.7] bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) #10930
  • bpo-10496: distutils check_environ() handles getpwuid() error #10931
  • bpo-10496: distutils check_environ() handles getpwuid() error #10931
  • bpo-10496: distutils check_environ() handles getpwuid() error #10931
  • [3.7] bpo-10496: distutils check_environ() handles getpwuid() error (GH-10931) #11212
  • [2.7] bpo-10496: distutils check_environ() handles getpwuid() error (GH-10931) #11213
  • bpo-31904: posixpath.expanduser() handles user home of None #23530
    		•
    Files
  • bug.c
  • nonexistent_user.patch
  • user.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/merwok'
closed_at = <Date 2018-12-18.16:37:38.410>
created_at = <Date 2010-11-22.00:59:48.070>
labels = ['3.7', 'tests', '3.8', 'type-bug', 'library']
title = 'Python startup should not require passwd entry'
updated_at = <Date 2020-12-17.20:40:51.564>
user = 'https://bugs.python.org/bbi5291'

    bugs.python.org fields:

    activity = <Date 2020-12-17.20:40:51.564>
actor = 'pxinwr'
assignee = 'eric.araujo'
closed = True
closed_date = <Date 2018-12-18.16:37:38.410>
closer = 'vstinner'
components = ['Library (Lib)', 'Tests']
creation = <Date 2010-11-22.00:59:48.070>
creator = 'bbi5291'
dependencies = []
files = ['19760', '21896', '47211']
hgrepos = []
issue_num = 10496
keywords = ['patch', 'needs review']
message_count = 50.0
messages = ['122086', '122272', '122640', '132771', '133626', '133674', '133675', '133676', '133742', '133743', '133830', '134984', '134987', '135044', '135047', '135195', '135551', '135592', '135596', '155530', '155534', '156373', '193987', '257513', '270771', '272115', '282502', '282503', '282504', '302176', '302355', '303943', '304128', '331100', '331103', '331108', '331115', '331123', '331125', '331127', '331128', '331129', '331133', '331135', '331136', '331163', '332065', '332067', '332070', '332071']
nosy_count = 29.0
nosy_names = ['loewis', 'georg.brandl', 'arekm', 'vstinner', 'christian.heimes', 'nadeem.vawda', 'tarek', 'ned.deily', 'eric.araujo', 'grahamd', 'r.david.murray', 'neologix', 'filippo', 'xuanji', 'bbi5291', 'Denis.Barmenkov', 'aikinci', 'serhiy.storchaka', 'matrixise', 'zaytsev', 'antoine.pietri', 'yan12125', 'Jonathon Reinhart', 'surajd', 'izbyshev', 'cinerar', 'Nicholas Brown', 'miss-islington', 'pxinwr']
pr_nums = ['3403', '10919', '10924', '10924', '10924', '10925', '10925', '10930', '10930', '10930', '10931', '10931', '10931', '11212', '11213', '23530']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10496'
versions = ['Python 2.7', 'Python 3.7', 'Python 3.8']

    Activity

    1. bbi5291
      added
      type-bugAn unexpected behavior, bug, or error
      on Nov 22, 2010

    2. bbi5291 commented on Nov 22, 2010

      @bbi5291
      bbi5291mannequin
      on Nov 22, 2010
      MannequinAuthor

      This bug is Linux-specific.

      When Python cannot find the home directory of the user invoking it, it prints "'import site' failed; use -v for traceback".

      This occurs only when both of these conditions are met:

      1. /etc/passwd contains no entry for the current real UID
      2. the HOME environment variable does not exist

      To duplicate this bug, compile and run the attached program as root.
      (This program assumes that 12345 is not a legitimate user on your system, and that Python is at /usr/bin/python ; if this is not the case then edit it accordingly.)

    3. ned-deily commented on Nov 24, 2010

      @ned-deily
      ned-deily
      on Nov 24, 2010
      Member

      The problem is reproducible on a current Debian Linux system although with different results for current versions of Python (only security issues are accepted against 2.6). Unlike with 2.6, 3.1 reports no error. 2.7 and 3.2 both fail with an exception:

      Traceback (most recent call last):
  File "/usr/lib/python2.7/site.py", line 548, in <module>
    main()
  File "/usr/lib/python2.7/site.py", line 530, in main
    known_paths = addusersitepackages(known_paths)
  File "/usr/lib/python2.7/site.py", line 257, in addusersitepackages
    user_site = getusersitepackages()
  File "/usr/lib/python2.7/site.py", line 232, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/usr/lib/python2.7/site.py", line 222, in getuserbase
    USER_BASE = get_config_var('userbase')
  File "/usr/lib/python2.7/sysconfig.py", line 541, in get_config_var
    return get_config_vars().get(name)
  File "/usr/lib/python2.7/sysconfig.py", line 449, in get_config_vars
    _CONFIG_VARS['userbase'] = _getuserbase()
  File "/usr/lib/python2.7/sysconfig.py", line 198, in _getuserbase
    return env_base if env_base else joinuser("~", ".local")
  File "/usr/lib/python2.7/sysconfig.py", line 185, in joinuser
    return os.path.expanduser(os.path.join(*args))
  File "/usr/lib/python2.7/posixpath.py", line 256, in expanduser
    userhome = pwd.getpwuid(os.getuid()).pw_dir
KeyError: 'getpwuid(): uid not found: 12345'

    4. IIIIllllIIIIllllIIIIllllIIIIllllIIIIll commented on Nov 28, 2010

      @IIIIllllIIIIllllIIIIllllIIIIllllIIIIll
      IIIIllllIIIIllllIIIIllllIIIIllllIIIIllmannequin
      on Nov 28, 2010
      Mannequin

      I tried running bug.c using the svn head of python and got this:

      Could not find platform independent libraries <prefix>
      Could not find platform dependent libraries <exec_prefix>
      Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
      Fatal Python error: Py_Initialize: Unable to get the locale encoding
      LookupError: no codec search functions registered: can't find encoding
      Aborted

    5. DenisBarmenkov commented on Apr 1, 2011

      @DenisBarmenkov
      DenisBarmenkovmannequin
      on Apr 1, 2011
      Mannequin

      I saw similar error on Python 2.6 installed on freeBSD.
      I had test SVN server and wrote pre-commit hook using python. When remote developer commited his changes to repository, hook called os.path.expanduser and exception was raised:
      # File "/usr/local/lib/python2.6/posixpath.py", line 259, in expanduser
      # userhome = pwd.getpwuid(os.getuid()).pw_dir
      #KeyError: 'getpwuid(): uid not found: 12345'

      So its not a just-Linux issue.

    6. vstinner commented on Apr 12, 2011

      @vstinner
      vstinner
      on Apr 12, 2011
      Member

      This issue remembers me the issue bpo-6612 (failure if the current directory was removed): the fix was to ignore os.getcwd().

      Attached patch ignores os.path.expanduser() error (KeyError) and keeps ~ in the path. Example without HOME var and with an non existent user (uid 12345):

      ----------------------

      $ env -i ./python
>>> import sysconfig
>>> sysconfig.get_config_var('userbase')
'~/.local'
      >>> sysconfig.get_paths(scheme='posix_user', expand=False)
{'platstdlib': '{userbase}/lib/python{py_version_short}', 'platlib': '{userbase}/lib/python{py_version_short}/site-packages', 'purelib': '{userbase}/lib/python{py_version_short}/site-packages', 'stdlib': '{userbase}/lib/python{py_version_short}', 'scripts': '{userbase}/bin', 'include': '{userbase}/include/python{py_version_short}', 'data': '{userbase}'}

>>> sysconfig.get_paths(scheme='posix_user')              
{'platstdlib': '~/.local/lib/python3.3', 'platlib': '~/.local/lib/python3.3/site-packages', 'purelib': '~/.local/lib/python3.3/site-packages', 'stdlib': '~/.local/lib/python3.3', 'scripts': '~/.local/bin', 'include': '~/.local/include/python3.3', 'data': '~/.local'}

      Example with an existant user but without HOME var:
      ----------------------

      marge$ env -i ./python
>>> import sysconfig
>>> sysconfig.get_config_var('userbase')
'/home/haypo/.local'
>>> sysconfig.get_paths(scheme='posix_user')      
{'platstdlib': '/home/haypo/.local/lib/python3.3', 'platlib': '/home/haypo/.local/lib/python3.3/site-packages', 'purelib': '/home/haypo/.local/lib/python3.3/site-packages', 'stdlib': '/home/haypo/.local/lib/python3.3', 'scripts': '/home/haypo/.local/bin', 'include': '/home/haypo/.local/include/python3.3', 'data': '/home/haypo/.local'}
    7. vstinner
      changed the title [-]"import site failed" when Python can't find home directory[/-] [+]"import site failed" when Python can't find home directory (sysconfig._getuserbase)[/+] on Apr 12, 2011

    8. merwok commented on Apr 13, 2011

      @merwok
      merwok
      on Apr 13, 2011
      Member

      Can someone explain how it can happen that a user has no home directory?

    9. bbi5291 commented on Apr 13, 2011

      @bbi5291
      bbi5291mannequin
      on Apr 13, 2011
      MannequinAuthor

      I discovered this while I was implementing and testing a sandbox for automatic evaluation of programs.

    10. merwok commented on Apr 13, 2011

      @merwok
      merwok
      on Apr 13, 2011
      Member

      Can you be more precise? IOW, why is this a Python bug rather than a system misconfiguration? Note that I don’t know a lot about POSIX, so I’m open to change my mind.

    11. neologix commented on Apr 14, 2011

      @neologix
      neologixmannequin
      on Apr 14, 2011
      Mannequin

      I'm not sure whether POSIX warrants anything about this behavior, but nothing prevents a process from running with a UID not listed in /etc/passwd (or NIS, whatever). For example, sudo allows running a command with a UID not listed in the password database, see http://linux.die.net/man/5/sudoers :
      """
      targetpw

      If set, sudo will prompt for the password of the user specified by the -u flag (defaults to root) instead of the password of the invoking user. Note that this precludes the use of a uid not listed in the passwd database as an argument to the -u flag. This flag is off by default.
      """

      UIDs not backed by users are useful for example if you're working with a sandbox, or virtual users such as in some FTP servers http://www.proftpd.org/docs/howto/VirtualUsers.html :
      """
      Question: What makes a user "virtual", then?
      Answer: A virtual user is, quite simply, a user that is not defined in the system /etc/passwd file. This file associates a user name, given by the system administrator, to a user ID (commonly shortened to UID) and a group ID (GID), among other details. The Unix kernel does not deal with users in terms of their user names; it only "knows" about UIDs and GIDs. This means that an application like proftpd can look up the IDs to use for a given user name however it sees fit. Using /etc/passwd is not strictly required.
      """

    12. vstinner commented on Apr 14, 2011

      @vstinner
      vstinner
      on Apr 14, 2011
      Member

      Because the patch is simple (just add a try/except), I think that it doesn't matter if only few people use users without entry in /etc/passwd and we should fix this issue.

    13. merwok commented on Apr 15, 2011

      @merwok
      merwok
      on Apr 15, 2011
      Member

      It’s not just a try/except, it’s a behavior change: after the patch, paths returned by sysconfig may not be fully expanded paths. I would like Tarek to make a call on this.

    14. merwok
      assigned
      tarekziade
      on Apr 15, 2011

    15. vstinner commented on May 2, 2011

      @vstinner
      vstinner
      on May 2, 2011
      Member

      I would like Tarek to make a call on this.

      So Tarek, what do you think?

    16. 32 remaining items

    17. izbyshev commented on Dec 5, 2018

      @izbyshev
      izbyshevmannequin
      on Dec 5, 2018
      Mannequin

      Would it make sense to backport this fix in 3.6 and 3.7?

      I'd like to see it there, given that this bug surfaced in many use cases not involving any modern features or systemd at all.

    18. vstinner commented on Dec 5, 2018

      @vstinner
      vstinner
      on Dec 5, 2018
      Member

      New changeset f2f4555 by Victor Stinner in branch 'master':
      bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919)
      f2f4555

    19. miss-islington commented on Dec 5, 2018

      @miss-islington
      miss-islington
      on Dec 5, 2018
      Contributor

      New changeset 983d1ab by Miss Islington (bot) in branch '3.7':
      bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919)
      983d1ab

    20. vstinner commented on Dec 5, 2018

      @vstinner
      vstinner
      on Dec 5, 2018
      Member

      New changeset 31b635d by Victor Stinner in branch '3.6':
      bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) (GH-10925)
      31b635d

    21. vstinner
      added
      3.7 (EOL)end of life
      3.8 (EOL)end of life
      on Dec 5, 2018

    22. vstinner commented on Dec 5, 2018

      @vstinner
      vstinner
      on Dec 5, 2018
      Member

      New changeset b50b33b by Victor Stinner in branch '2.7':
      bpo-10496: posixpath.expanduser() catchs pwd.getpwuid() error (GH-10919) (GH-10930)
      b50b33b

    23. vstinner commented on Dec 18, 2018

      @vstinner
      vstinner
      on Dec 18, 2018
      Member

      New changeset 17d0c05 by Victor Stinner in branch 'master':
      bpo-10496: distutils check_environ() handles getpwuid() error (GH-10931)
      17d0c05

    24. miss-islington commented on Dec 18, 2018

      @miss-islington
      miss-islington
      on Dec 18, 2018
      Contributor

      New changeset 6e96fb4 by Miss Islington (bot) in branch '3.7':
      bpo-10496: distutils check_environ() handles getpwuid() error (GH-10931)
      6e96fb4

    25. vstinner commented on Dec 18, 2018

      @vstinner
      vstinner
      on Dec 18, 2018
      Member

      New changeset ea6b322 by Victor Stinner in branch '2.7':
      bpo-10496: distutils check_environ() handles getpwuid() error (GH-10931) (GH-11213)
      ea6b322

    26. vstinner commented on Dec 18, 2018

      @vstinner
      vstinner
      on Dec 18, 2018
      Member

      A few minor tests are still failing when the uid doesn't exist in the password database, but the main issue in pwdhas been fixed and distutils should work more or lesss. I close this 8 years old issue. Thanks to everyone who has been involved in helping to fix it!

    27. vstinner
      added
      stdlibStandard Library Python modules in the Lib/ directory
      testsTests in the Lib/test dir
      on Dec 18, 2018
    28. vstinner
      closed this as completedon Dec 18, 2018
    29. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.7 (EOL)end of life3.8 (EOL)end of lifestdlibStandard Library Python modules in the Lib/ directorytestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions