tempfile seems to treat a file as a directory when processing an exception in the onerror()

@serhiy-storchaka

BPO	43153
Nosy	@serhiy-storchaka, @Fidget-Spinner, @akulakov, @mehwhatever0
PRs	bpo-43153: Don't mask `PermissionError` with `NotADirectoryError` during tempdirectory cleanup #29940

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-02-07.09:52:44.316>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'tempfile seems to treat a file as a directory when processing an exception in the onerror()'
updated_at = <Date 2021-12-06.14:13:37.485>
user = 'https://github.com/mehwhatever0'

bugs.python.org fields:

activity = <Date 2021-12-06.14:13:37.485>
actor = 'kj'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-02-07.09:52:44.316>
creator = 'mehwhatever0'
dependencies = []
files = []
hgrepos = []
issue_num = 43153
keywords = ['patch']
message_count = 4.0
messages = ['386587', '397118', '407376', '407435']
nosy_count = 4.0
nosy_names = ['serhiy.storchaka', 'kj', 'andrei.avk', 'mehwhatever0']
pr_nums = ['29940']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43153'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

>>> import sys, tempfile, os
>>> sys.version
'3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)]'
>>> folder = tempfile.TemporaryDirectory()
>>> file = open(os.path.join(folder.name, "example.txt"), 'wb')
>>> folder.cleanup()
Traceback (most recent call last):
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\shutil.py", line 616, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] Процесс не может получить доступ к файлу, так как этот файл занят другим процессом: 'C:\\Users\\user0\\AppData\\Local\\Temp\\tmpin8xeb9b\\example.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\tempfile.py", line 801, in onerror
    _os.unlink(path)
PermissionError: [WinError 32] Процесс не может получить доступ к файлу, так как этот файл занят другим процессом: 'C:\\Users\\user0\\AppData\\Local\\Temp\\tmpin8xeb9b\\example.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\tempfile.py", line 830, in cleanup
    self._rmtree(self.name)
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\tempfile.py", line 812, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\shutil.py", line 740, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\shutil.py", line 618, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\tempfile.py", line 804, in onerror
    cls._rmtree(path)
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\tempfile.py", line 812, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\shutil.py", line 740, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\shutil.py", line 599, in _rmtree_unsafe
    onerror(os.scandir, path, sys.exc_info())
  File "C:\Users\user0\AppData\Local\Programs\Python\Python39\lib\shutil.py", line 596, in _rmtree_unsafe
    with os.scandir(path) as scandir_it:
NotADirectoryError: [WinError 267] Неверно задано имя папки: 'C:\\Users\\user0\\AppData\\Local\\Temp\\tmpin8xeb9b\\example.txt'

This is a duplicate of https://bugs.python.org/issue43219 .

My last comment was wrong, the issue I linked is unrelated.

I think the actual issue here is the code here:

cpython/Lib/tempfile.py

Line 826 in 4b97d97

except (IsADirectoryError, PermissionError):

If path is a file that causes a permission error, rmtree is called and causes a NotADirectory error.

This can be confusing when debugging and can lead to a bug if user's code tries to catch a PermissionError it expects but instead gets NotADirectory error.

A solution is probably to check if path is a file and error is PermissionError and if ignore_errors=False, re-raise it instead of calling rmtree(). If ignore_errors=True, return instead of calling rmtree().

I don't have windows so can't test the OP code with this approach.

Serhiy: adding you since you wrote def onerror() in tempfile module, do you think that is the cause of the issue and the right solution for it?

I agree with Andrei. I'll test this approach on Windows.

pythongh-87319: Simplify TemporaryDirectory cleanup using os.path.isj…

gh-87319: Simplify TemporaryDirectory cleanup using os.path.isjunctio…

pythongh-87319: Simplify TemporaryDirectory cleanup using os.path.isj…

[3.12] gh-87319: Simplify TemporaryDirectory cleanup using os.path.is…

pythongh-87319: Simplify TemporaryDirectory cleanup using os.path.isj…

Revert cleaning cache changes on Windows (#3217)

Also trap NotADirectoryError, masking the PermissionError.

Navigation Menu

Repository navigation

tempfile seems to treat a file as a directory when processing an exception in the onerror() #87319

Linked PRs

5 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

tempfile seems to treat a file as a directory when processing an exception in the onerror() #87319

Description

Linked PRs

Activity

mehwhatever0 commented on Feb 7, 2021

akulakov commented on Jul 7, 2021

akulakov commented on Nov 30, 2021

serhiy-storchaka commented on Dec 1, 2021

5 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions