Repository navigation
shutil.get_terminal_size() throws ValueError if stdout is detached, no fallback #69154
Description
| BPO | 24966 |
|---|---|
| Nosy | @bitdancer, @eryksun, @iritkatriel |
| Superseder |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2021-12-02.23:22:51.969>
created_at = <Date 2015-08-30.22:59:13.397>
labels = ['type-bug', 'expert-IO']
title = 'shutil.get_terminal_size() throws ValueError if stdout is detached, no fallback'
updated_at = <Date 2021-12-02.23:22:51.966>
user = 'https://bugs.python.org/rsc1975'bugs.python.org fields:
activity = <Date 2021-12-02.23:22:51.966>
actor = 'eryksun'
assignee = 'none'
closed = True
closed_date = <Date 2021-12-02.23:22:51.969>
closer = 'eryksun'
components = ['IO']
creation = <Date 2015-08-30.22:59:13.397>
creator = 'rsc1975'
dependencies = []
files = []
hgrepos = []
issue_num = 24966
keywords = []
message_count = 13.0
messages = ['249372', '249374', '249375', '249387', '249391', '249405', '249411', '249423', '249435', '249443', '249462', '407554', '407556']
nosy_count = 4.0
nosy_names = ['r.david.murray', 'eryksun', 'rsc1975', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '26801'
type = 'behavior'
url = 'https://bugs.python.org/issue24966'
versions = ['Python 3.4']
Activity
rsc1975 commentedon Aug 30, 2015
After the stdout stream has been overwritten and detached, the method shutils.get_terminal_size throws a ValueError exception and no fallback value is returned.
Code to reproduce it:
>>> import os, sys, codecs, shutils >>> sys.stdout = codecs.getwriter("utf-8")(sys.stdout.detach()) >>> os.get_terminal_size() os.terminal_size(columns=160, lines=37) >>> shutil.get_terminal_size((200, 30)) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python3.4/shutil.py", line 1057, in get_terminal_size size = os.get_terminal_size(sys.__stdout__.fileno()) ValueError: underlying buffer has been detachedIs this the expected behavior ? IMHO, the given fallback values should be returned.
The official doc: "If the terminal size cannot be successfully queried, either because the system doesn’t support querying, or because we are not connected to a terminal, the value given in fallback parameter is used. fallback defaults to (80, 24) which is the default size used by many terminal emulators"
BTW, the function os.get_terminal_size() returns the correct size when it's invoked without parameters, maybe It could be a good fallback if sys.__stdout__ is not available and there isn't any user fallback values.
[-]shutil.get_terminal_size() throws ValueError is stdout is detached, no fallback[/-][+]shutil.get_terminal_size() throws ValueError if stdout is detached, no fallback[/+]bitdancer commentedon Aug 31, 2015
See also bpo-24920.
bitdancer commentedon Aug 31, 2015
In this case it might be better to leave the error. The buffer being detached would seem to be a programming error. What would be the use case for using get_terminal_size in a program that detaches the standard stream buffer?
rsc1975 commentedon Aug 31, 2015
In my case, I modify the stdout but the error was thrown by a third party (click lib), I need to overwrite the stdout because the program has a CLI and the locale env in the terminal seems to be wrong:
codecs.lookup(locale.getpreferredencoding()).name == 'ascii'
sys.getdefaultencoding() == 'utf-8'
For these cases I try to fix it forcing the the IO encoding to the value given by sys.getdefaultencoding(), otherwise the script in a CLI won't work properly.
I can apply a monkey-patch on the third party method that return "os.get_terminal_size()" if a ValueError exception is thrown, actually, The script is working now with the monkey-patch, but I thought that a bit of more "magic" in get_terminal_size would help to deal with the encoding issues in python3 CLI programs.
rsc1975 commentedon Aug 31, 2015
I've added a new issue: http://bugs.python.org/issue24968 that is related with this one, because is the root cause to modify the stdin/stdout streams.
bitdancer commentedon Aug 31, 2015
So, the problem is needing to compensate for a broken (according to posix) locale setup. I don't think quieting get_terminal_size is necessarily the answer here. See bpo-19846 for more than you want to know about the ASCII locale issue.
See also bpo-15216, which is probably a solution to your problem, but is an open issue.
rsc1975 commentedon Aug 31, 2015
Thank you for the references, changing the encoding of an open stream sounds interesting, but It's targeted for v. 3.6 so It seems a bit far away.
Anyway, as I said I fixed on my side with monkey-patching on the third-party lib, the broken locale is just the context, just to explain how I got this issue, I opened this because I honestly though that could be better to return the fallback value that raise an exception, but you know better the "big picture" so, if It's better as is, It's ok for me.
BTW, I don't find the opened issue to use utf8 as default instead ascii (except for Windows)
bitdancer commentedon Aug 31, 2015
Yeah, I think I was thinking of bpo-19846, which was fixed by using surrogateescape on the standard streams instead of "guessing" utf-8.
I'm not utterly opposed to turning the ValueError into a default return value, but as you can see, the fact that the ValueError was returned pointed toward the deeper problem (needing to have a way to change the encoding of an already-existing TextIO stream).
rsc1975 commentedon Aug 31, 2015
I totally agree, on the other hand, if the detach() method can cause that some core methods like shutil.get_termina_size() raise an exception then I think that should be warned in the doc of detach().
In all places that I've seen, the detach() method is used to overwrite the stdout in py3, so a sort of advice about the using of detach() It'd save some problems.
bitdancer commentedon Sep 1, 2015
"After the raw stream is detached, the buffer is in an unusable state" pretty much says everything that needs to be said, doesn't it? It certainly seems like anything more does not belong in the detach docs. A mention could possibly be added to the sys.stdin/etc docs, though.
rsc1975 commentedon Sep 1, 2015
Yeah, sorry, actually I was thinking in the stdin/out section, the detach() doc itself is Ok, the problem comes when is called in stdin/out.
iritkatriel commentedon Dec 2, 2021
I am unable to reproduce this, was it fixed?
eryksun commentedon Dec 2, 2021
This was fixed in 3.5.