Repository navigation
ast.literal_eval() shouldn't accept booleans as numbers in AST #77074
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2020-05-16.19:59:42.950>
created_at = <Date 2018-02-21.09:56:21.602>
labels = ['3.8', 'type-feature', 'library']
title = "ast.literal_eval() shouldn't accept booleans as numbers in AST"
updated_at = <Date 2020-05-16.19:59:42.949>
user = 'https://github.com/serhiy-storchaka'bugs.python.org fields:
activity = <Date 2020-05-16.19:59:42.949>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2020-05-16.19:59:42.950>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2018-02-21.09:56:21.602>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 32893
keywords = ['patch']
message_count = 6.0
messages = ['312485', '312498', '312500', '312687', '312700', '369063']
nosy_count = 4.0
nosy_names = ['rhettinger', 'terry.reedy', 'benjamin.peterson', 'serhiy.storchaka']
pr_nums = ['5798', '340']
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue32893'
versions = ['Python 3.8']
Activity
serhiy-storchaka commentedon Feb 21, 2018
Currently ast.literal_eval() accepts AST representing expressions like "+True" or "True+2j" if constants are represented as Constant. This is because the type of the value is tested with
isinstance(left, (int, float))and since bool is a subclass of int it passes this test.The proposed PR makes ast.literal_eval() using tests for exact type. I don't think it is worth backporting since it affects only passing AST to ast.literal_eval(). Usually ast.literal_eval() is used for evaluating strings.
rhettinger commentedon Feb 21, 2018
What harm comes from accepting expressions like "+True" or "True+2j"? While weird looking, those are valid Python expressions, so this doesn't seem like a bug. A key feature of booleans is that they are interoperable with integers.
serhiy-storchaka commentedon Feb 21, 2018
This doesn't look like Python literal. And if the function accepts a one particular non-literal the user can except that it accepts other looking similarly non-literal, that is false.
Actually ast.literal_eval("+True") is error. But ast.literal_eval(ast.UnaryOp(ast.UAdd(), ast.Constant(True))) is successful by oversight.
And look at this from other side. What is the benefit of accepting "+True"? This doesn't make the code simpler.
terryjreedy commentedon Feb 24, 2018
Whoops, my previous response was wrong as written because I wrongly thought that if literal_eval accepts number_literal + imaginary_literal, it would also accept number_literal + number_literal. I am replacing it with the following.
The ast.literal_eval doc says
"The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and None."
Since 'True', '(True,)', '[True]', '{True:True}', '+1', '-1' and '1+1j' are accepted as 'literal structures', it seems arbitrary that at least '+True' and 'True+1j' are not. ('+1', '1+1j' and especially '-1' seem to violate the limitations to 'literal', 'container', and 'no operator', but that is a different issue.)
I strongly agree that the acceptable string inputs and acceptable AST inputs should match. The question is which version of the domain should be changed. I would at least mildly prefer that the issue be "ast.literal_eval should consistly treat False and True the same as 0 and 1.", which means expanding the string version. As Raymond said, this is the general rule in Python. What is the benefit of having a different rule for this one function?
serhiy-storchaka commentedon Feb 24, 2018
The definition of "Python literal structures" is not specified, but it is implied that ast.literal_eval() should accept signed numbers and tuple/list/set/dict displays consisting of "Python literal structures".
ast.literal_eval() accepts unary minus for supporting negative numbers. In Python 2 "-42" was parsed as a single AST node Num(42), but in Python 3 it is parsed as UnaryOp(USub(), Num(42)). ast.literal_eval() accepts binary plus and minus for supporting complex numbers (there are oddities in this). The support of unary plus was added somewhere for unknown reason (see 884c71cd8dc6, no issue number), but it doesn't harm.
Below in the documentation: "It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing." Arbitrarily complex expressions involving addition and subtraction (like "2017-10-10") was supported in Python 3 (but not in Python 2), but it was fixed in 3.7 (see bpo-31778). It doesn't support other operations or operations with non-numeric literals. It is oversight that UnaryOp(USub(), Constant(True)) is still accepted.
serhiy-storchaka commentedon May 16, 2020
No longer reproduced.