Repository navigation
ast.literal_eval() doesn't do what the documentation says #66715
Description
| BPO | 22525 |
|---|---|
| Nosy | @birkenfeld, @rhettinger, @ezio-melotti, @bitdancer |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2014-11-05.19:21:44.295>
created_at = <Date 2014-09-30.14:57:19.911>
labels = ['type-feature', 'docs']
title = "ast.literal_eval() doesn't do what the documentation says"
updated_at = <Date 2014-11-05.19:27:22.775>
user = 'https://bugs.python.org/BehdadEsfahbod'bugs.python.org fields:
activity = <Date 2014-11-05.19:27:22.775>
actor = 'python-dev'
assignee = 'docs@python'
closed = True
closed_date = <Date 2014-11-05.19:21:44.295>
closer = 'python-dev'
components = ['Documentation']
creation = <Date 2014-09-30.14:57:19.911>
creator = 'Behdad.Esfahbod'
dependencies = []
files = ['36764']
hgrepos = []
issue_num = 22525
keywords = ['patch']
message_count = 12.0
messages = ['227941', '227956', '227963', '227965', '227966', '227967', '227986', '227988', '230670', '230706', '230707', '230708']
nosy_count = 7.0
nosy_names = ['georg.brandl', 'rhettinger', 'ezio.melotti', 'r.david.murray', 'docs@python', 'python-dev', 'Behdad.Esfahbod']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue22525'
versions = []
Activity
BehdadEsfahbod commentedon Sep 30, 2014
The documentation says:
"""
Safely evaluate an expression node or a string containing a Python expression. The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and None.
This can be used for safely evaluating strings containing Python expressions from untrusted sources without the need to parse the values oneself.
"""
This makes me to believe that this is a useful replacement for eval() that is safe. However, it fails to make it clear that it parses **one literal**, NOT an expression. Ie. it can't handle "2*2". Weirdly enough, at least with my Python 3.2.3, it does handle "2+2" with no problem.
This seriously limits the usefulness of this function. Is there really no equivalent that parses simple expressions of literals?
birkenfeld commentedon Sep 30, 2014
I agree that the wording can be improved. The term "expression" is used here loosely, but should be clarified.
"a+b" with numbers a, b must be handled in order to evaluate complex literals as in "1+1j".
Allowing operator expressions with literals is possible, but much more complex than the current implementation. A simple implementation is not safe: you can induce basically unbounded CPU and memory usage with no effort (try "9**9**9" or "[None] * 9**9").
As for the usefulness, this function is useful to "read back" literal values and containers as stringified by repr(). This can for example be used for serialization in a format that is similar to but more powerful than JSON.
BehdadEsfahbod commentedon Sep 30, 2014
I think it should be made much more clear that this is not a blanket "safe eval() replacement".
Re complex literals, note that Python 2.7.x only implemented the binary plus operator if the second argument was complex. This seems to have been relaxed in Python 3.
Regarding DoS attack with a safe eval(), I understand the concern, but that's still a huge improvement over security risks of eval().
birkenfeld commentedon Sep 30, 2014
The function is still called literal_eval(), not safe_eval().
I'm not saying a safe eval() isn't useful. But an implementation that is only partly safe is not going to help anyone.
birkenfeld commentedon Sep 30, 2014
Attaching proposed doc change.
BehdadEsfahbod commentedon Sep 30, 2014
Thanks. In your proposed text:
+ Safely evaluate an expression node or a string containing a Python literal or container display.
I suggest changing it to "...containing a single Python literal or..."
Ie, adding "single".
bitdancer commentedon Sep 30, 2014
It should probably also say "or a container display containing only literals". (That's a lot of "contain" :)
(I had the same confusion the first time I read these docs, BTW).
birkenfeld commentedon Sep 30, 2014
Sure, feel free to reword.
An example or two might also be helpful.
rhettinger commentedon Nov 5, 2014
Georg's proposed wording reads well and is clearer than the current wording. The patch is ready to apply.
python-dev commentedon Nov 5, 2014
New changeset 5c5909740026 by Georg Brandl in branch '3.4':
Closes bpo-22525: clarify documentation for ast.literal_eval().
https://hg.python.org/cpython/rev/5c5909740026
birkenfeld commentedon Nov 5, 2014
Thanks, Raymond.
python-dev commentedon Nov 5, 2014
New changeset 3e8d3c4bc17e by Georg Brandl in branch '2.7':
Closes bpo-22525: clarify documentation for ast.literal_eval().
https://hg.python.org/cpython/rev/3e8d3c4bc17e