This repository was archived by the owner on Jul 5, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Jul 5, 2023. It is now read-only.
Are there any specs for the typed AST? #4
Description
I've started experimenting with this project and it works really nice so far, but what I find that is missing is the description of the typed AST. For ordinary AST there is some effort going on here: https://greentreesnakes.readthedocs.io/en/latest/nodes.html . However, is there any kind of documentation with regard to how typed_ast.ast35 should look?
E.g. type_comment seems to be treated as just a string. However what is done e.g. in mypy is the type comment is treated as code. Is that intended?
my_string = None # type: typing.Optional[str]
Module(
body=[Assign(
targets=[Name(
id='my_string',
ctx=Store())],
value=NameConstant(value=None),
type_comment='typing.Optional[str]')],
type_ignores=[])
Another example is here, where type annotations are not treated as strings, but are parsed normally:
def function(a: int, b: int) -> int:
return a + b
Module(
body=[FunctionDef(
name='function',
args=arguments(
args=[
arg(
arg='a',
annotation=Name(
id='int',
ctx=Load())),
arg(
arg='b',
annotation=Name(
id='int',
ctx=Load()))],
vararg=None,
kwonlyargs=[],
kw_defaults=[],
kwarg=None,
defaults=[]),
body=[Return(value=BinOp(
left=Name(
id='a',
ctx=Load()),
op=Add(),
right=Name(
id='b',
ctx=Load())))],
decorator_list=[],
returns=Name(
id='int',
ctx=Load()),
type_comment=None)],
type_ignores=[])
Activity
ddfisher commentedon Jun 16, 2016
The difference between
ast35and Python 3.5'sastmodule are documented briefly in the module docstring.At parse time, type comments and type annotations are very different beasts. Type annotations are parsed as expressions because they actually are expressions that are executed at function definition time. (This behavior is already part of the standard Python 3.5
astmodule and is unchanged inast35.) Type comments are always returned as strings in the initial parse, but mypy calls back into the parser (withmode='eval') to parse them as expressions.That said, it seems like it might now make sense to parse the type comments as expressions. I'll think about doing that.
mbdevpl commentedon Jun 23, 2016
Thanks a lot for the answer! This will do. Next time I should read the docstrings more closely.
As a side note, I've published an experimental unparser for your typed-ast, called typed-astunparse: https://github.com/mbdevpl/typed-astunparse - I use it for my research, but if it would have any use otherwise, I'd be very glad :)
BTW, since you seem to be on the forefront of Python development, do you happen to know of any discussion about introducing AST unparsing into core?
Actually, there has been a hidden
unparse.pyscript in Python source distribution for a very long time now, someone even extracted it and repackaged into astunparse module... It's just thatast.unparse()is not that hard to imagine, and it would be so nice :) Andcompile()flag enabling type comments parsing would also be sooo nice :)Or, question more related to typed-ast project: would typed ast unparser be welcome in this module, if there happens to be a volunteer to write it? It's ok to have ast/astunparse typed-ast/typed-astunparse - it's just that for me these really belong together.
Asking just out of curiosity, please answer if you happen to have some free time :)
gvanrossum commentedon Jun 23, 2016
I'm curious about the use case for unparsing without preservation of non-type comments. How common is this need? Wouldn't you be better off saving the original source code?
mbdevpl commentedon Jun 24, 2016
I'm doing a kind of source-to-source translation project. The original code (and sometimes target code) is not necessarily Python. I'm experimenting with type hints as a method of increasing performance.
gvanrossum commentedon Jun 24, 2016
OK, so the output is not meant for human consumption, right? You're essentially "transpiling" some language to Python so it can be executed. Then the ast unparser makes total sense.
Merge pull request python#4 from nneonneo/master