Repository navigation
role assignment in a = b = c #97
Description
If you have this code:
a = b = c
var1 == var2 == var3
var4 < var5 < var6And run UAST extraction you have strange role assignment.
Please take a look at the code. https://gist.github.com/zurk/66a3045746287bdb5002c0812b94f611
Here is output (the same gist):
https://gist.github.com/zurk/66a3045746287bdb5002c0812b94f611#file-output
Comments for output:
//*[@roleIdentifier] :
<a>: ['LEFT', 'IDENTIFIER', 'EXPRESSION']
<b>: ['LEFT', 'IDENTIFIER', 'EXPRESSION']
<c>: ['RIGHT', 'IDENTIFIER', 'EXPRESSION']
<var2>: ['IDENTIFIER', 'EXPRESSION']
<var3>: ['IDENTIFIER', 'EXPRESSION']
<var1>: ['IDENTIFIER', 'EXPRESSION', 'EXPRESSION', 'BINARY', 'LEFT']
<var5>: ['IDENTIFIER', 'EXPRESSION']
<var6>: ['IDENTIFIER', 'EXPRESSION']
<var4>: ['IDENTIFIER', 'EXPRESSION', 'EXPRESSION', 'BINARY', 'LEFT']
I am not sure how it should be but at least var3 and var6 are on right side. :)
Why we have 'EXPRESSION', 'BINARY' for var1 and var4? I think it is just EXPRESSION as for all others. BINARY is upper level in UAST tree.
Also, I am not sure that we can call second and last expressions as binary at all.
Maybe, the first line of code can be considered as two binary expressions.
//*[@roleLeft] :
<a>: ['LEFT', 'IDENTIFIER', 'EXPRESSION']
<b>: ['LEFT', 'IDENTIFIER', 'EXPRESSION']
<var1>: ['IDENTIFIER', 'EXPRESSION', 'EXPRESSION', 'BINARY', 'LEFT']
<var4>: ['IDENTIFIER', 'EXPRESSION', 'EXPRESSION', 'BINARY', 'LEFT']
Ok, it can be true for 'a' and 'b'.
//*[@roleRight] :
<c>: ['RIGHT', 'IDENTIFIER', 'EXPRESSION']
<>: ['EXPRESSION', 'BINARY', 'RIGHT']
<>: ['EXPRESSION', 'BINARY', 'RIGHT']
tokens var4 and var6 missing?
//*[@roleBinary] :
<>: ['BINARY', 'THIS', 'EXPRESSION']
<>: ['EXPRESSION', 'BINARY']
<>: ['EXPRESSION', 'BINARY', 'RIGHT']
<var1>: ['IDENTIFIER', 'EXPRESSION', 'EXPRESSION', 'BINARY', 'LEFT']
<>: ['EXPRESSION', 'BINARY', 'OPERATOR']
<==>: ['BINARY', 'OPERATOR', 'EQUAL']
<==>: ['BINARY', 'OPERATOR', 'EQUAL']
<>: ['EXPRESSION', 'BINARY']
<>: ['EXPRESSION', 'BINARY', 'RIGHT']
<var4>: ['IDENTIFIER', 'EXPRESSION', 'EXPRESSION', 'BINARY', 'LEFT']
<>: ['EXPRESSION', 'BINARY', 'OPERATOR']
<<>: ['BINARY', 'OPERATOR', 'LESS_THAN']
<<>: ['BINARY', 'OPERATOR', 'LESS_THAN']
Everything is fine for a = b = c statement maybe except THIS role, but I am not sure. Please take a look at the defenition and if it suitable here.
And there is a mess for second two lines of code.
<>: ['EXPRESSION', 'BINARY', 'OPERATOR']
seems that it is the node for the full ternary operator because it is without a token. Not sure.
Hope it helps to investigate the problem.
Activity
juanjux commentedon Sep 21, 2017
Let me generate UAST tree for those to see exactly what's in the tree and I'll be back at you.
juanjux commentedon Sep 21, 2017
Mmmm, I'm not getting the "THIS" role anywhere, could you try again with the python driver 1.0.1?
juanjux commentedon Sep 22, 2017
These are really several things so I will keep is organized in a list. But the first thing to understand, it that the Python AST doesn't have any kind of ternary operators, it will create a tree of pairs for things like
a = b = c(withabeing the left side, and the right side being a subtree with another assignment expression which in turn would havebandc).When having doubts, it help to see the full UAST tree instead of only the search results because some nodes are grouped by parent nodes with the right roles and without token (like what happens here with the "right" nodes for comparisons) and in the filtered search this is not always clear.
THISrole: can't reproduce with the last version.RIGHTrole: Working as intended; they have a parent node with theRIGHTrole in the UAST (the one that shows without token in your search for right).BINARYrole: confirmed, probably they shouldn't have it, they have a parent node with the role already.OPERATORrole**: Confirmed, it should not have it since it's a grouping node and the children (the operators) already have the role.zurk commentedon Sep 22, 2017
@juanjux oops, It was a stupid mistake in my gist, I fix it. Please, take a look.
https://gist.github.com/zurk/66a3045746287bdb5002c0812b94f611
Also, I extend it with
THISand role numbers and find the strange situation. I cannot findTHISby xpath query, but role # 105 has nameTHISis protobuf modules. Maybe there is a mistake?PS. Last server and driver versions.
zurk commentedon Sep 22, 2017
About structure.
Create a new gist that outputs structure http://nbviewer.jupyter.org/gist/zurk/d7d7d230fa14ffeef0f1c6b01be533aa
(by the way, I decided to use Jupyter notebooks, is it ok for you? I think it more convenient for both sides)
So, there is some mess in output (tokens --- roles):
Everything is fine with
RIGHTrole.But from this subtree
no one can understand binary operation of this
BINARYNode. For some reason bothOPERATORin one subtree.So, all of these somehow related to https://docs.python.org/3/reference/expressions.html#comparisons
juanjux commentedon Sep 22, 2017
Yes, that's the structure of the Compare node in the native AST, check the comment in the description :)
https://greentreesnakes.readthedocs.io/en/latest/nodes.html#Compare
PS: the boolean nodes and the default/kwarg arguments are also odd in their own way. These are the first I want to change in the future when we can normalize some parts of the tree structure.
zurk commentedon Sep 22, 2017
Ok, I like this plan.
So after bug fixing, we can create one more about enhancement.
juanjux commentedon Sep 22, 2017
About the mysterious THIS role, we've just fixed a bug with a desync of the role list, I wasn't seeing it because I had the good version locally, the fix is in Python driver 1.0.2 and Python client 1.0.1, check again when you can to see if that role goes away.
juanjux commentedon Sep 28, 2017
Fixed by #112