This repository was archived by the owner on Mar 8, 2020. It is now read-only.
Repository navigation
This repository was archived by the owner on Mar 8, 2020. It is now read-only.
Suspicious positions assignment #152
Description
Run python code to reproduce:
import bblfsh
content = "from os import path".encode()
r = bblfsh.BblfshClient("0.0.0.0:9432").parse("", "Python", contents=content)
for node in bblfsh.iterator(r.uast, 0):
print(node.token,
[bblfsh.role_name(r) for r in node.roles],
node.start_position.line,
node.start_position.col)You get:
['FILE', 'MODULE'] 0 0
['IMPORT', 'DECLARATION', 'STATEMENT'] 1 6
os ['IMPORT', 'PATHNAME', 'IDENTIFIER'] 0 0
path ['IMPORT', 'PATHNAME', 'IDENTIFIER'] 0 0
What we can see: ['IMPORT', 'DECLARATION', 'STATEMENT'] starts at 1st line and 6th column.
Why 6? there is an identifier os so I expect to see such position in next Node.
As I understand now we provide positions from native driver only, so it can be not a bug, but Python parser related thing. Feel free to close it if it so.
Activity
juanjux commentedon Apr 6, 2018
Looks like the odd (mis?)behaviour is in the native AST for nodes with the internal type
ImportFrom(vsImportwhere the position is correct). Looks like it sets it as the position of the module importing the symbols:{ "PY3AST":{ "ast_type":"Module", "body":[ { "ast_type":"ImportFrom", "col_offset":6, "end_col_offset":7, "end_lineno":1, "level":0, "lineno":1, "module":"os", "names":[ { "asname":null, "ast_type":"alias", "name":"path" } ] } ] } }I'll take a look at our layer between the native AST and the UAST to see if the problem could be there, but it doesn't look good.
juanjux commentedon Apr 6, 2018
Good news, the problem was in our layer (the first module is taken as token for ImportFrom nodes and then we do a pass synchronizing token positions, which was wrong in this case). I'll make a PR in a few minutes.
juanjux commentedon Apr 6, 2018
PR #153 should fix the problem (do a review if you want to accelerate merging, it's just three lines).