Driver returns twice the same comment #36

Activity

1. dpordomingo commented on Sep 17, 2018

   

   dpordomingo

   on Sep 17, 2018

   Last edited by dpordomingo

   MemberAuthor

   More actions

   For comment blocks, the same comment can be returned three times, not only two

   var user = [];
   /**
    * adds two numbers togeter
    */
   function add(first) {
   }

   
2. zurk commented on Sep 28, 2018

   

   zurk

   on Sep 28, 2018

   Last edited by zurk

   More actions

   I confirm that for drivers v1.2.0 and v2.2.0 And it happens not only for comments.
   This JS code produses next UAST tree:

   import { PassportConfiguratorrr } from '@freecodecamp/loopback-component-passport';

   #  Token                   Internal Role      Roles Tree                                         
                                                                                                    
   1  ||                      File               FILE                                               
   1  ||                      Program            ┣ MODULE                                           
   1  ||                      ImportDeclaration  ┃ ┣ STATEMENT, DECLARATION, IMPORT                 
   1  ||                      ImportSpecifier    ┃ ┃ ┣ IMPORT                                       
   1  |PassportConfigurator|  Identifier         ┃ ┃ ┃ ┣ EXPRESSION, IDENTIFIER, IMPORT             
   1  |PassportConfigurator|  Identifier         ┃ ┃ ┃ ┗ EXPRESSION, IDENTIFIER, IMPORT             
   1  |@freecodecamp/loopba|  StringLiteral      ┗ ┗ ┗ EXPRESSION, LITERAL, STRING, IMPORT, PATHNAME
   
3. dennwc commented on Sep 28, 2018

   

   dennwc

   on Sep 28, 2018

   Member

   More actions

   @zurk It doesn't show the internalRole field of these nodes. I'm quite sure they are different. It's probably a name and an alias fields (package-local name).
4. zurk commented on Sep 28, 2018

   

   zurk

   on Sep 28, 2018

   More actions

   oh, I just mixed, and column Internal Role corresponds to node.internal_type field. So, internal types are the same or we have a bug in there
5. dennwc commented on Sep 28, 2018

   

   dennwc

   on Sep 28, 2018

   Member

   More actions

   @zurk Yes, but I'm talking about a different field :) internal_type in UAST v1 corresponds to a @type field in v2, so yes, you will have two Identifiers there. But there is an InternalRole property in v1 which corresponds to a field name (or predicate) in v2. This field should have two different values in your example. So there are two different identifiers - one for the external import name and the second one for the internal name (alias for current file/package). Or at least this is what I assume, looking at the sample.

   But in case that @dpordomingo described, we can remove these duplicates from the tree safely. But for the case with names vs aliases, removing one of them will cause half of the UAST queries to break.

   The correct solution will require adding support for DAGs in SDK that I was advocating from the very beginning. This will happen later during v2 cycle, or even in v3.
6. zurk commented on Sep 28, 2018

   

   zurk

   on Sep 28, 2018

   More actions

   yeah, now I get it. If in Internal Role I use node.properties["internalRole"] I get

   #  Token                   Internal Role  Roles Tree                                         
                                                                                                
   1  ||                                     FILE                                               
   1  ||                      program        ┣ MODULE                                           
   1  ||                      body           ┃ ┣ STATEMENT, DECLARATION, IMPORT                 
   1  ||                      specifiers     ┃ ┃ ┣ IMPORT                                       
   1  |PassportConfigurator|  imported       ┃ ┃ ┃ ┣ EXPRESSION, IDENTIFIER, IMPORT             
   1  |PassportConfigurator|  local          ┃ ┃ ┃ ┗ EXPRESSION, IDENTIFIER, IMPORT             
   1  |@freecodecamp/loopba|  source         ┗ ┗ ┗ EXPRESSION, LITERAL, STRING, IMPORT, PATHNAME
   

   before it was node.internal_type on this place. UAST even more complicated than I think.

   👍React with 👍1Reacted by Denys Smirnov
7. bzz commented on Nov 29, 2018

   

   bzz

   on Nov 29, 2018

   Contributor

   More actions

   This seems still to be the case: the query to the same file //*/uast:Comment results in entities for me:

   {"@pos":{"@type":"uast:Positions","end":{"@type":"uast:Position","col":22,"line":1,"offset":21},"start":{"@type":"uast:Position","col":8,"line":1,"offset":7}},"@type":"uast:Comment","Block":false,"Prefix":"","Suffix":"","Tab":"","Text":"comment here"}
   {"@pos":{"@type":"uast:Positions","end":{"@type":"uast:Position","col":20,"line":2,"offset":41},"start":{"@type":"uast:Position","col":1,"line":2,"offset":22}},"@type":"uast:Comment","Block":true,"Prefix":" ","Suffix":" ","Tab":"","Text":"block comment"}
   {"@pos":{"@type":"uast:Positions","end":{"@type":"uast:Position","col":22,"line":1,"offset":21},"start":{"@type":"uast:Position","col":8,"line":1,"offset":7}},"@type":"uast:Comment","Block":false,"Prefix":"","Suffix":"","Tab":"","Text":"comment here"}
   {"@pos":{"@type":"uast:Positions","end":{"@type":"uast:Position","col":20,"line":2,"offset":41},"start":{"@type":"uast:Position","col":1,"line":2,"offset":22}},"@type":"uast:Comment","Block":true,"Prefix":" ","Suffix":" ","Tab":"","Text":"block comment"}

   query.go

   package main
   
   import (
   	"encoding/json"
   	"fmt"
   
   	"gopkg.in/bblfsh/client-go.v3"
   	"gopkg.in/bblfsh/client-go.v3/tools"
   	"gopkg.in/bblfsh/sdk.v2/uast/nodes"
   )
   
   func main() {
   	client, err := bblfsh.NewClient("localhost:9432")
   	if err != nil {
   		panic(err)
   	}
   
   	res, _, err := client.NewParseRequest().ReadFile("js-driver-issue-36.js").UAST()
   	if err != nil {
   		panic(err)
   	}
   
   	query := "//*/uast:Comment"
   	it, _ := tools.Filter(res, query)
   	var nodeAr nodes.Array
   	for it.Next() {
   		nodeAr = append(nodeAr, it.Node().(nodes.Node))
   	}
   
   	data, err := json.MarshalIndent(nodeAr, "", "  ")
   	if err != nil {
   		panic(err)
   	}
   	fmt.Println(string(data))
   }

   $ go run ./client.go | jq -c '.[]' | wc -l
   

   4

   @dennwc do you know what is the reason? This seems like a 2 identical copies to me
8. dennwc commented on Nov 29, 2018

   

   dennwc

   on Nov 29, 2018

   Member

   More actions

   @bzz This is exactly the problem I talked about when reffering to DAGs. There are two pointers to the same comment node in the native AST.

   👍React with 👍1Reacted by Alex
9. bzz

   added

   bug

   on Nov 30, 2018