Add an option to the CLI to extract AST as JSON

[iabuhilal]

in the documentation, there is the following
"Print AST
If you want to print the AST of a source code, run the following command:
rust-code-analysis-cli -p /path/to/your/file/or/directory -d
The -d option prints the entire AST on the shell."

not sure what is the output format to parse it.

is there is a way to export the AST as a JSON file using rust-code-analysis-cli ?

[marco-c]

The output format is a custom textual format, e.g.:

`- {program:121} from (1, 1) to (3, 1) 
   |- {lexical_declaration:134} from (1, 1) to (1, 18) : const x = y => y; 
   |  |- {const:14} from (1, 1) to (1, 6) : const 
   |  |- {variable_declarator:135} from (1, 7) to (1, 17) : x = y => y 
   |  |  |- {identifier:1} from (1, 7) to (1, 8) : x 
   |  |  |- {=:39} from (1, 9) to (1, 10) : = 
   |  |  `- {arrow_function:180} from (1, 11) to (1, 17) : y => y 
   |  |     |- {identifier:1} from (1, 11) to (1, 12) : y 
   |  |     |- {=>:52} from (1, 13) to (1, 15) : => 
   |  |     `- {identifier:1} from (1, 16) to (1, 17) : y 
   |  `- {;:33} from (1, 17) to (1, 18) : ; 
   `- {lexical_declaration:134} from (2, 1) to (2, 20) : const z = (t) => t; 
      |- {const:14} from (2, 1) to (2, 6) : const 
      |- {variable_declarator:135} from (2, 7) to (2, 19) : z = (t) => t 
      |  |- {identifier:1} from (2, 7) to (2, 8) : z 
      |  |- {=:39} from (2, 9) to (2, 10) : = 
      |  `- {arrow_function:180} from (2, 11) to (2, 19) : (t) => t 
      |     |- {formal_parameters:206} from (2, 11) to (2, 14) : (t) 
      |     |  |- {(:19} from (2, 11) to (2, 12) : ( 
      |     |  |- {identifier:1} from (2, 12) to (2, 13) : t 
      |     |  `- {):20} from (2, 13) to (2, 14) : ) 
      |     |- {=>:52} from (2, 15) to (2, 17) : => 
      |     `- {identifier:1} from (2, 18) to (2, 19) : t 
      `- {;:33} from (2, 19) to (2, 20) : ; 

[marco-c]

@iabuhilal an alternative to get the AST in JSON format is to use the web service (run rust-code-analysis-web, send a request to the "ast" endpoint)

[iabuhilal]

Thank you

[iabuhilal]

hi @marco-c
I first tried to use rust-code-analysis-cli
cargo install -f rust-code-analysis-cli
and try to run the server
rust-code-analysis-cli --serve --port 9090
I got the following error.
error: Found argument '--serve' which wasn't expected, or isn't valid in this context

I tried to use rust-code-analysis-web.
so I installed rust-code-analysis-web and ran it using the following:
cargo install -f rust-code-analysis-web
rust-code-analysis-web --port 9090

then ran

nmap localhost
9090/tcp open  zeus-admin

I also ran

sudo netstat -tulpen
tcp        0      0 127.0.0.1:9090          0.0.0.0:*               LISTEN      1000       159234     33992/rust-code-ana 

I used postman
POST to
http://127.0.0.1:9090/ast
file_name as the absolute path to a source file

but I get nothing.

am I missing something? also is there documentation on how to use the REST API other than
https://mozilla.github.io/rust-code-analysis/commands/rest.html ?

[marco-c]

I first tried to use rust-code-analysis-cli
cargo install -f rust-code-analysis-cli
and try to run the server
rust-code-analysis-cli --serve --port 9090
I got the following error.
error: Found argument '--serve' which wasn't expected, or isn't valid in this context

This is a mistake in the docs. We moved the server to rust-code-analysis-web, as you noticed. I've filed #571 to update the docs.

[marco-c]

am I missing something? also is there documentation on how to use the REST API other than
https://mozilla.github.io/rust-code-analysis/commands/rest.html ?

There is an example usage from Python in https://github.com/mozilla/bugbug/blob/master/bugbug/rust_code_analysis_server.py.

[marco-c]

@iabuhilal example usage:

import requests

r = requests.post(URL_TO_RUST_CODE_ANALYSIS_SERVER, json={
    "id": "whatever",
    "file_name": YOUR_FILE_NAME,
    "code": YOUR_FILE_CONTENT,
    "comment": False,
    "span": True,
})

print(r.json())

[iabuhilal]

@marco-c thank you, I can get the AST as JSON now.

one last note, comment = false returns comments, i think it should return if comment = true

{

    "id": "python.py",

    "file_name": "python.py",

    "code": "var1=11\n    function fun1: var2 = 22 # comment test 123",

    "comment": false,

    "span": true
}

[Luni-4]

@marco-c

Can we close this one?

[marco-c]

@Luni-4 we still don't have a CLI command to get it, or do we?