Integrating the Nimrod compiler service in IDEs

This document provides a short description about how the Nimrod compiler service is integrated in nimrod.vim.

The nimrod compiler service in a long-running process responsible for continuos compilation of a single project and providing many modern IDE capabilities such as completion suggestions, jumping to definitions, locating references (and so on) in a quick and responsive way.

Let's see what are the steps taken by nimrod.vim when interacting with the nimrod compiler. Follow the links to see the relevant code.

1. When vim opens a new nimrod file, the first thing nimrod.vim will do is to execute the command:

nimrod dump --verbosity:0 --dump.format:json <current_file>

This will produce a large response such as:

{
   "version":"0.9.1",
   "project_path":"/Users/zahary/Projects/grip-suite/grip-lang/compiler/nimrod.nim",
   "defined_symbols":[
      "nimmixin",
      "emulatedthreadvars",
      "niminheritable",
      "gcmarkandsweep",
      "clang",
      "nimbabel",
      "nimrod",
      "nimeffects",
      "booting",
      "nimhygiene",
      "unix",
      "macintosh",
      "x8664",
      "cpu64",
      "MacOSX",
      "littleendian",
      "posix",
      "amd64"
   ],
   "lib_paths":[
      "/Users/zahary/Projects/grip-suite/grip-lang/compiler",
      "/Users/zahary/Projects/nim/lib/packages/docutils",
      "/Users/zahary/Projects/grip-suite/grip-lang/compiler/..",
      "/Users/zahary/Projects/grip-suite/grip-lang/compiler/llvm",
      "/Users/zahary/Projects/nim/lib/pure/unidecode",
      "/Users/zahary/Projects/nim/lib/js",
      "/Users/zahary/Projects/nim/lib/posix",
      "/Users/zahary/Projects/nim/lib/windows",
      "/Users/zahary/Projects/nim/lib/wrappers/zip",
      "/Users/zahary/Projects/nim/lib/wrappers/x11",
      "/Users/zahary/Projects/nim/lib/wrappers/sdl",
      "/Users/zahary/Projects/nim/lib/wrappers/readline",
      "/Users/zahary/Projects/nim/lib/wrappers/pcre",
      "/Users/zahary/Projects/nim/lib/wrappers/opengl",
      "/Users/zahary/Projects/nim/lib/wrappers/lua",
      "/Users/zahary/Projects/nim/lib/wrappers/gtk",
      "/Users/zahary/Projects/nim/lib/wrappers/cairo",
      "/Users/zahary/Projects/nim/lib/wrappers",
      "/Users/zahary/Projects/nim/lib/impure",
      "/Users/zahary/Projects/nim/lib/pure/collections",
      "/Users/zahary/Projects/nim/lib/pure",
      "/Users/zahary/Projects/nim/lib/core",
      "/Users/zahary/Projects/nim/lib"
   ]
}

The most important bits here are the project_path, locating the main module of the project, which the currently edited source is part of and the lib_paths array, specifying the module import paths derived from the project config file. In order to locate the main module of the project, currently nimrod.vim requires that the user provides a nimrod.cfg, featuring the mainmodule: directive.

For each unique project path, nimrod.vim will start a separate instance of the compiler service using the following command:

nimrod serve --server.type:stdin <project_path>

From now on, we'll use the standard/input and output streams to communicate with the long-running compiler process. The communication protocol is line-oriented. Nimrod will parse the text lines sent from us in the same way it parses command-line arguments. (e.g. sending c will trigger compilation, sending idetools --track:<current_file,current_line,current_columnn> --def will locate the definition under the cursor and so on). The response from the service process will be exactly the same as when executing the respective stand-alone command. After each response, an empty line will be outputted serving as a separator.

Parsing the responses of the various idetools commands is beyond the scope of this document, but it suffices to say that the lines of interest will contain tab-separated data and the first column will indicate the type of the returned datum. Here is an example for parsing the code completion suggestions provided by nimrod.