README.md

Source code overview

Since Neovim has inherited most code from Vim, some information in its README still applies.

This document aims to give a high level overview of how Neovim works internally, focusing on parts that are different from Vim. Currently this is still a work in progress, especially because I have avoided adding too many details about parts that are constantly changing. As the code becomes more organized and stable, this document will be updated to reflect the changes.

If you are looking for module-specific details, it is best to read the source code. Some files are extensively commented at the top (e.g. terminal.c, screen.c).

Top-level program loops

First let's understand what a Vim-like program does by analyzing the workflow of a typical editing session:

1. Vim dispays the welcome screen
2. User types: :
3. Vim enters command-line mode
4. User types: edit README.txt<CR>
5. Vim opens the file and returns to normal mode
6. User types: G
7. Vim navigates to the end of the file
8. User types: 5
9. Vim enters count-pending mode
10. User types: d
11. Vim enters operator-pending mode
12. User types: w
13. Vim deletes 5 words
14. User types: g
15. Vim enters the "g command mode"
16. User types: g
17. Vim goes to the beginning of the file
18. User types: i
19. Vim enters insert mode
20. User types: word<ESC>
21. Vim inserts "word" at the beginning and returns to normal mode

Note that we have split user actions into sequences of inputs that change the state of the editor. While there's no documentation about a "g command mode" (step 16), internally it is implemented similarly to "operator-pending mode".

From this we can see that Vim has the behavior of a input-driven state machine (more specifically, a pushdown automaton since it requires a stack for transitioning back from states). Assuming each state has a callback responsible for handling keys, this pseudocode (a python-like language) shows a good representation of the main program loop:

def state_enter(state_callback, data):
  do
    key = readkey()                 # read a key from the user
  while state_callback(data, key)   # invoke the callback for the current state

That is, each state is entered by calling state_enter and passing a state-specific callback and data. Here is a high-level pseudocode for a program that implements something like the workflow described above:

def main()
  state_enter(normal_state, {}):

def normal_state(data, key):
  if key == ':':
    state_enter(command_line_state, {})
  elif key == 'i':
    state_enter(insert_state, {})
  elif key == 'd':
    state_enter(delete_operator_state, {})
  elif key == 'g':
    state_enter(g_command_state, {})
  elif is_number(key):
    state_enter(get_operator_count_state, {'count': key})
  elif key == 'G'
    jump_to_eof()
  return true

def command_line_state(data, key):
  if key == '<cr>':
    if data['input']:
      execute_ex_command(data['input'])
    return false
  elif key == '<esc>'
    return false

  if not data['input']:
    data['input'] = ''

  data['input'] += key
  return true

def delete_operator_state(data, key):
  count = data['count'] or 1
  if key == 'w':
    delete_word(count)
  elif key == '$':
    delete_to_eol(count)
  return false  # return to normal mode

def g_command_state(data, key):
  if key == 'g':
    go_top()
  elif key == 'v':
    reselect()
  return false  # return to normal mode

def get_operator_count_state(data, key):
  if is_number(key):
    data['count'] += key
    return true
  unshift_key(key)  # return key to the input buffer
  state_enter(delete_operator_state, data)
  return false

def insert_state(data, key):
  if key == '<esc>':
    return false  # exit insert mode
  self_insert(key)
  return true

While the actual code is much more complicated, the above gives an idea of how Neovim is organized internally. Some states like the g_command_state or get_operator_count_state do not have a dedicated state_enter callback, but are implicitly embedded into other states (this will change later as we continue the refactoring effort). To start reading the actual code, here's the recommended order:

1. state_enter() function (state.c). This is the actual program loop, note that a VimState structure is used, which contains function pointers for the callback and state data.
2. main() function (main.c). After all startup, normal_enter is called at the end of function to enter normal mode.
3. normal_enter() function (normal.c) is a small wrapper for setting up the NormalState structure and calling state_enter.
4. normal_check() function (normal.c) is called before each iteration of normal mode.
5. normal_execute() function (normal.c) is called when a key is read in normal mode.

The basic structure described for normal mode in 3, 4 and 5 is used for other modes managed by the state_enter loop:

• command-line mode: command_line_{enter,check,execute}()(ex_getln.c)
• insert mode: insert_{enter,check,execute}()(edit.c)
• terminal mode: terminal_{enter,execute}()(terminal.c)

Async event support

One of the features Neovim added is the support for handling arbitrary asynchronous events, which can include:

• msgpack-rpc requests
• job control callbacks
• timers (not implemented yet but the support code is already there)

Neovim implements this functionality by entering another event loop while waiting for characters, so instead of:

def state_enter(state_callback, data):
  do
    key = readkey()                 # read a key from the user
  while state_callback(data, key)   # invoke the callback for the current state

Neovim program loop is more like:

def state_enter(state_callback, data):
  do
    event = read_next_event()       # read an event from the operating system
  while state_callback(data, event) # invoke the callback for the current state

where event is something the operating system delivers to us, including (but not limited to) user input. The read_next_event() part is internally implemented by libuv, the platform layer used by Neovim.

Since Neovim inherited its code from Vim, the states are not prepared to receive "arbitrary events", so we use a special key to represent those (When a state receives an "arbitrary event", it normally doesn't do anything other update the screen).