Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add region-restricted parsing and highlighting #202

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ubolonton
Copy link
Collaborator

@ubolonton ubolonton commented Jan 11, 2022

This will allow jupyter-repl to integrate with tree-sitter and tree-sitter-hl. See #78 for more context.

I initially tried to make tree-sitter parse only the narrowed region. However, since tree-sitter's incremental parsing requires precisely tracking all changes to the source code, there are some issues with that approach:

  • Emacs's change tracking mechanism are the hooks before-change-functions and after-change-functions, which can be run while narrowing is enabled for a different region than the intended region.
  • Tree-sitter needs to be able to track changes of narrowed regions, since they are code changes. This cannot be done automatically by tree-sitter itself.

The design of this new functionality is instead like followed:

  • tree-sitter still tracks all changes in the buffer.
  • When tree-sitter parses, it asks the major mode for the region to parse, by calling the function tree-sitter-get-parse-regions.
  • When tree-sitter-hl highlights the buffer, it does so only for the parsed region, while using the old region-fontification function for other regions.
  • When the major mode wants to change the region to be parsed (e.g. moving to the next input cell):
    • It calls tree-sitter-hl-dry-up-region to "persist" the highlighted region (now-frozen input cell).
    • It calls tree-sitter-pause, and then tree-sitter-resume afterwards, to start a new, non-incremental parse.

Tasks:

  • Parsing
  • Highlighting
  • jupyter-repl integration
  • inferior-python-mode integration
  • Tests

@orzechowskid
Copy link

looking forward to this PR landing in master. I was just attempting to add tree-sitter support to a CSS-in-JS mode I've been working on, and I think this will be of immense use once it's ready.

thanks!

@ubolonton
Copy link
Collaborator Author

looking forward to this PR landing in master. I was just attempting to add tree-sitter support to a CSS-in-JS mode I've been working on, and I think this will be of immense use once it's ready.

The design in this PR is intended for one-at-a-time parsing of code blocks, so I'm not sure it would help a CSS-in-JS mode (a multi-language use case). How would you use it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants