Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] TeXpresso integration #2975

Draft
wants to merge 25 commits into
base: master
Choose a base branch
from
Draft

Conversation

tsung-ju
Copy link
Contributor

@tsung-ju tsung-ju commented Jul 6, 2024

TeXpresso (by @let-def) is a program that provides a "live rendering" experience when editing LaTeX documents. This PR aims to integrate TeXpresso into VimTeX as a compiler.

Unlike other latex compilers, TeXpresso combines a compiler based on tectonic and a pdf viewer based on MuPDF, and communicates file changes, quickfix items, and synctex through stdio. In this PR, these are implemented by opening a continuous compiler and changing the input mode to 'pipe' in autoload/vimtex/compiler/_template.vim.

Most main features of TeXpresso including live editing, synctex, quickfix, theme color syncing are working. However, the code still have some rough edges (hence marked as draft). Please feel free to try the code out. Any suggestion would be really appreciated.

@lervag
Copy link
Owner

lervag commented Jul 7, 2024

First: I'm very happy to see such a contribution! And I look forward to reviewing the code. But I'm on vacation right now, and I can't promise that I'll be able to review this immediately. I'll look at it as soon as I get the time!

TeXpresso (by @let-def) is a program that provides a "live rendering" experience when editing LaTeX documents. This PR aims to integrate TeXpresso into VimTeX as a compiler.

I was not aware of TeXpresso. It looks very interesting. It would be nice to hear more about how it works - how is it possible to be "so fast" wrt. compile times? Does it work well even for large and complex documents?

@let-def
Copy link

let-def commented Jul 7, 2024

Hi @tsung-ju & @lervag, I am glad to see this contribution. I don't have much free time these days, so I will only follow this from a distance.

I was not aware of TeXpresso. It looks very interesting. It would be nice to hear more about how it works - how is it possible to be "so fast" wrt. compile times?

I can explain a bit how TeXpresso works. The two main pieces are the engine, a fork of XeTeX, and the driver, which plays two roles: feeding the engine and rendering the output.

Let's look at the engine first. It's a fork of XeTeX, currently the one found in the Tectonic distribution, but i would like to synchronize with TeXlive in the future. The main purpose of the fork is to redirect the I/O, so that the data can come from a custom source, e.g. an editor mode, and not necessarily from the file system. This is also used to keep track of the contents observed by the TeX process, in order to recompute things only when they change.

The driver use this to (1) feed the engine with data coming either from the file system, a cache of the TeX packages (this directly comes from Tectonic at the moment) or the buffers of the editor, and (2) take snapshots from time to time by forking the engine (fork as in unix's fork(2)). When contents change in the editor, the TeX process is resumed from the fork closest to the change. On average, only the visible page has to be recomputed.

The rendering pipeline is entirely custom and is built on top of the DVI/XDV format. Mupdf is not really used as a PDF renderer but rather as a framework for managing fonts and drawing to a canvas. The PDF renderer is used only to implement \includegraphics with pdf files.

Does it work well even for large and complex documents?

I think it works well with large and complex documents, as long as they are XeTeX compatible.
I have tested it with books full of graphics (e.g. https://github.com/fabiensanglard/gebbdoom) and large math documents (close to a thousand pages).
There are some limitations though. First, TeXpresso does only a single pass: to have a table of contents or a bibliography, one will need to run LaTeX or BiBTeX passes separately. TeXpresso can use the aux files produced by an external build system. It checks for update to these files when receiving SIGUSR1, so appending killall -SIGUSR1 texpresso to a build script is usually sufficient for a seamless integration.
Second, the project is quite experimental. I did the minimum to provide a decent experience for my use cases, but it is far from being rock solid. I experience a crash every once in a while, but it can vary a lot depending on the features used (I debugged the things I use the most, the rest is alpha quality :)). We also found some bugs in neovim, e.g. neovim/neovim#24931, so the mode can desynchronize from time to time. :e! is sufficient to force a resynchronization, so I did not bother much with that.

@lervag
Copy link
Owner

lervag commented Jul 11, 2024

Hi @tsung-ju & @lervag, I am glad to see this contribution. I don't have much free time these days, so I will only follow this from a distance.

👍

I was not aware of TeXpresso. It looks very interesting. It would be nice to hear more about how it works - how is it possible to be "so fast" wrt. compile times?

I can explain a bit how TeXpresso works. …

Thanks for the info! Sounds like this may be a very interesting idea and a useful addition to VimTeX!

@@ -304,7 +304,7 @@ endfunction
let s:compiler_jobs = {}
function! s:compiler_jobs.exec(cmd) abort dict " {{{1
let l:options = {
\ 'in_io': 'null',
\ 'in_io': 'pipe',
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious, is it safe to use pipe here also for the existing engines?


if has('nvim')
let s:nvim_attach = luaeval('
\ function()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would prefer to define the function in a .lua file under /lua/vimtex/.... It would make for a slightly cleaner code here and it becomes easier to lint and maintain the lua code. Further, I would also prefer to add type hints to the function.

" }}}1

function! s:compiler.__init() abort dict " {{{1
let self.start = function('s:compiler_start', [self.start])
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would look slightly cleaner to define the .start and .stop methods more "explicitly" with e.g. function! s:compiler.start(...) abort dict in line 66.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, sorry, I now noticed that this way allows to retain the "super" method. That's neat!

@lervag
Copy link
Owner

lervag commented Jul 11, 2024

I've looked at the code and I think in general it seems very good. Thanks for following my conventions!

I've not tested it myself yet. I look forward to testing it.

One major thing that is missing for now is documentation. I think a nice starting point is to add something similar to the docs for _latexrun.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants