Figure out how to best call Sphinx from Python

[choldgraf]

Most people interact with Sphinx via the command line (in particular, sphinx-build). We'll want to interact with it from within Python as a part of the controller process.

It looks like the Sphinx building / initializing step is here:

https://github.com/sphinx-doc/sphinx/blob/3.x/sphinx/cmd/build.py#L269

However, there is some logic in there that we should investigate. In particular, there are some context managers that seem to be patching docutils before calling the Sphinx app, and also some parsing of arguments etc. We should make sure those aren't doing something important before just calling Sphinx(

Maybe @chrisjsewell has experience with this?

[chrisjsewell]

Yeh, for all the good things about Sphinx, documentation for using its API is not one!

We should make sure those aren't doing something important before just calling Sphinx(

I would definitely say that Sphinx should be called within these context manager's.

Related to this, myst_parser.docutils_renderer.SphinxRenderer now contains options to run them within a 'minimal' Sphinx environment, which uses these contexts managers. See its use at: https://myst-parser.readthedocs.io/en/latest/using/use_api.html#sphinx-renderer

[choldgraf]

Nice - I am learning a little bit about Click now so will try and mock-up a prototype. Between the myst_parser and myst_nb packages, this yaml metadata plugin, and this global toc plugin I think we have all we need for a (very alpha) CLI.

[choldgraf]

(as an aside click is so much more intuitive than argparse, omg)

[chrisjsewell]

(as an aside click is so much more intuitive than argparse, omg)

Exactly, its just a lot more Object-Orientated, and thus modular. Definitely look at how I have structured it injupyter-cache (in-particular in executablebooks/jupyter-cache#20); with separate cmnd_ modules and re-usable arguments/options.

[mmcky]

This should be importable -- but doesn't seem documented. I don't think many call sphinx-build through a python API. https://github.com/sphinx-doc/sphinx/blob/3.x/sphinx/cmd/make_mode.py

[choldgraf]

I'm working on this and should push a prototype to this repo soon!

[choldgraf]

OK, just merged #2 which gives us a working CLI! I will try and sharpen this up but it'd be great if others took a look at the general scope of what it's doing to see if this makes sense at all. Once we are OK with the structure we can write some tests and docs.

It's focusing on HTML outputs right now, but should be easy to add Latex/PDF outputs.

[mmcky]

ok thanks @choldgraf will take a look.

[choldgraf]

this now works - still improvements to be made I'm sure, but I think good enough to close this issue