Convert wiki to Sphinx/ReadTheDocs (REP-004 part 1)

[JeanChristopheMorinPerso]

Closes #1292 (at least partially). Also relates to #1277.

This PR converts our GitHub wiki to Sphinx. It will be automatically built by ReadTheDocs and each PR will be built and will have a preview (see the actions on the PR for the link of the preview).

Preview link: https://rez--1522.org.readthedocs.build/en/1522/

Note that this is almost a dumb conversion, in the sense that there isn't new documentation. Changes are:

• The Python API docs and the docs for rez are now all in one place.
• All environment variables are now fully documented (even for the settings), except for the *_JSON variables.
• I added a huge amount of cross references between all the pages. Settings can now be easily references (and they are), environment variables can also be referenced.
• Settings now have proper type hints (where possible) in the docs (not in the code).
• The rex functions are also typed hinted in the docs (not the code).
• The generation process for the settings docs is fully integrated in Sphinx, which means that it's now a single step process step process :)
• The documentation is now fully searchable (thanks to ReadTheDocs).
• The order and grouping in the table of content was re-organized to help finding what you are looking for and also helping our new users discover pages more "naturally" (it's probably the wrong term, but it's the only one that comes to mind).

There is a few pages that were left out:

• The glossary, because I felt like it was redundant.
• The rez-pip docs, because the new rez-pip2 is coming soon. I converted it.
• the CLI docs, because I felt they weren't that useful. To be useful, it would require to use sphinx directives for proper cross-referencing in the docs. This would need some custom code in the conf.py file. The CLI docs are now automatically generated.

I could probably add the glossary and the rez-pip docs if others feel like they are important enough.

As it is right now, I think we could try it and see how it goes. I personally think that's it's already a good improvement over what we have.

TODO:

• Determine if we want to remove the wiki directory now or in a seperate PR.
• Remove wiki workflow if we remove the wiki directory.
• If we remove the wiki directory, disable the wiki entirely on the repo after the merge.

[davidlatwe]

The new doc looks awesome! And really great that it comes with light/dark mode switch.

I have a suggestion for the final polishing work. My system uses dark mode, and this is what I saw in first page:

The black outline and arrows are not really easy to see.

I am thinking each image we used in the doc should have 2 versions, one for light mode and one for the dark. Or just one, with a solid backgound that fits in both modes.

I can help with those images if you need, just let me know. 👍🏼

[JeanChristopheMorinPerso]

Hi @davidlatwe ! Thanks for taking a look!

Regarding dark mode problems with the images, I indeed saw that but deciced that it could be improved in a subsequent PR. Furo supports dark/light figures (https://pradyunsg.me/furo/reference/images/#different-images-for-dark-light-mode), so it would be possible to have a dark and light version of the images.

If you feel like contributing new diagrams in dark and light mode, I can definitely add the to the PR though!

[JeanChristopheMorinPerso]

@davidlatwe I made an attempt at changing the images/diagrams using CSS. They now render properly when viewing in dark mode. It's not perfect, but it avoids having to deal with duplicating the diagrams for each theme.

Let me know what you think.

[davidlatwe]

Thanks @JeanChristopheMorinPerso !

I noticed this, the images were not inverted when the theme is set to respect system preference. (My system has set to dark)

But I suppose that should be fixed by adding @media (prefers-color-scheme: dark) to the selector. (didn't build the doc on my end 😅 )

@media (prefers-color-scheme: dark)
body[data-theme=dark] .rez-diagram {
    ...
}

Anyway, I like this solution 👍🏼

[JeanChristopheMorinPerso]

Alright, I made the change @davidlatwe. I tested all modes (manual overrides, OS overrides, etc) and everything seems to be working now.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: JeanChristopheMorinPerso / name: Jean-Christophe Morin (0519834, 9c2c87d, 4810557, 1eaf4a9, 7699af3, 586e20a, 1e62245, b223972, eb864a6, 84a3b34, 4cfedbb, e55ee33, 2f9ca98, 683e16f, 904fbad, 2af4b9c, 401779e, ddf9d49, 485a57c, 116c47d, 5899d7a, 3b11862, 17510be, 1a46faa, 5dbd3ab, e40c02d, ee8f4ed, 3321226, 5175fb2)
• ✅ login: davidlatwe / name: David Lai (3b2848a)

[JeanChristopheMorinPerso]

@davidlatwe I integrated your commit and the CLA and DCO checks are failing. I see two ways to fix this:

1. One is for you to authorize me to add the appropriate "Signed-off-by: Author Name [email protected]" bit in your commit and you'll have to sign the CLA.
2. I can take ownership of the commit and put you as co-author (So I would add "Co-authored-by: David Lai ".

Do you have a preference?

[davidlatwe]

This is a reply for the Slack chat https://academysoftwarefdn.slack.com/archives/C0321B828FM/p1693697597478139

Feedback (My 2 cents)

Is it readable?

You bet. 👍🏼

Are you able to search and find what you need (for example, try searching for settings, environment variables, etc)

Yes, but not perfect.🤔 When I was trying to search include, the decorator for sharing code across packages, the search results were not really ideal.

Screenshot

Same for early.

Screenshot

The top 3 search results were Python API entries, which I think they are too advance for most cases. Secondly, when you click one of the results, the page doesn't scroll to the expected paragraph, it only brings you to that page.

Is the table of content of the left side clear and intuitive (by that I mean the grouping and ordering, etc)?

There should have a "Resolving Packages" page between "Building Packages" and "Context" page. Perhaps we should breakdown "Basic Concepts" into smaller chunks?

What do you think of configuration settings, is it easy to read and understand?

It's good that the doc also mentioned the equivalent environment variable, but would be better if we can have those env var displayed with different color, e.g. gray.

Same for package attributes and package commands objects.

These are awesome! So much better with typing and example.

[davidlatwe]

@davidlatwe I integrated your commit and the CLA and DCO checks are failing. I see two ways to fix this:

1. One is for you to authorize me to add the appropriate "Signed-off-by: Author Name [email protected]" bit in your commit and you'll have to sign the CLA.
2. I can take ownership of the commit and put you as co-author (So I would add "Co-authored-by: David Lai ".

Do you have a preference?

Didn't notice that! I just clicked the link from that EasyCLA and signed it, so all should be good? Both options work for me but since I have CLA signed, option 1 would be better I guess? 🤔

[JeanChristopheMorinPerso]

Alright, DCO is fixed.

[JeanChristopheMorinPerso]

Thanks for the feedback (and the help too :))!

Are you able to search and find what you need (for example, try searching for settings, environment variables, etc)

Yes, but not perfect.🤔 When I was trying to search include, the decorator for sharing code across packages, the search results were not really ideal.

Yeah, I saw that too. By default RTD indexes everything in ElasticSearch and they hijack the search bar to read from ElasticSearch (see https://docs.readthedocs.io/en/stable/server-side-search/index.html). But for whatever reason it doesn't seem to be the case right now. I'm guessing it's because we haven't released the docs to the default version. They probably don't index all commits coming for PRs (because it would obviously be cost and resource prohibitive). I'm crossing my fingers that it'll improve once we merge this in master.

The top 3 search results were Python API entries, which I think they are too advance for most cases. Secondly, when you click one of the results, the page doesn't scroll to the expected paragraph, it only brings you to that page.

Like I said, I'm guessing this is related to the fact that the PR isn't indexed in ElasticSearch. As for the Python API stuff, we could de-prioritize the Python API using https://docs.readthedocs.io/en/stable/config-file/v2.html#search-ranking. In fact I tried it and it wasn't having any effect (again probably because they don't index PR builds).

There should have a "Resolving Packages" page between "Building Packages" and "Context" page. Perhaps we should breakdown "Basic Concepts" into smaller chunks?

Indeed. The current "Basic Concepts" page is annoying me (and I never liked it). It mixes concepts with implementation details, and other stuff like tutorials (resolving an environment). I was hoping that we could review all this in a second phase though. I interpret your reply as "the ordering and grouping is fine but we should split some pages and re-group things", is that correct?

What do you think of configuration settings, is it easy to read and understand?

It's good that the doc also mentioned the equivalent environment variable, but would be better if we can have those env var displayed with different color, e.g. gray.

Thanks! It's been annoying me too. I actually spent a day trying to improve this without involving CSS and couldn't find a solution that satisfied me. I kind of know what I want, but I'm not sure how simple it would be to implement and would require quite a bit of custom Sphinx code (I think), and I don't even know if what I want is doable. And I already have 350 lines of code in my custom sphinx extension... I think I'll open an issue on the Sphinx repo to ask for help.

[instinct-vfx]

This is a huge step forward and my vote is to get this merged asap and make it available to a wider audience. We can then start iterating and further improving! Thanks a lot @JeanChristopheMorinPerso !