Add markdown doc structure and gh-pages workflow #5942

rljacob · 2023-09-20T19:16:40Z

Add infrastructure to build markdown-based documentation located in each components "docs" directory into a single web-page of documentation. Makes use of mkdocs-monorepo-plugin.

Also add a skeleton of documentation subdirs and markdown files for EAM, ELM and MOSART.

Also add a github action to build and deploy the documentation on e3sm-project.github.io/e3sm whenever master is updated. And an action to build the docs on every PR.

[BFB]

Add docs skeleton for ELM and verify it renders.

Add more basic structure for ELM docs including tech and dev guid subdirs change users-guide to user-guide

Add tech and dev guide skeleton to ELM index

Remove out-of-date eam/doc dir which only had old ChangeLog and ReleaseNotes (no actual docs)

Add EAM documentation skeleton. subdirs, index.md and mkdocs.yml

Add a theme to start with and some useful mkdocs extensions

Add docs skeleton for MOSART and include it in the main docs

Change Intro to Introduction in ELM, EAM and main

Update repo in gh-pages from fork to main repo

rljacob · 2023-09-20T19:40:21Z

You can see what the documentation looks like at: https://rljacob.github.io/E3SM/

If you want to start writing docs, see https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/3924787306/Developing+Documentation

Need to add more python modules to the pip install command

bartgol

This looks good Rob! A couple of suggestions (up to you, we can do that later):

we can make the action conditional to any of the docs folder being modified (no need to re-gen the docs if no docs changed). That said, it's not an expensive thing (and runs on the cloud), so it's fine to update docs at every master update.
we could add the pr-preview action, so we get a preview of the new documentation inside every pr. See this snippet in the scream fork

Edit: I'm striking my first suggestion. It is possible that some packages do not touch their docs folder, and yet require a rebuild of the docs (SCREAM does need a rebuild, if the defaults XML file in the cime_config folder changes). So always rebuilding is the safest choice.

docs/index.md

mahf708

Unless you want to defer some improvements, let's incorporate a few things while here

.github/workflows/gh-pages.yml

mahf708 · 2023-09-20T21:05:50Z

.github/workflows/gh-pages.yml

Can we rename this file in anticipation for upstream/downstream merging? How about e3sm-gh-pages? or eam-elm-mosart-gh-pages?

I thought it had to be named gh-pages.yml. How does github know to run e3sm-gh-pages.yml to update the gh-pages branch?

No, it doesn't matter at all. The name can be anything. Github knows about gh-pages branch from the mkdocs gh-deploy command

https://www.mkdocs.org/user-guide/deploying-your-docs/#github-pages

Also, GitHub will run every .yml file in workflows individually. They can be linked, etc. and all sorts of interactions. We can also add more jobs inside one single yml file. A lot of things can happen in these actions 😸

mahf708 · 2023-09-20T21:10:03Z

Thanks @rljacob!!! 🥳

I left some comments on improving the GitHub actions file. Non-blockers.

we could add the pr-preview action, so we get a preview of the new documentation inside every pr. See this snippet in the scream fork

I strongly support this. I would advise that we want people to base the docs building from previewing the PRs rather than building locally. In other words, I think we should encourage people to write the docs on the fly so that it is as easy as it gets without having to think or set up stuff locally. To this end, two edits are important:

Testing in the PRs (see my suggestions above) with --strict --verbose
PR preview as Luca suggested

dismissed!

Modify workflow to operate on both PR creation and pushes to master. Update version of checkout action. Co-authored-by: Naser Mahfouz <[email protected]>

Rename gh-pages workflow file to e3sm-gh-pages.yml

Remove submodule checkout in pages workflow. Will be faster.

mahf708 · 2023-09-21T23:25:05Z

.github/workflows/e3sm-gh-pages.yml

+      - uses: actions/checkout@v4
+        with:
+          repository: e3sm-project/e3sm
+          ref: master


This is why the action is failing, we are asking it to pull master no matter what. We don't need that. I will suggest an alternative

You can prob use ${{github.ref}}

Actually, I think that may be the default, so no need to specify the ref.

Yeah, but there's really no need. We can just rely on the actions/checkout to do the right thing here

bartgol · 2023-09-21T23:25:50Z

.github/workflows/gh-pages.yml

  workflow_dispatch:

+concurrency:
+  # Prevent 2+ copies of this workflow from running concurrently
+  group: e3sm-docs-action


To get really fancy, you change this to

group: ${{ github.ref }}

so that different PRs can still build at the same time (they deploy to different preview pages anyways)

You can also add

cancel-in-progress: true

to the concurrency section, since we prob don't care about finishing to build docs if a commit is pushed...

That's a good point. We will have to adjust these settings especially once we add the previews to avoid users writing previews simultaneously. For now, likely okay as-is

rljacob · 2023-09-21T23:27:45Z

@mahf708 I accepted all your suggested changes.

mahf708

Just one minor thing, and everything should be good to go. We can trust the action to just fetch the right thing

.github/workflows/e3sm-gh-pages.yml

mahf708 · 2023-09-21T23:33:59Z

@rljacob if the --strict --verbose part becomes a burden, we can revert it. The idea is to make it fail even on warnings as mkdocs will proceed happily even if there are broken links, but it won't with --strict

Don't specify which repo/branch to check out. Action will do the right thing. Co-authored-by: Naser Mahfouz <[email protected]>

bartgol · 2023-09-22T03:40:49Z

.github/workflows/e3sm-gh-pages.yml

+      - uses: actions/checkout@v4
+        with:
+          show-progress: false
+          fetch-depth: 0 # Needed, or else gh-pages won't be fetched, and push rejected


At some point we may want to use another action for deployment, which will get rid of the need for fetch-depth: 0. E.g., in scream repo we use JamesIves/github-pages-deploy-action@v4, which plays nicely with rossjrw/pr-preview-action@v1, which we use to deploy preview of PR docs.

As it is now, if running on a PR, this action only builds the docs but does not show it anywhere. We can merge as is, but I would soon work on making a preview of docs available on the PR page.

Yes lets do that in a later PR.

Add infrastructure to build markdown-based documentation located in each components "docs" directory into a single web-page of documentation. Makes use of mkdocs-monorepo-plugin. Also add a skeleton of documentation subdirs and markdown files for EAM, ELM and MOSART. Also add a github action to build and deploy the documentation on e3sm-project.github.io/e3sm whenever master is updated. And an action to build the docs on every PR. [BFB]

rljacob · 2023-09-22T03:47:01Z

Was tested on the rljacob fork. Nothing to test with next.

mahf708 · 2023-09-22T03:50:29Z

http://e3sm-project.github.io/E3SM/ 🚀

rljacob added 12 commits August 27, 2023 00:15

Add an index.md file

5f22a71

Add basic mkdocs.yaml file

73cef25

Tell .gitignore to ignore the site dir

63edd77

Add gh-pages build and publish workflow

0213140

Add skeleton for ELM

e974b5c

Add docs skeleton for ELM and verify it renders.

Add more basic structure for ELM docs

f2aa4ea

Add more basic structure for ELM docs including tech and dev guid subdirs change users-guide to user-guide

Add tech and dev to ELM index

d945092

Add tech and dev guide skeleton to ELM index

Remove out-of-date eam/doc dir

65c6976

Remove out-of-date eam/doc dir which only had old ChangeLog and ReleaseNotes (no actual docs)

Add EAM documentation skeleton

5c23495

Add EAM documentation skeleton. subdirs, index.md and mkdocs.yml

Add themes and extensions

433c4f3

Add a theme to start with and some useful mkdocs extensions

Add docs skeleton for MOSART

24bdb5e

Add docs skeleton for MOSART and include it in the main docs

Change Intro to Introduction

716ebbe

Change Intro to Introduction in ELM, EAM and main

rljacob added the Documentation label Sep 20, 2023

rljacob requested review from bartgol, bishtgautam and mahf708 September 20, 2023 19:16

rljacob self-assigned this Sep 20, 2023

Update repo in gh-pages

1a37666

Update repo in gh-pages from fork to main repo

Add more installs to workflow.

2271d8a

Need to add more python modules to the pip install command

bartgol approved these changes Sep 20, 2023

View reviewed changes

docs/index.md Show resolved Hide resolved

mahf708 previously requested changes Sep 20, 2023

View reviewed changes

.github/workflows/gh-pages.yml Outdated Show resolved Hide resolved

mahf708 reviewed Sep 20, 2023

View reviewed changes

.github/workflows/gh-pages.yml Outdated Show resolved Hide resolved

mahf708 reviewed Sep 20, 2023

View reviewed changes

rljacob and others added 3 commits September 21, 2023 18:15

Apply suggestions from code review

19e571e

Modify workflow to operate on both PR creation and pushes to master. Update version of checkout action. Co-authored-by: Naser Mahfouz <[email protected]>

Rename gh-pages workflow file

97092c8

Rename gh-pages workflow file to e3sm-gh-pages.yml

Remove submodule checkout in pages workflow

a1e0c08

Remove submodule checkout in pages workflow. Will be faster.

mahf708 reviewed Sep 21, 2023

View reviewed changes

bartgol reviewed Sep 21, 2023

View reviewed changes

bartgol approved these changes Sep 21, 2023

View reviewed changes

mahf708 approved these changes Sep 21, 2023

View reviewed changes

.github/workflows/e3sm-gh-pages.yml Outdated Show resolved Hide resolved

Don't specify repo/branch to check out

7e3658e

Don't specify which repo/branch to check out. Action will do the right thing. Co-authored-by: Naser Mahfouz <[email protected]>

bartgol reviewed Sep 22, 2023

View reviewed changes

rljacob merged commit 4311d16 into E3SM-Project:master Sep 22, 2023
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add markdown doc structure and gh-pages workflow #5942

Add markdown doc structure and gh-pages workflow #5942

rljacob commented Sep 20, 2023 •

edited

Loading

rljacob commented Sep 20, 2023

bartgol left a comment •

edited

Loading

mahf708 left a comment

mahf708 Sep 20, 2023 •

edited

Loading

rljacob Sep 20, 2023 •

edited

Loading

mahf708 Sep 20, 2023

mahf708 Sep 20, 2023

mahf708 Sep 20, 2023

mahf708 commented Sep 20, 2023 •

edited

Loading

mahf708 Sep 21, 2023 •

edited

Loading

bartgol Sep 21, 2023

bartgol Sep 21, 2023

mahf708 Sep 21, 2023

bartgol Sep 21, 2023

bartgol Sep 21, 2023

mahf708 Sep 21, 2023

rljacob commented Sep 21, 2023

mahf708 left a comment

mahf708 commented Sep 21, 2023

bartgol Sep 22, 2023 •

edited

Loading

rljacob Sep 22, 2023

rljacob commented Sep 22, 2023

mahf708 commented Sep 22, 2023

Add markdown doc structure and gh-pages workflow #5942

Add markdown doc structure and gh-pages workflow #5942

Conversation

rljacob commented Sep 20, 2023 • edited Loading

rljacob commented Sep 20, 2023

bartgol left a comment • edited Loading

Choose a reason for hiding this comment

mahf708 left a comment

Choose a reason for hiding this comment

mahf708 Sep 20, 2023 • edited Loading

Choose a reason for hiding this comment

rljacob Sep 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mahf708 commented Sep 20, 2023 • edited Loading

mahf708 Sep 21, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rljacob commented Sep 21, 2023

mahf708 left a comment

Choose a reason for hiding this comment

mahf708 commented Sep 21, 2023

bartgol Sep 22, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rljacob commented Sep 22, 2023

mahf708 commented Sep 22, 2023

rljacob commented Sep 20, 2023 •

edited

Loading

bartgol left a comment •

edited

Loading

mahf708 Sep 20, 2023 •

edited

Loading

rljacob Sep 20, 2023 •

edited

Loading

mahf708 commented Sep 20, 2023 •

edited

Loading

mahf708 Sep 21, 2023 •

edited

Loading

bartgol Sep 22, 2023 •

edited

Loading