GitHub action to publish docs to GH Pages

[mickmcgrath13]

For #1096

Things to note:

• This is a first test. I'll be unable to carry this further; I just wanted to help get it started
• The makefile updates the python version from 3.6 to 3.8 because that's whats provided in the GH Action runner
• The action cannot run until the action config file exists in the base branch of the repo (i.e. master)
• Once this is working, we'll still need to update DNS for docs.stackstorm.com, but until then, you should be able to go to stackstorm.github.io/st2docs. As a test, check out my forked repo: https://mickmcgrath13.github.io/st2docs/

[CLAassistant]

All committers have signed the CLA.

[arm4b]

@mickmcgrath13 Thanks! 🚀

Could you please sign a CLA thing?

[mickmcgrath13]

@armab I did the CLA thing. I also updated the PR to remove circleci as I don't think it'd be needed anymore. It looks like it still wants check from CircleCI, though.

Also, this does not yet fully implement everything that the CircleCI pipeline does (i.e. it looks like it does branch-based deploys).

As I mentioned before, I'll be unable to work on this further, though I'd be happy to recreate the PR with a branch directly in StackStorm/st2docs (rather than a PR from my personal account) if that would help others wrap this up.

[arm4b]

@mickmcgrath13 Thanks! 👍

I think it makes sense to keep CircleCI just for some time before switching the prod to GH Actions and verifying they're matching 1:1.

Left some pointers about the branching if you'd like to continue, or other folks want to help.

[arm4b]

Good start,

Per existing CircleCI config, there's also a need for additional routing logic depending on the branch:

st2docs/.circleci/config.yml

Lines 41 to 61 in 6c65852

	name: Deploy to docs.stackstorm.com
	command: |
	if [ "${CIRCLE_BRANCH}" = "master" ]; then
	aws s3 sync generated-site/st2docs/ \
	s3://${ST2DOCS_BUCKET}/latest
	else
	S3_OBJ=$(echo "${CIRCLE_BRANCH}" | sed 's/^v\(.*\)$/\1/')
	aws s3 sync generated-site/st2docs/ \
	s3://${ST2DOCS_BUCKET}/${S3_OBJ}
	fi
	- run:
	# Check the install scripts to see if this is the current GA version
	name: Check if current GA branch, in which case also deploy to main site
	command: |
	GA_VER=$(curl -sSL https://stackstorm.com/packages/install.sh|grep ^BRANCH=|sed 's/[^0-9.]*//g')
	if [ "${CIRCLE_BRANCH}" = "v${GA_VER}" ]; then
	aws s3 sync generated-site/st2docs/ \
	s3://${ST2DOCS_BUCKET}/
	else
	echo "Not current GA version"
	fi

From what I understand the logic looks this way:

• master branch build -> /latest/ st2docs dir
• v.X.Y.Z latest stable branch -> / st2docs dir
• other vX.Y.Z branch build -> vX.Y st2docs dir

[mickmcgrath13]

this might be possible with the destination_dir: subdir option:
https://github.com/peaceiris/actions-gh-pages#%EF%B8%8F-deploy-to-subdirectory-destination_dir

[mickmcgrath13]

so, the destination_dir: subdir option works well for /latest and /X.Y site directories. so 🎉

One difference, however, is that if we deploy to the root for the latest GA version like we do for S3, we'll lose all other subdirectories. For example, if we have /1.0 and /latest, and then we determine that the current version is also the latest GA release, deploying the site to / will wipe away /1.0 and /latest. I've confirmed this with testing.

To get around this, we could implement the action to deploy the latest GA release to a /ga subdirectory, and we can use URL routing to point docs.stackstorm.com to docs.stackstorm.com/ga. Thoughts?

[mickmcgrath13]

actually, we have a bigger issue:

When we push to github pages, a bot will run a pages-build-deployment pipeline. If we push to gh-pages again while an existing pages-build-deployment pipeline is running, github will cancel the previous deployment in favor of the subsequent one:

This means that we cannot push to gh-pages while there is an ongoing gh pages deployment, and it is non-trivial to "wait" until the gh pages deployment is finished. I suspect it's possible, though I've not looked into how it might be done (it'd probably involve the github api and some sort of polling logic).

This will also impact us when we want to run a deployment to existing version branches as we'd have to wait until each is finished before triggering the next one..

One thing that might help is that we could utilize the concurrency feature for our workflow. This will help ensure that our workflows don't run while another of our workflows are running, but we don't seem to have control over the pages-build-deployment pipeline.
https://stackoverflow.com/questions/70631168/github-actions-going-through-with-pages-build-deployment

Using concurrency might help because it's likely the gh pages will finish before any subsequent builds do, but it's a risk. It's possible that the gh pages deployment will take some time in which case our st2docs build might finish before and cancel the existing pages-build-deployment pipeline.
Concurrency was not working with my testing of multiple jobs in the same workflow, however.. (test here: https://github.com/StackStorm/st2docs/pull/1129/files#diff-7b112bd01a3c7000d73727a7e66cbfb9cf00a47702c94fda157e23f46e20db17R1)

However, it also means that we have to run two builds for the same thing...

One possible workaround is to use routing logic in Route53 or cloudfront to point the latest GA release instead of using a /ga subdirectory. I can think of 2 ways to do this:

1. have some logic in the github action pipeline that modifies entries in route53/cloudfront
2. manually update entries in route53/cloudfront as part of the GA release process

alternatively, it might be possible in the future to publish to GH Pages without the bot, but for now, that's our only option, I think:
https://github.community/t/github-pages-bot-added-to-our-repos-unannounced/218214/13

[mickmcgrath13]

example pipeline here: https://github.com/mickmcgrath13/st2docs/actions/runs/2531438591

[cognifloyd]

I don't think I understand the issue. The pages-build-deployment pipeline that github runs will be deploying whatever is on the gh-pages branch.

So, if we have two merges into the gh-pages branch at about the same time, and two builds start, then the build for the older commit should be canceled. But, because the newer commit should include everything from the previous commit, I don't think that's a problem. What else am I missing here?

[amanda11]

i think the issue was that if we are deploying say to version 3.7 of the docs, and then do a deploy to latest whilst its still running the deploy for v3.7. The v3.7 one will be cancelled, and we won't get the update to v3.7.
Though it isn't a frequent occurrance, that we push doc changes to the older branches, so as long as we can re-trigger the branched build I would have thought that would be ok.

[cognifloyd]

By the time pages-build-deployment runs for latest (cancelling the previous deployment), the updated 3.7 sources will already be committed to the gh-pages branch. pages-build-deployment redeploys everything in gh-pages afaik, so the next pages-build-deployment run will include both 3.7 and latest in the deployed changes.

[mickmcgrath13]

from my previous notes:

One difference, however, is that if we deploy to the root for the latest GA version like we do for S3, we'll lose all other subdirectories. For example, if we have /1.0 and /latest, and then we determine that the current version is also the latest GA release, deploying the site to / will wipe away /1.0 and /latest. I've confirmed this with testing.

One thing that can possibly work around these issues is to always only build gh pages in a pipeline based on the base branch. the base branch gh-pages pipeline will also clone other versions/subdirectories and build those.

then we have pipelines for version-specific branches that just trigger the base gh-pages pipeline to rebuild everything

an all-in-one pipeline at the base branch that builds root and subdirectories would also mitigate special routing requirements (i.e. we just build all the subdirectories we need)

just some thoughts..