Service Guru

[usercont-release-bot]

Every Monday:

• (On stand-up) Check with the team whether deploying new changes from main branches is safe.
• Make sure stage deployment is OK, check:
  • Openshift cluster
  • Sentry
  • Dashboard

If all is fine and things are good to go:

• Move content from main branches to stable branches.
  • From a clone of the deployment repo run make move-stable command. Your local ssh keys have to be associated with your github user to be able to use this script.
    • If pushing the stable branch fails for a repo (e.g. because some changes had to be cherry-picked into production), force push the current state of main as stable to sync up the history
  • Go to the Actions tab (example of the respective repos) and make sure the images from the stable branches are built & pushed to Quay.io.
• Create a blog post about what's been changed in packit & packit-service main branches since their stable branch. Use title Packit [in] $MONTH $YEAR. Use scripts/move_stable.py github-query to get a link to all the PRs from the past week which are marked to have release notes. You can copy those for a start, and check with other team members to make sure nothing was left out. You can also use the blogpost template provided by [make move-stable]. (https://github.com/packit/deployment/blob/main/Makefile). A blog post should meet the following requirements:
  • This is meant to be read by general public - make it easy to read for everyone.
  • Focus on new features, notable bug fixes, documentation updates, UX improvements.
  • Don't talk about internal things (e.g. most ogr/specfile changes), refactoring, CI changes etc.
  • Write it in a way so that people with a little knowledge of the project can get a clue what is the change about:
    • NO: packit's behaviour in dealing with spec files was improved
    • YES: packit can now find a spec file inside a bottle of rum which is very handy if you're thirsty
  • Consider posting the most interesting changes on our Mastodon account (but be aware that the limit for a post is 500 characters, so do a shorter version or split the content to multiple posts and post it throughout the week)
• If there were any changes in the Kubernetes/Openshift objects or in the vault/Packit/! Changelog ! that have not been already applied to prod (ask the author of the change), do so with DEPLOYMENT=prod make deploy.
• If the deployment was reverted in the previous week, make sure to check the docs and follow the instructions for the manual steps that need to be run.
• If you're doing this AFTER new images have already been built (see previous point) you should run DEPLOYMENT=prod make import-images first (or after), to avoid possible Error: ImagePullBackOff.

Monday night & Tuesday morning:

• Newer images are automatically imported from Quay.io and pods are recreated from the new images.

On Tuesday:

• Make sure prod runs as expected - revert the deployment if all jobs fail or there is significant regression in the functionality.
• Merge/publish your blog post.
• Create a Mastodon post mentioning the top news.

[mfocko]

Warning
Don't forget to redo the repositories for moving stable branch or just clone the additional repo with git clone --recurse-submodules [email protected]:packit/production-monorepo.git