-
Notifications
You must be signed in to change notification settings - Fork 107
TaggingAndReleasing
Since WMCore is a common repository for several different software/services, we need to have different types of tags where each of them has a different purpose and a different validation. Before discussing them, it's important to mention that we have two types of validation in WMCore and those define the type of tags we have. The validations are:
- WMCore validation for CMSWEB services: it involves mostly our CherryPy backend applications, REST APIs, CouchDB/CouchApp-related applications, java script, WMSpecs and some Services modules (like DBS/Rucio/StompAMQ). In short, this is how we define when ReqMgr/WMStats/Global WorkQueue/MicroServices/ACDC are ready or not to go to production.
- WMCore validation for WMAgent: most of the WMCore core code - except for the REST layer - it also includes the agent components, integration with all the supported databases (CouchDB/MariaDB/Oracle), services modules and external service integration (MonIT, DBS, Rucio, CRIC, etc) and the runtime code (WMRuntime package). The WMAgent validation usually starts from a CMSWEB production-ready release, so a good fraction of the code has already been tested. In short, this is how we define whether a WMCore tag is ready to be used for WMAgent or not.
Starting in WMCore cycle 2.1.5, we adopted a new WMCore tagging convention to keep it consistent with PEP-440 and PyPi.
Regarding the type of tags we create in the WMCore repository, we have (where a, b, c, d are digits):
- a.b.crcX: these are release candidate tags which are created several times during a release cycle/month. As the name suggests, they are not meant to be used by external projects (T0/CRAB/Unified/etc), since they're not production-ready releases. These release candidate (or pre-release) tags are usually deployed in CMSWEB, as we keep adding new features/changes and validating them.
-
a.b.c: this is a final and CMSWEB/WMAgent production-ready release. This stable release has the same baseline code and software stack as the latest release candidate - the one that actually gets deployed in CMSWEB. In other words, if CMSWEB is running WMCore services on
2.1.5rc3, it means that the final and stable release in that cycle would be2.1.5; their baseline code is exactly the same. - a.b.c.d: these tags are only meant for WMAgent, where we create a new branch (branching off from the final release) such that we can isolate changes needed by WMAgents. This way, all WMAgent bug fixing has to be done against this special branch (named as a.b.c_wmagent), and a new patch tag is released. Note that the majority of the developments made against this wmagent branch are also ported to the master branch (thus, 2 pull requests are expected from the developers).
Special attention to a CMSWEB or CRAB hotfix will be required, and the tagging and branch setup hasn't been fully defined yet.
NOTE: The following convention has been deprecated in WMCore 2.1.4, after deciding to consolidate WMCore and PyPi tagging.
Regarding the type of tags we create in the WMCore repository, we have:
- a.b.c.preX: these are pre-release tags which are created several times during the month. As the name suggests, they are not meant to be used by other services, since they're not production ready releases. However, these pre-release tags are requested to be pushed to CMSWEB (testbed and production), so the last pre-release tag in the same cycle can be considered production-ready and so used by other services (e.g. 1.1.14.pre4)
- a.b.c: this is a final and CMSWEB production-ready release. The latest pre-release tag (the one that went into CMSWEB production) is the baseline for this final release and sometimes they point to the same commit. However, there might be cases with WMAgent side-only changes, that we still add a few PR on top of that before the final release is out.
- a.b.c.patchX: these tags are only meant for WMAgent, where we create a new branch (usually branching off the final release) such that we can isolate changes needed by WMAgents. This way, all WMAgent bug fixing has to be done against this special branch (named as a.b.c_wmagent), and a new patch tag is released. In addition to that, we usually have the same fix pushed to the master branch as well.
- a.b.c.crabX: these tags are only meant for CRAB TaskWorker, where we create a new branch (usually branching off the final release) such that we can isolate changes needed by TaskWorker. This way, all TaskWorker bug fixing has to be done against this special branch (named as a.b.c_crab), and a new crab patch tag is released. In addition to that, we usually have the same fix pushed to the master branch as well.
- a.b.c.cmswebX: only used for critical fix in CMSWEB production. So another branch is created from the latest pre-release (the one in CMSWEB production) and the required fix is pushed to that branch (and master, of course), then a new tag is release and asked to be put in CMSWEB production.
It's suggested to start this procedure in a fresh new terminal/repository to avoid any surprises. First, we clone the WMCore repository:
amaltaro@vocms055:/build/amaltaro $ git clone [email protected]:dmwm/WMCore.git && cd WMCore
Then we use the bin/buildrelease.sh script to cut a tag, it will basically:
- update the CHANGES file with the last changes for that specific branch
- commit this file and create a new tag
- push these changes to the remote repository.
You can see the helper function with:
amaltaro@vocms055:/build/amaltaro/WMCore $ sh bin/buildrelease.sh -h Tag & build a WMCore/WMAgent release buildrelease.sh [options] options: -r Remote repository to update, default to origin. -g Git branch to be tagged, default to master. -t Tag name to be created. -h Show this helper.
If you are cutting a tag against the master branch, your command line would be something like (don't forget updating your tag):
amaltaro@vocms055:/build/amaltaro/WMCore $ ./bin/buildrelease.sh -t 2.1.5rc2
the command above would create the 1.0.14.patch2 tag against the master branch, which is the default in case you do not provide any branch. During the process, it will open vi editor such that you can edit the CHANGES file. The CHANGES file usually contain two commits per change (a merge and the actual commit). After you are done with these changes, save and close the editor. It will then tag and push this commit + tag to the remote repository.
In case you're tagging a specific branch, your command line would look like (don't forget updating the branch and tag in the command line below):
amaltaro@vocms055:/build/amaltaro/WMCore $ ./bin/buildrelease.sh -g 2.1.6_wmagent -t 2.1.6.1
which would tag the 1.0.14_wmagent branch with a tag 1.0.14.patch2, CHANGES file would be updated reflecting all changes made in this branch.
When the pre-release has been finally validated and pushed to production, we might want to create a final/production tag pointing to the same commit that the latest pre-release tag does. For that, you need to find the commit hash that you want to tag and run a command like
git tag TAG_NAME COMMIT_HASH_ID
then assuming upstream is the dmwm remote official repository, you can push this tag only with the following command:
git push upstream master TAG_NAME
In case we hit a critical bug in CMSWEB production, we need to branch off from the latest tag that got deployed to CMSWEB production (usually the last pre-release, which is also tagged as a final release). In order to accomplish that, we need to find the hash of the commit which contains our tag. With that in hands, run the following command to create your new branch at that specific commit:
git branch BRANCH_NAME COMMIT_HASH_ID e.g. git branch 1.1.20_cmsweb 2ffe7d42a369c48ba89cfedded7c82f89008544c
Check everything is correct with git log command and then push it, but push it to the upstream instead of origin, thus:
git log -20 --pretty=oneline --decorate then git push upstream BRANCH_NAME