From 003959d0dcbc83bbeccd9dfacb464277c15a34c8 Mon Sep 17 00:00:00 2001 From: Gautam Bisht Date: Mon, 9 Oct 2023 20:50:46 -0700 Subject: [PATCH] Adds notes on running e3sm land developer testsuite --- components/elm/docs/dev-guide/index.md | 3 + components/elm/docs/dev-guide/testing.md | 183 +++++++++++++++++++++++ 2 files changed, 186 insertions(+) create mode 100644 components/elm/docs/dev-guide/testing.md diff --git a/components/elm/docs/dev-guide/index.md b/components/elm/docs/dev-guide/index.md index 3188951d887b..95366c925a23 100644 --- a/components/elm/docs/dev-guide/index.md +++ b/components/elm/docs/dev-guide/index.md @@ -1 +1,4 @@ start of the ELM Developer's Guide + +### +* [Testing development branches](testing.md): Generating baselines and comparing new development against baselines diff --git a/components/elm/docs/dev-guide/testing.md b/components/elm/docs/dev-guide/testing.md new file mode 100644 index 000000000000..8beacacc28ce --- /dev/null +++ b/components/elm/docs/dev-guide/testing.md @@ -0,0 +1,183 @@ +When a developer is ready to issue a PR to integrate their new developments into E3SM's `master` branch, they need to test their code development to ensure the new development does not introduce unexpected bugs. A land developer needs to atleast run the `e3sm_land_developer` test suite that involves following two steps: + +- **Step-1**: Generate the baselines using the hash on `master` that was the starting point of the + development branch, and +- **Step-2**: Switch to the development branch and re-run the test suite to compare against + the baselines generated in Step-1. + +For the purpose of these document, let's assume the developer branch is `bishtgautam/lnd/emi-example`. + +## Step-1: Generate Baselines + +Find the starting hash on `master` by switching to the developer branch and looking at the Git +graph. + +``` +cd + +# Switch to your branch +git checkout bishtgautam/lnd/emi-example + +# Let's look at the graph +git log --oneline --decorate --graph +* 9a2cd8459 (HEAD -> bishtgautam/lnd/emi-example, origin/bishtgautam/lnd/emi-example) Updates the Stub EM code +* d11fb8c52 Adds a readme and makefile for EMI Demo code +* 15a0e4b8d Minor update to the EMI demo code +* 63e2f6dcd Minor fix to ELM configure script +* 551fc79cc Updates ELM stub and demo code to use CNCarbonStateType +* 8c9d41f87 Adds code to exchange CNCarbonStateType +* 0000ded51 Minor fix +* 2baa4ebec Adds fixes to cmake +* f77ae2cea Rename few cmake files +* b48b66dd6 Adds a demo for a Stub EM +* fa94aea0d Fixes length of variable to store name of EMI data +* cca0e0be1 Adds capability to print all EMI Data in a list +* beb1fe9ee Adds .gitignore files for EMI +* 301acfe7a Renames ExternalModelInterfaceDataMod.F90 +* f0eb5b40b Renames ExternalModelInterfaceDataDimensionMod.F90 +* 076100882 Restructures ELM's EMI directory +* cfb7fc2b2 Merge branch 'qzhu-lbl/lnd/ch4_inundation_bugfix' (PR #2814) +|\ +| * ec99b1cb4 (origin/qzhu-lbl/lnd/ch4_inundation_bugfix) - fix ch4 inundatation parameter read in bug +* | c59c98116 Merge branch 'darincomeau/mpaso/enable_eddystats' (PR #2821) +|\ \ +| * | c21bb46a4 Turn on eddy stats for oEC60to30v3 and oEC60to30v3wLI ocn grids by default +* | | a2dd2fce5 Merge pull request #2811 from E3SM-Project/jayeshkrishna/pio2_cime_changes +``` + +The above Git graph tells us that `bishtgautam/lnd/emi-example` started with the `cfb7fc2b2` hash on the `master`. So, we will generate the baselines using `cfb7fc2b2`. Checkout `cfb7fc2b2` and update the submodels via: + +``` +git checkout cfb7fc2b2 +Note: checking out 'cfb7fc2b2'. + +You are in 'detached HEAD' state. You can look around, make experimental +changes and commit them, and you can discard any commits you make in this +state without impacting any branches by performing another checkout. + +If you want to create a new branch to retain commits you create, you may +do so (now or later) by using -b with the checkout command again. Example: + + git checkout -b + +HEAD is now at cfb7fc2b2... Merge branch 'qzhu-lbl/lnd/ch4_inundation_bugfix' (PR #2814) + +git submodule update --init +``` + +Now we will use `cime/scripts/create_test` to generate baseline for the `e3sm_land_developer` +test suite. + +``` +cd cime/scripts +``` + +A few things that you need to decide at this stage include the directory location where the baselines will be saved, the name and ID for the baselines, the project allocation that you will use for running the simulation, if you would like to recieve email notirications about tests, etc. + +``` +# Define the directory to hold the baseline +export MY_BASELINE_DIR=/global/cscratch1/sd/gbisht/e3sm_baselines + +# Do you want to use a name for the baseline? +# One choice could be the git hash that is being used to generate the baselines. +export BASELINE_NAME=cfb7fc2b2 + +# Let's set TEST_ID to be same as BASELINE_NAME +export TEST_ID=${BASELINE_NAME} + +# If you are a member of E3SM, you could use 'e3sm' project allocation +export PROJECT=e3sm + +# Set your email +export MAIL_USER= + +``` + +Use `./create_test --help` to get a complete list of arguments. Below are some additional +useful arguments for `./create_test`: + +``` +# Other arguments +# -v : Verbose option +# -g : Generate the baseline +# -q : If you want to use a particular job queue (e.g. 'debug' queue on NERSC) +# --walltime : Specify the wall time for jobs (e.g. 30 min is max allowable for 'debug' queue on NERSC) +# --mail-user: If you want to receive emails about your jobs +# --mail-type: When to receive emails. Options are: never, all, begin, end, fail. +# -j : Number of parallel jobs +``` + +Now run the `e3sm_land_developer` + +``` +./create_test e3sm_land_developer \ +--baseline-root ${MY_BASELINE_DIR} \ +-b ${BASELINE_NAME} \ +-t ${TEST_ID} \ +-q regular \ +-p ${PROJECT} \ +--walltime 00:30:00 \ +--mail-user $MAIL_USER \ +--mail-type all \ +-g \ +-v \ +-j 4 +``` + +The cases would be named `*.G.*` to denote one is generating the baselines. +It will take a while to compile all the cases and submit the code. +After the cases have been successfully compiled and submitted, you can check that +status of test by running the `cs.status.${TEST_ID}` file that was created in the scratch directory. + + + +## Step-2: Compare against previously generated baselines + +Now, switch to the development branch and be sure to update submodules. + + +``` +cd +git checkout bishtgautam/lnd/emi-example +Previous HEAD position was cfb7fc2b2... Merge branch 'qzhu-lbl/lnd/ch4_inundation_bugfix' (PR #2814) +Switched to branch 'bishtgautam/lnd/emi-example' +Your branch is up-to-date with 'origin/bishtgautam/lnd/emi-example'. + +# Checkout the appropriate submodules +git submodule update --init +``` + +Again initialize few settings. + +``` +cd cime/scripts + +# Let's use the settings as the last time +export MY_BASELINE_DIR=/global/cscratch1/sd/gbisht/e3sm_baselines + +# IMPORTANT: One needs to use the same BASELINE_NAME as in Step-1 because one wants +# to compare against the baselines previously generated in Step-1. +export BASELINE_NAME=cfb7fc2b2 + +# You can use the hash at the tip of your branch as the ID +export TEST_ID=9a2cd8459 +``` + +Run the test suite and compare (via `-c`) against previously generated baselines. + +``` +./create_test e3sm_land_developer \ +--baseline-root ${MY_BASELINE_DIR} \ +-b ${BASELINE_NAME} \ +-t ${TEST_ID} \ +-q debug \ +--walltime 00:30:00 \ +--mail-user $MAIL_USER \ +--mail-type all \ +-c \ +-v \ +-j 4 +``` + +The cases would be named `*.C.*` to denote one is comparing against previously generated baselines. Similar to the last time, a new `cs.status.${TEST_ID}` would be created and +you can check the status of test by running it.