Skip to content

Commit

Permalink
Update Working Practices with LFRic Apps instructions (#144)
Browse files Browse the repository at this point in the history
  • Loading branch information
@jennyhickson
jennyhickson authored Mar 5, 2024
1 parent 63f4b01 commit e8e8cae
Show file tree
Hide file tree
Showing 26 changed files with 344 additions and 227 deletions.
2 changes: 1 addition & 1 deletion source/Reviewers/codereview.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ will include areas such as:
The Ticket Summary/Code Review templates in each repository contain the details
of when testing against other repositories are required. These highlight where
the code is likely to interact. *e.g. if code in the shared/science folder in*
*JULES is modified then both the UM and LFRIc test suites will need to be run with that change.*
*JULES is modified then both the UM and LFRIc Apps test suites will need to be run with that change.*

If this testing doesn't pass then either
a) the change in ticket will need modifying so that the parent repository's test suite passes
Expand Down
32 changes: 14 additions & 18 deletions source/Reviewers/committinglinkedtickets.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ api between the codebases is broken. For this reason, where possible, all parts
a linked ticket should be committed on the same day to avoid nightly tests failing.

:ref:`Multi-repository <multirepo>` changes are nested, and the different branches
will need approaching in the correct order. The UM and LFRIc are the key places where
these overlap.
will need approaching in the correct order. The UM and LFRIc Apps are the key
places where these overlap.

1. Everything except UM and LFRic can be worked on seperately and should be committed first.
2. The UM relies on code from all of the above code bases, and will need that code for both testing and committing.
3. LFRic relies on code from all of the above, and will need that code for both testing and committing.
1. Everything except UM and LFRic Apps can be worked on separately and should be committed first.
2. The UM relies on code from all of the above code bases (except LFRic Core), and will need that code for both testing and committing.
3. LFRic Apps relies on code from all of the above, and will need that code for both testing and committing.

.. tip::

Expand Down Expand Up @@ -48,8 +48,8 @@ Details for testing multi-repository tickets are included :ref:`here <multirepo>

**In summary:**

* JULES, UKCA and other child repositories can be tested using their standalone
test suites as described on the How to Commit page.
* JULES, UKCA, LFRic Core and other child repositories can be tested using their
standalone test suites as described on the How to Commit page.

* Local working copies can be passed to the UM on the command line

Expand All @@ -61,7 +61,7 @@ Details for testing multi-repository tickets are included :ref:`here <multirepo>
(e.g. in the above example the jules and ukca groups are tested).

* Local working copies of any linked UM, JULES, UKCA or other repositories
can be passed to LFRic through <lfric_trunk>/lfric_atm/fcm-make/parameters.sh.
can be passed to LFRic Apps through <lfric_apps_trunk>/dependencies.sh.

.. code-block:: RST
Expand All @@ -73,7 +73,7 @@ Details for testing multi-repository tickets are included :ref:`here <multirepo>
It is always important that branches and working copies used for testing
multiple repositories together have been taken at the same point in time. If
this isn't the case then API breaking changes may be included in one repository
but not another which will cause tests to break.
but not another which will cause tests to fail.

The developer will likely have used branches taken from the last releases which
are a known set of stable revisions which work together.
Expand All @@ -85,7 +85,7 @@ Details for testing multi-repository tickets are included :ref:`here <multirepo>
.. tip::

If some of the changes in this set of tickets have already been committed
then see steps 2, 4 and 5 below on how to include those changes in your testing.
then see steps 2 and 4 below on how to include those changes in your testing.
This is instead of the steps described above.

e.g. If JULES changes have been committed and the revision number modified in
Expand All @@ -99,7 +99,7 @@ Committing linked tickets

Once you are happy with all your testing then the commit sequence is as follows:

1. Commit all trunks **except** UM and LFRic. Make note of the commit revision numbers.
1. Commit all trunks **except** UM and LFRic Apps. Make note of the commit revision numbers.

2. Update <um_trunk>/rose-stem/rose-suite.conf

Expand All @@ -113,7 +113,7 @@ Once you are happy with all your testing then the commit sequence is as follows:
3. Commit UM

4. Update <lfric_trunk>/lfric_atm/fcm-make/parameters.sh
4. Update <lfric_apps_trunk>/dependencies.sh

* Modify ``*_rev`` variables for all other repositories you have updated to point to the the new commit revisions.
* Remove any branch references from the ``*_sources`` variables.
Expand All @@ -127,10 +127,6 @@ Once you are happy with all your testing then the commit sequence is as follows:
export um_sources=
export jules_sources=
5. **For changes including JULES and LFRic** also update <lfric_trunk>/miniapps/lfric_coupled/rose-stem/app/fcm_make_river/rose-app.conf
5. Commit LFRic Apps

* Modify ``JULES_SOURCE_VN`` to also point to the same revision as parameters.sh.

5. Commit LFRic

You may choose to run a subset of tests before completing the UM and LFRic commits in turn to validate your changes.
You may choose to run a subset of tests before completing the UM and LFRic Apps commits in turn to validate your changes.
120 changes: 62 additions & 58 deletions source/Reviewers/howtocommit.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ The process for committing a ticket follows this sequence with details for each
Before You Start:
* Is anyone else committing?

* `Trunk Status`_ is used to coordinate trunk commits for UM, JULES and UKCA.
* LFRic Trunk commits are coordinated through the dashboard in the Model Systems Teams Chat.
* `Trunk Status`_ is used to coordinate trunk commits for UM, JULES, LFRic Apps and UKCA.
* LFRic Core Trunk commits are coordinated through the dashboard in the Model Systems Teams Chat.
* Simple, not conflicting commits can be done in parallel if reviewers all agree.
* Changes with KGO or Macros usually require sole access to the trunk.
* Check how many commits have happened today. Suggested limit per day, per repository is 4.
Expand Down Expand Up @@ -68,7 +68,16 @@ resolve any conflicts.
fcm merge fcm:ukca.x_br/dev/dev_name/branch_name
fcm cf
.. tab-item:: LFRic
.. tab-item:: LFRic Apps

.. code-block:: RST
fcm co fcm:lfric_apps.x_tr chosen_name
cd chosen_name
fcm merge fcm:lfric_apps.x_br/dev/dev_name/branch_name
fcm cf
.. tab-item:: LFRic Core

.. code-block:: RST
Expand Down Expand Up @@ -197,8 +206,8 @@ are no clashes with what else has gone on trunk.
.. code-block:: RST
rose stem --group=debug_compile
rose stem --group=developer,ex1a_developer
rose stem --group=all,ex1a
OR rose stem --group=developer,ex1a_developer
OR rose stem --group=all,ex1a
If there is a change to the build configs then you may need to turn off
prebuilds. To do so update `rose-stem/site/meto/variables.rc` such that
Expand Down Expand Up @@ -234,17 +243,30 @@ are no clashes with what else has gone on trunk.
rose stem --group=developer,ukca --source=. --source=/path/to/UKCA/working/copy
.. tab-item:: LFRic
.. tab-item:: LFRic Apps

LFRic Apps rose-stem contains tests spanning all the applications
included in the repository. At the very least run the developer group
which gives a basic level of tests spanning everything. The full set of
tests may be warranted for any application that has had more complex changes.

LFRic has many rose-stem suites for its different applications. Run the
test suite command from the top level of the repository to run a complete
set of the rose-stem suites.
.. code-block::
export CYLC_VERSION=8
rose-stem --group=developer
OR e.g. rose-stem --group=developer,gungho_model
cylc play <working copy name>
.. tab-item:: LFRic Core

Run the test suite command from the top level of the repository to run
a complete set of the rose-stem developer suites.

.. code-block::
make test-suite
and
make test-suite SUITE_GROUP=nightly
.. tab-item:: UM docs

Expand Down Expand Up @@ -301,6 +323,8 @@ for all affected tests before you commit to the trunk.
[rose-ana]
kgo-database=.true.
.. _kgo_instructions:

.. tab-set::

.. tab-item:: UM + LFRic Inputs
Expand Down Expand Up @@ -409,7 +433,31 @@ for all affected tests before you commit to the trunk.
4. Rerun the rose-stem tests to make sure nothing is broken.

.. tab-item:: LFRic

.. tab-item:: LFRic Apps

KGO Checksums are stored in the repository alongside the code and can
be updated using a script. This can be done by either the code reviewer
or by the developer (before submitting their changes for review). In the
latter case, the update will need redoing by the reviewer before commit
if there are merge conflicts in the checksum files.

1. Run the rose stem tasks that require a KGO update, plus any other testing required (see above) - if unsure run the `all` group.

.. code-block:: RST
export CYLC_VERSION=8
rose stem --group=all
cylc play <suite name>
2. Run the checksum update script stored in `<working copy>/rose-stem/bin`.

.. code-block::
python3 ./rose-stem/bin/update_branch_kgos.py -s <suite name> -w <path to working copy>
.. tab-item:: LFRic Core

KGO Checksums are stored in the repository alongside the code. If there
is a merge conflict within these files it is the developers responsibility
Expand Down Expand Up @@ -453,66 +501,22 @@ An editor will open requesting a log message which should be in this format:

.. tab-set::

.. tab-item:: UM

.. code-block::
#ticket_number : Author : Reason for the change : ticket_type : code_area : regression : severity
This layout enables a script to parse the commits that make up each release.

1. Ticket number
2. Author : SRS username
3. Reason for the change : Title of the ticket
4. Ticket Type

* enhancement, defect, task, optimisation

5. Code Area - Select the most appropriate from:

* technical, dynamics, ukca, bl_jules, convection, radiation, gwd, lsp_cloud, stochastic_physics, coupling, idealised, rose_stem, fcm_make, meta_data, utils, fieldcalc, other

6. Regression:

* kgo_and_macro, update_kgo, upgrade_macro, regression

7. Severity

* wholesale, significant, minor, trivial


.. tab-item:: JULES & JULES docs

.. code-block::
#<ticket number> for <original author> - <ticket title>
where original author is the srs username.

.. tab-item:: UKCA
.. tab-item:: All others

.. code-block::
#ticket_number : Author : Ticket title
where author is the srs username.

.. tab-item:: LFRic
.. tab-item:: LFRic Core

.. code-block::
#<ticket number> for <original author>: <ticket title>
where original author is the authors proper name.

.. tab-item:: UM docs

.. code-block::
#ticket_number : Author : Description : XXX (YYY ZZZ etc)
where XXX, YYY etc are the three letter codes for any UMDPs modified.

.. note::
New!! Remove any **blocks:** and **blockedby:** keywords from this ticket and any referenced. Comment on any unblocked tickets to alert the developers.

Expand Down
2 changes: 1 addition & 1 deletion source/WorkingPractices/TestSuites/jules.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ stored :ref:`KGO files <kgo>`.
.. note::
If there are JULES changes to shared science code or metadata then these
changes will need to be tested :ref:`with the UM<um_testing>` and
:ref:`with LFRic<lfric_test>`. If you have access to LFRic, the
:ref:`with LFRic Apps<lfric_apps_test>`. If you have access to LFRic, the
:ref:`traclog` will state whether LFRic testing is required based on the branch
diff. If you do not have LFRic access, this testing will need to be completed by
your Met Office contact.
Expand Down
93 changes: 93 additions & 0 deletions source/WorkingPractices/TestSuites/lfric_apps.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
.. _lfric_apps_test:

Testing LFRic Apps
==================

LFRic Apps testing is run with the following commands from a working copy:

.. code-block::
export CYLC_VERSION=8
rose stem --group=developer
cylc play <working copy name>
-----

The LFRic Apps rose stem includes a range of tests to exercise all the applications
stored in this repository, using multiple compilers, and checksum and plot tasks to
confirm the outputs.

.. tip::

For LFRic Apps it is possible to update the checksum files on your branch to
aid with testing and development. To do so run the rose-stem suite as described
to cover all failing tests, then run the following from your working copy

.. code-block::
python3 ./rose-stem/bin/update_branch_kgos.py -s <suite name> -w <path to working copy>
More details on the KGO update proceedures for all repositories can be found
:ref:`here <kgo_instructions>`.

Below is a (by no means comprehensive) set of groups that you may wish to use on
Met Office systems. Note that there is a lot of overlap between these groups,
and that you can specify more than one at once, e.g. ``--group=developer,gungho_model``.



+--------------------+----------------------------------------------------------+
| Group | Description |
+====================+==========================================================+
| developer | Standard group of tests that every change is expected |
| | |
| | to pass before commit. It is a useful checkpoint during |
| | |
| | development. |
+--------------------+----------------------------------------------------------+
| nightly | More thorough testing group. This includes everything in |
| | |
| | developer plus some longer and more complex tests. It is |
| | |
| | run automatically every night and monitored by the SSD |
| | |
| | team. |
+--------------------+----------------------------------------------------------+
| all | The complete test suite, including all longer runs and |
| | |
| | less commonly used utilites. This is run automatically |
| | |
| | every week and monitored by the SSD team. All |
| | |
| | :ref:`KGO <kgo>` changing tickets need to run this group.|
+--------------------+----------------------------------------------------------+
+--------------------+----------------------------------------------------------+
| build | Compile tasks for all applications and science areas |
+--------------------+----------------------------------------------------------+
| unit_tests | Unit tests for all applications and science areas |
+--------------------+----------------------------------------------------------+
| integration_tests | Integration tests for all applications and science areas |
+--------------------+----------------------------------------------------------+
| xc40/spice/monsoon | All tests designed to run on the named platform. |
+--------------------+----------------------------------------------------------+
| scripts | All of the auxillary scripts that are designed to check |
| | |
| | the code standards in ways that aren't tested by the |
| | |
| | compiler. |
+--------------------+----------------------------------------------------------+

As well as these general groups, each area in ``<lfric_apps>/applications`` and
``<lfric_apps>/science`` have a set of specific groups that are structured as below,
with ``name`` matching the directory name of the area.

+--------------------+----------------------------------------------------------+
| Group | Description |
+====================+==========================================================+
| <name> | Full set of tests for this area |
+--------------------+----------------------------------------------------------+
| <name>_<group> | Tests for this area that are part of this group |
| | |
| | e.g. gungho_developer or lfric_atm_spice |
+--------------------+----------------------------------------------------------+

6 changes: 3 additions & 3 deletions source/WorkingPractices/TestSuites/lfric.rst → ...orkingPractices/TestSuites/lfric_core.rst
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _lfric_test:
.. _lfric_core_test:

Testing LFRic
=============
Testing LFRic Core
==================

LFRic testing is run with the following command from a working copy:

Expand Down
Loading

0 comments on commit e8e8cae

Please sign in to comment.