-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move multi repo testing #168
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
15fc710
move multi-repo testing
james-bruten-mo 3b94b02
merge in main
james-bruten-mo 0ad09bf
modify multi-repo testing page
james-bruten-mo 86b74e7
delete tip box deleted in previous conflicting change
james-bruten-mo 3057554
review changes
james-bruten-mo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
106 changes: 106 additions & 0 deletions
106
source/WorkingPractices/TestSuites/multi-repo_testing.rst
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
.. _multi-repo_testing: | ||
|
||
Multi-Repository Testing | ||
======================== | ||
|
||
Multi-repository changes are expected to pass the regression tests for all the | ||
repositories involved. To carry out the tests involved in a linked ticket it can | ||
be helpful to refer to the :ref:`semi-concentric circles figure <multirepo>`; layering the testing | ||
from the inside out as needed. Further details of how testing in each | ||
repository is handled can be found :ref:`here <testing>`. Compatible | ||
code revisions are needed for testing across repositories as described above. | ||
|
||
Testing changes in JULES, LFRic Core, UKCA, or any other child repositories is | ||
as simple as running the standalone test procedures for these codebases. | ||
|
||
.. important:: | ||
|
||
When specifying an alternative source in the ``dependencies.sh`` file the revision for the source **must** be updated. | ||
|
||
* If setting the source as an fcm URL, the mirror needs to be used and the revision can either be blank (for latest commit) or any valid revision for that branch. | ||
* If setting the source as a Working Copy, the hostname needs to be provided (as Hostname:Path) and the revision must be blank. | ||
|
||
Testing the UM with other repositories | ||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
To test the UM, any changes to JULES, UKCA, Socrates, CASIM etc will also need | ||
to be included. This is done by adding another source to the rose stem command | ||
line. | ||
|
||
1. :ref:`Checkout<checkout>` a UM working copy | ||
|
||
- this may be your branch from a linked ticket, or a clean trunk copy | ||
at either the last release or a suitable head of trunk revision. | ||
|
||
2. Run rose stem, including a source code path to every branch involved. As a minimum | ||
run ``developer`` group and all groups that cover the repositories being tested. | ||
|
||
.. code-block:: | ||
|
||
rose stem --group=developer,jules,ukca --source=. --source=/path/to/jules/changes --source=/path/to/ukca/changes | ||
|
||
The source paths involved can either be to local working copies or links to the | ||
fcm source control e.g. ``fcm:jules.xm_br/dev/user/branch_name``. As many source | ||
paths as needed can be added to the list. | ||
|
||
Testing LFRic Apps with other repositories | ||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
LFRic Apps testing needs to encompass all of the other repositories affected. | ||
Paths to the other codebases involved should be added to | ||
``dependencies.sh`` under each of the ``*_sources`` variables. Again | ||
these paths can either be to local changes or those in the repository. | ||
|
||
1. :ref:`Checkout<checkout>` an LFRic Apps working copy | ||
|
||
- this may be your branch from a linked ticket, or a clean trunk copy | ||
at either the last release or a suitable head of trunk revision. | ||
|
||
2. Update dependencies.sh to point to all other code changes, e.g. | ||
|
||
.. code-block:: RST | ||
|
||
lfric_core_rev= | ||
lfric_core_sources=fcm:lfric.xm_br/path/to/branch | ||
|
||
um_rev= | ||
um_sources=vldXXX:/path/to/um/working/copy | ||
|
||
3a. Run the lfric_atm developer test-suite | ||
|
||
- suitable for testing changes in other repositories that do not | ||
include any LFRic Apps changes | ||
|
||
.. code-block:: | ||
|
||
export CYLC_VERSION=8 | ||
rose stem --group=lfric_atm_developer | ||
cylc play <working copy name> | ||
cylc gui | ||
|
||
3b. Run the full developer test-suite | ||
|
||
- suitable for testing LFRic Apps changes with other repositories, or expanding | ||
testing if lfric_atm tests have shown errors. | ||
|
||
.. code-block:: | ||
|
||
export CYLC_VERSION=8 | ||
rose stem --group=developer | ||
cylc play <working copy name> | ||
cylc gui | ||
|
||
More details on LFRic Apps testing are found :ref:`here<lfric_apps_test>`. | ||
|
||
.. note:: | ||
If any of the testing shows up failures then there are two possible ways to | ||
proceed: | ||
|
||
1. The changes made should be re-written to avoid breaking the dependant | ||
repositories | ||
|
||
2. The changes made directly affect the interface between repositories and | ||
therefore a change is also needed to the parent repository to adapt to that change. | ||
|
||
If you're uncertain which route to take then the Code Owners involved will | ||
hopefully be able to advise. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
worth adding the cylc gui line to this code block and the one below while we're at it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done