-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add regression tests #29
Conversation
This PR is done and ready to review. Could @lxavier or @kosterried pick this up. Should not be a big thing, since it basically adds regression testing. The edits in the testsuite Python code are minimal. |
@ofuhrer seems to work, but I get an error on kesch when doing make test, see below:
|
@lxavier Sorry, obviously my strategy for cleaning away the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good to me, I was even able to make the test fail by changing the thresholds !
I am still trying to get our jenkins to work, I will merge once this is passed
@guydemorsier there is a PR from Oli but it fails with int2lm here, do you see what could be wrong : |
the issue is in the was the testsuite is integrated in int2lm, I will fix it in int2lm. |
@@ -66,13 +68,17 @@ def check(): | |||
# check for output lists | |||
try: | |||
nout_list = get_param(rundir+nlfile, lnout) | |||
assert(nout_list != '') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks to the assert, I saw this is in fact not working in some tests, see
#37
@kosterried and @guydemorsier I merge this, but I will need to fix some of the cosmo test see |
@ofuhrer congratulation first external contribution :) |
This PR adds regression tests to the test suite repository. The regression tests use a surrogate model
model.py
which emulates the behavior of the COSMO model and is used for rapid regression testing. Some fixes had to be applied to the testsuite Python scripts in order to make all tests pass. Would be great if we could add this to the testing plan (which I don't think I have access to).Note, to run these tests on Piz Kesch, you can simply execute
make test
in the top-level testuite directory (see image below for results).