Testing Standards

Code Testing

All models and tools within the IDAES code base are expected to have accompanying tests, which check for both coding errors and final results.

Testing Tools

All IDAES tests are written using pytest (pytest documentation).

Types of Tests

IDAES test are divided into three categories, which are used for organizing tests into different levels of rigor and complexity (and thus execution time). Lower level tests are are expected to run frequently, and thus need to keep execution time to a minimum to avoid delays, whilst higher level tests are run less frequently and can thus take longer to complete. The three categories used by IDAES are:

• unit: Unit tests are used to test basic functionality and execution of individual parts of the code. Within IDAES, these tests are generally used to test model construction, but not model solutions. These tests should be fast to run (less than 2 seconds), and should not require the use of a solver. Unit tests should ideally cover all lines of code within a model with the exception of those requiring a solver to complete (initialization and final solves).
• component: Component tests are used to test model solutions for single example cases in order to check that a model can be solved. These test obviously require the use of a solver, but should still be relatively quick to execute (ideally less than 10 seconds).
• integration: The final level of tests are integration tests, which are used for longer duration verification and validation tests. These tests are used to confirm model accuracy and robustness over a wide range of conditions, and as such can take longer to execute.

When and How to Run Tests

Developers

Testing Standards

How to Write Good Tests

Things to Include

• All tests should include a pytest.mark to indicate the type of test.
• Tests should be written that execute all branches in conditional statements, and should check to make sure the correct branch was taken.
• Any Exceptions raised by the code should be tested. You can use pytest.raises(ExceptionType, matches=str) to check that the correct type of Exception was raised and that the message matches the expected string.

Things to Avoid

• Tests should always contain an assert statement (or equivalent). It is easy to write a test that executes the code, but unless you add checks for specific behaviors all the test will tell you is if there are any Exceptions during execution.