How to: tests

Testing the code

This guide's aim is to provide instructions and clarifications regarding making and using tests for DKB. Therefore, it is presumed that general questions, such as "What is testing?" or "Why do we need to make tests?", are already answered. See "Further reading" section if some of them are not.

For now, DKB uses standard unittest module for Python testing.

It should be noted that working with DKB tests means adhering to rules of two sets:

1. Unittest module's guidelines and requirements.
2. DKB's own decisions regarding tests.

This guide is describing the second set of rules which is more strict than the first one. For example, unittest states that files with tests can be discovered as long as their names follow a pattern which is 'test*.py' by default (but can be changed), while it was decided that DKB's test files' names should start with "test_" and must correlate with code files' names. The first set of rules can be found in unittest's documentation.

Running tests

Each module should be tested in a similar way – running python -m unittest discover command from the module directory. Possibility of doing so relies on meeting some requirements regarding names and other things. It should be noted that this guide is not reflecting the unittest's requirements

Directory structure

Each stage should have a directory called tests. Test files' names in this directory should mirror the code files' names. For example, if a stage consists of two scripts called stage123.py and utils.py, the test files should be called test_stage123.py and test_utils.py.

Stage-level tests file: TO DO.

pyDKB structure: TO DO.

Writing tests

Apart from imports and other standard things, a test file should include two constructions: tests cases and test suite setup.

Test case

A test case must be a subclass of unittest.Testcase and should contain test methods and optional utility methods setUp() and tearDown(). A test method's name must start with "test" (to be found with discover) and be as informative as possible (it won't be called by hand too often, but will be displayed when something goes wrong). A test method's goal is to ensure that either something is correct:

def test_sum(self):
    s = my_func_sum(2, 2)
    self.assertEqual(s, 4)

or an exception is raised:

def test_divide_by_zero(self):
    with self.assertRaises(ZeroDivisionError):
        a = b / 0

See unittest documentation for a complete list of assert functions.

Sometimes several tests are performing identical actions before/after doing something. Consider the following code:

def test1(self):
    data = {… very complex combination of variables …}
    r1 = some_function(data)
    self.assertEqual(r1, r2)
def test2(self):
    data = {… same data as above …}
    r1 = some_other_function(data)
    self.assertEqual(r1, r2)

Such structure means that the data definition must be written twice (or copied) and both instances must be updated when the change occurs. This can be simplified by setUp() method which is executed before every test. In a similar way, tearDown() method runs after every test. Using these methods, the code above can be rewritten:

def setUp(self):
    self.data = {… very complex combination of variables …}
def tearDown(self):
    self.data = None
def test1(self):
    r1 = some_function(self.data)
    self.assertEqual(r1, r2)
def test2(self):
    r1 = some_other_function(self.data)
    self.assertEqual(r1, r2)

Note the "cleanup" of used self.data in tearDown().

Test suite setup

Test suite is used to make tests discover-able. It can be more or less copypasted from existing tests:

test_cases = (
    MyFirstTestCase,
    MySecondTestCase,
)


def load_tests(loader, tests, pattern):
    suite = unittest.TestSuite()
    for case in test_cases:
        suite.addTest(loader.loadTestsFromTestCase(case))
    return suite

Where test_cases should be adjusted according to names of cases.

Further reading

https://docs.python-guide.org/writing/tests

https://docs.python.org/2/library/unittest.html

Examples

Code in sample_tests branch - note that it wasn't updated to fit this guide to a letter yet (for example, test structure does not reflect the code structure). However, it can still be used as a working demonstration of DKB testing.