Create proxy function to detect the class to be used to run tests

[flo-renaud]

PRCI is crashing when the following test definition is used:

  fedora-30/test_smb:
    requires: [fedora-30/build]
    priority: 50
    job:
      class: RunADTests
      args:
        build_url: '{fedora-30/build_url}'
        test_suite: test_integration/test_smb.py
        template: *ci-master-f30
        timeout: 4800
        topology: *ad_master_2client

The journal logs for PRCI logs show the following:

bash[31394]: Traceback (most recent call last):
bash[31394]:   File "/root/freeipa-pr-ci/github/prci.py", line 316, in <module>
bash[31394]:     main()
bash[31394]:   File "/root/freeipa-pr-ci/github/prci.py", line 291, in main
bash[31394]:     for task in process_pull_request(world, pull_request, repo_url):
bash[31394]:   File "/root/freeipa-pr-ci/github/prci.py", line 124, in process_pull_request
bash[31394]:     pull_request.author, repository_url, task_data, JobDispatcher
bash[31394]:   File "/root/freeipa-pr-ci/github/internals/entities.py", line 593, in __init__
bash[31394]:     "git_refspec": self.ref_spec
bash[31394]:   File "/root/freeipa-pr-ci/github/internals/entities.py", line 809, in __init__
bash[31394]:     self.task_class = getattr(tasks, job_data["class"])
bash[31394]: AttributeError: module 'tasks.tasks' has no attribute 'RunADtests'
systemd[1]: prci.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: prci.service: Failed with result 'exit-code'.

So we have 2 issues:

• using RunADTests makes PRCI crash
• when PRCI crashes, the PR is showing as green (see for instance PR 3602) and we don't detect that some of the tests were not triggered.

[wladich]

I want to note that In #3602 there is no sign of failed job for test_sssd.py as if it is not defined in prci definition at all

[netoarmando]

The problem here is that the wrong class was used:

bash[31394]: AttributeError: module 'tasks.tasks' has no attribute 'RunADtests'

It should have been RunADTests, uppercase T. I believe that the test definition was fixed and the logs are older than that OR executed by another runner, so that's why we can have the crash on one runner and a green execution later on another runner.

Anyway, among other things, PR-CI breaks if there is any typo in the class specified in the definitions.

We can fix that by handling this case and reporting back the status to Github, so developers can fix their test definitions.

[netoarmando]

A better solution for that it to remove this 'class' argument from test definitions, replacing it with a proxy function, in PR-CI, that will map some rules and the class to return, examples:

• test contains ad then return class RunADTests
• test contains py then return class RunPytest

[netoarmando]

Renamed the issue to describe the goal.