Split up the notebook-e2e-test to sepereted groups #53010
Conversation
|
/pj-rehearse more |
|
@atheo89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
openshift-ci
bot
added
the
approved
|
/pj-rehearse ci/rehearse/opendatahub-io/notebooks/main/notebooks-ubi9-e2e-tests |
|
@atheo89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@atheo89: job(s): ci/rehearse/opendatahub-io/notebooks/main/notebooks-ubi9-e2e-tests either don't exist or were not found to be affected, and cannot be rehearsed |
|
/pj-rehearse ci/rehearse/opendatahub-io/notebooks/main/codeserver-notebook-e2e-tests |
|
@atheo89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@atheo89: job(s): ci/rehearse/opendatahub-io/notebooks/main/codeserver-notebook-e2e-tests either don't exist or were not found to be affected, and cannot be rehearsed |
|
/pj-rehearse more |
|
@atheo89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
Looks legit to me. I am not at all familiar with OPC-CI yamls, so I'd want to see this running before I'd have the confidence to lgtm it. I am slightly concerned that the jobs end up touching images they are not testing. For example, |
|
Ok, 3 out of 8 rehearsals jobs failed - with almost 62% of success. Maybe in a second iteration of retest they would pass.
@jiridanek Indeed. However there is no filtering for image builds on the ocp-ci documentation, so i think we can not avoid it :/ |
|
/pj-rehearse pull-ci-opendatahub-io-notebooks-main-notebooks-ubi8-e2e-tests |
|
@atheo89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@atheo89: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
1 similar comment
|
@atheo89: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
|
/pj-rehearse pull-ci-opendatahub-io-notebooks-main-notebooks-ubi8-e2e-tests |
|
@atheo89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-opendatahub-io-notebooks-main-rstudio-notebook-e2e-tests |
|
@atheo89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@atheo89: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/pj-rehearse pull-ci-opendatahub-io-notebooks-main-intel-notebooks-e2e-tests |
|
@atheo89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@atheo89, If the problem persists, please contact Test Platform. |
that's broken currently, it won't pass https://issues.redhat.com/browse/RHOAIENG-8388 |
|
Upstream tracking issue: opendatahub-io/notebooks#562 |
|
@jiridanek: changing LGTM is restricted to collaborators In response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Signed-off-by: Nahshon Unna-Tsameret <[email protected]>
atheo89
force-pushed
the
seperate-e2e-test-v2
branch
from
8215b9b
to
fcc188b
Compare
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: atheo89, jiridanek The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
removed
the
approved
There was a problem hiding this comment.
That's some mistake, making changes in this file, no?
There was a problem hiding this comment.
I didn't, maybe some mistake during the rebase happened here...
There was a problem hiding this comment.
Maybe I will open a seperate PR, and i will close this
|
Close this in favor of : #53476 |
Related to: https://issues.redhat.com/browse/RHOAIENG-8399
This PR aims to improve testing efficiency and reduce testing time.
To archive that we had to change a bit the ocp-ci configuration.
Currently, all notebook tests are grouped under a single test suite called "notebooks-e2e-tests". However, this approach had several drawbacks:
To address these issues, we propose breaking down the unified test suite into separate tests, triggered only when relevant changes occur using the run_if_changed: (regex) option, this way will reduce the time and will spot easier problematic notebooks.
So the notebooks-e2e-tests broke into the following:
notebooks-ubi8-e2e-tests
Triggered when changes occur in the following repository folders:
run_if_changed: (base/ubi8-python-3.8/)|(jupyter/minimal/ubi8-python-3.8/)|(jupyter/datascience/ubi8-python-3.8/)|(jupyter/pytorch/ubi8-python-3.8/)|(jupyter/tensorflow/ubi8-python-3.8/)|(jupyter/trustyai/ubi8-python-3.8/)
anaconda-ubi8-e2e-tests
Triggered when changes occur in the following repository folders: (base/anaconda-python-3.8/)|(jupyter/datascience/anaconda-python-3.8/)
notebooks-ubi9-e2e-tests
Triggered when changes occur in the following repository folders:
run_if_changed: (base/ubi9-python-3.9/)|(jupyter/minimal/ubi9-python-3.9/)|(jupyter/datascience/ubi9-python-3.9/)|(jupyter/pytorch/ubi9-python-3.9/)|(jupyter/tensorflow/ubi9-python-3.9/)|(jupyter/trustyai/ubi9-python-3.9/)
codeserver-notebook-e2e-tests
Triggered when changes occur in the following repository folders:
run_if_changed: (base/ubi9-python-3.9/)|(codeserver/ubi9-python-3.9/)
rstudio-notebook-e2e-tests
Triggered when changes occur in the following repository folders:
run_if_changed: (base/c9s-python-3.9/)|(rstudio/c9s-python-3.9/)
runtimes-ubi8-e2e-test
Triggered when changes occur in the following repository folders:
run_if_changed: (base/ubi8-python-3.8/)|(runtimes/datascience/ubi8-python-3.8/)|(runtimes/pytorch/ubi8-python-3.8/)|(runtimes/tensorlow/ubi8-python-3.8/)
runtimes-ubi9-e2e-test
Triggered when changes occur in the following repository folders:
run_if_changed: (base/ubi9-python-3.9/)|(runtimes/datascience/ubi9-python-3.9/)|(runtimes/pytorch/ubi9-python-3.9/)|(runtimes/tensorlow/ubi9-python-3.9/)
8 intel-notebooks-e2e-tests
Triggered when changes occur in the following repository folders:
run_if_changed: (intel/base/gpu/ubi9-python-3.9/)|(jupyter/intel/pytorch/ubi9-python-3.9/)|(intel/runtimes/tensorflow/ubi9-python-3.9/*)
PS: The habana-notebooks-e2e-tests was already separated