Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add performance test #542

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

davidbrochart
Copy link
Member

This test launches 10 clients, all executing the same notebook. The notebook consists of 100 cells, each cell sleeps for 0.1 second. We check:

  • the overall time execution of all notebooks, taking into account the kernel launch time (0.5 s) and the notebook execution time (100 * 0.1 = 10 s), with a margin of 100% on the notebook execution time (TimeoutError if exceeded).
  • the mean cell execution time (0.1 s) with a 50% margin (exitcode 1 if exceeded).

@davidbrochart
Copy link
Member Author

There might be a temporary conda issue on OSX, but other than that this is ready to be reviewed @jtpio.

@SylvainCorlay
Copy link
Member

This looks pretty good. I am happy to get this in if you think this is ready @davidbrochart.

I think we can probably set up ASV after this.

@davidbrochart
Copy link
Member Author

Should it be only in ASV? The thing is that we don't know which machine we get in the CI, so if the machine is very slow the test can fail, or we should somehow take into account the machine's CPU frequency in the test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants