UI visual regression testing to cover UI widgets visibility #17
ui_notebooks_test.yaml
on: pull_request
verify-0_basic_ray
8m 37s
Annotations
10 errors and 1 notice
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
861 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-3-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-3-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-3-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
Expected an image 592px by 78px, received 592px by 77px. 2102 pixels (ratio 0.05 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-4-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-4-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-4-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
586 pixels (ratio 0.05 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-5-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-5-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-5-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
831 pixels (ratio 0.03 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-6-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-6-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-6-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
4236 pixels (ratio 0.04 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-7-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-7-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-7-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
1062 pixels (ratio 0.04 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-8-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-8-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-8-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
4701 pixels (ratio 0.04 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-9-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-9-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-9-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
760 pixels (ratio 0.05 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-10-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-10-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-10-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
Expected an image 656px by 387px, received 656px by 404px. 5118 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-11-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-11-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-11-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs:
ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
Error: Screenshot comparison failed:
Expected an image 592px by 77px, received 592px by 78px. 253 pixels (ratio 0.01 of all image pixels) are different.
Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-12-linux.png
Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-12-actual.png
Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-12-diff.png
56 |
57 | if (captures[i]) {
> 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture
| ^
59 | continue;
60 | }
61 | }
at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
|
🎭 Playwright Run Summary
1 failed
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
ipywidgets-test-assets
|
8.19 MB |
|
ipywidgets-test-report
|
8.39 MB |
|