Skip to content

UI visual regression testing to cover UI widgets visibility #17

UI visual regression testing to cover UI widgets visibility

UI visual regression testing to cover UI widgets visibility #17

Re-run triggered September 19, 2024 16:24
Status Failure
Total duration 8m 53s
Billable time 18m
Artifacts 4

ui_notebooks_test.yaml

on: pull_request
verify-0_basic_ray
8m 37s
verify-0_basic_ray
Fit to window
Zoom out
Zoom in

Annotations

10 errors and 1 notice
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: 861 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-3-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-3-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-3-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: Expected an image 592px by 78px, received 592px by 77px. 2102 pixels (ratio 0.05 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-4-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-4-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-4-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: 586 pixels (ratio 0.05 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-5-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-5-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-5-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: 831 pixels (ratio 0.03 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-6-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-6-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-6-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: 4236 pixels (ratio 0.04 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-7-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-7-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-7-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: 1062 pixels (ratio 0.04 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-8-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-8-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-8-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: 4701 pixels (ratio 0.04 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-9-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-9-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-9-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: 760 pixels (ratio 0.05 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-10-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-10-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-10-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: Expected an image 656px by 387px, received 656px by 404px. 5118 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-11-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-11-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-11-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs: ui-tests/tests/widget_notebook_example.test.ts#L58
1) tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs Error: Screenshot comparison failed: Expected an image 592px by 77px, received 592px by 78px. 253 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts-snapshots/widgets-cell-12-linux.png Received: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-12-actual.png Diff: /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/test-results/tests-widget_notebook_exam-bf2ca-ok-and-capture-cell-outputs/widgets-cell-12-diff.png 56 | 57 | if (captures[i]) { > 58 | expect.soft(captures[i]).toMatchSnapshot(image); // Compare pre-existing capture | ^ 59 | continue; 60 | } 61 | } at /home/runner/work/codeflare-sdk/codeflare-sdk/ui-tests/tests/widget_notebook_example.test.ts:58:34
🎭 Playwright Run Summary
1 failed tests/widget_notebook_example.test.ts:30:7 › Visual Regression › Run notebook and capture cell outputs

Artifacts

Produced during runtime
Name Size
ipywidgets-test-assets
8.19 MB
ipywidgets-test-report
8.39 MB