Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests: Cypress tests imrpovements #2816

Merged
merged 12 commits into from
Nov 20, 2023
Merged

Tests: Cypress tests imrpovements #2816

merged 12 commits into from
Nov 20, 2023

Conversation

mike10ca
Copy link
Contributor

What it solves

Resolves #2814

How this PR fixes it

  • Create smaller automation tests flows
  • Move other tests outside of smoke test

Copy link

github-actions bot commented Nov 16, 2023

Branch preview

✅ Deploy successful!

https://cypress_test_improvements--walletweb.review-wallet-web.5afe.dev

Copy link

github-actions bot commented Nov 16, 2023

ESLint Summary View Full Report

Annotations are provided inline on the Files Changed tab. You can also see all annotations that were generated on the annotations page.

Type Occurrences Fixable
Errors 0 0
Warnings 0 0
Ignored 0 N/A
  • Result: ✅ success
  • Annotations: 0 total

Report generated by eslint-plus-action

Copy link

github-actions bot commented Nov 16, 2023

📦 Next.js Bundle Analysis for safe-wallet-web

This analysis was generated by the Next.js Bundle Analysis action. 🤖

Three Pages Changed Size

The following pages changed size from the code in this PR compared to its base branch:

Page Size (compressed) First Load
/addOwner 3.56 KB (🟡 +39 B) 1.4 MB
/new-safe/create 11.83 KB (🟡 +8 B) 1.41 MB
/settings/setup 24.21 KB (🟡 +78 B) 1.42 MB
Details

Only the gzipped size is provided here based on an expert tip.

First Load is the size of the global bundle plus the bundle for the individual page. If a user were to show up to your website and land on a given page, the first load size represents the amount of javascript that user would need to download. If next/link is used, subsequent page loads would only need to download that page's bundle (the number in the "Size" column), since the global bundle has already been downloaded.

Any third party scripts you have added directly to your app using the <script> tag are not accounted for in this analysis

Next to the size is how much the size has increased or decreased compared with the base branch of this PR. If this percentage has increased by 20% or more, there will be a red status indicator applied, indicating that special attention should be given to this.

Copy link

github-actions bot commented Nov 16, 2023

Coverage report

St.
Category Percentage Covered / Total
🟡 Statements
75.12% (-0.01% 🔻)
10095/13439
🔴 Branches
49.67% (-0.02% 🔻)
2051/4129
🔴 Functions 57.62% 1504/2610
🟡 Lines
76.7% (-0.01% 🔻)
9126/11899
Show files with reduced coverage 🔻
St.
File Statements Branches Functions Lines
🟢
... / hooks.ts
77.36%
54.29% (-3.29% 🔻)
68.42% 80.85%

Test suite run success

1105 tests passing in 155 suites.

Report generated by 🧪jest coverage report action from 13104c6

Copy link

github-actions bot commented Nov 20, 2023

ESLint Summary View Full Report

Annotations are provided inline on the Files Changed tab. You can also see all annotations that were generated on the annotations page.

Type Occurrences Fixable
Errors 0 0
Warnings 0 0
Ignored 0 N/A
  • Result: ✅ success
  • Annotations: 0 total

Report generated by eslint-plus-action

@mike10ca mike10ca changed the title Tests: Update tests structure Tests: Cypress tests imrpovements Nov 20, 2023
@mike10ca mike10ca marked this pull request as ready for review November 20, 2023 11:23
@@ -14,7 +14,7 @@ jobs:
strategy:
fail-fast: false
matrix:
containers: [1, 2, 3]
containers: [1, 2, 3, 4, 5]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change? From what I see each container still runs for about 7 minutes. There's more overhead starting the machine than running the tests themselves.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katspaugh we will be adding more tests to the suite. Adding more machines will reduce time spent on tests going forward.

name: Regression on demand tests

on:
workflow_dispatch:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katspaugh no, from what I understand it must be merged to dev first so the workflow is available

@@ -15,7 +15,7 @@ jobs:
strategy:
fail-fast: false
matrix:
containers: [1, 2, 3]
containers: [1, 2]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another arbitrary change, what's the reasoning here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katspaugh there are not many tests in safe apps, that's why reducing number of machines makes sense. As you said, there's more overhead starting the machine than running the tests themselves.

Copy link
Member

@katspaugh katspaugh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Next time please try to limit the scope of your PRs and break them down into several, documented, PRs. This one mixes a lot of changes from tests themselves to CI settings across several workflows, and not everything is reflected in the PR description.

@mike10ca mike10ca merged commit 4cd3f3c into dev Nov 20, 2023
15 checks passed
@mike10ca mike10ca deleted the cypress-test-improvements branch November 20, 2023 13:02
@github-actions github-actions bot locked and limited conversation to collaborators Nov 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create smoke happy path automation flow
2 participants