Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests: Cypress tests imrpovements #2816

Merged
merged 12 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/workflows/ondemand.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Regression on demand tests

on:
workflow_dispatch:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katspaugh no, from what I understand it must be merged to dev first so the workflow is available


concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
e2e:
runs-on: ubuntu-20.04
name: Cypress Regression on demand tests
strategy:
fail-fast: false
matrix:
containers: [1, 2, 3, 4, 5]
steps:
- uses: actions/checkout@v3

- uses: ./.github/workflows/yarn

- name: Install Cypress
run: |
./node_modules/.bin/cypress install

- uses: ./.github/workflows/build
with:
secrets: ${{ toJSON(secrets) }}
e2e_mnemonic: ${{ secrets.NEXT_PUBLIC_CYPRESS_MNEMONIC }}

- name: Serve
run: yarn serve &

- uses: cypress-io/github-action@v4
with:
parallel: true
spec: cypress/e2e/**/*.cy.js
browser: chrome
record: true
config: baseUrl=http://localhost:8080
group: 'Regression on demand tests'
env:
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
48 changes: 48 additions & 0 deletions .github/workflows/regression.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Regression tests

on:
pull_request:
branches:
- 'release**'
- 'release/**'

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
e2e:
runs-on: ubuntu-20.04
name: Cypress Regression tests
strategy:
fail-fast: false
matrix:
containers: [1, 2, 3]
steps:
- uses: actions/checkout@v3

- uses: ./.github/workflows/yarn

- name: Install Cypress
run: |
./node_modules/.bin/cypress install

- uses: ./.github/workflows/build
with:
secrets: ${{ toJSON(secrets) }}
e2e_mnemonic: ${{ secrets.NEXT_PUBLIC_CYPRESS_MNEMONIC }}

- name: Serve
run: yarn serve &

- uses: cypress-io/github-action@v4
with:
parallel: true
spec: cypress/e2e/**/*.cy.js
browser: chrome
record: true
config: baseUrl=http://localhost:8080
group: 'Regression tests'
env:
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
2 changes: 1 addition & 1 deletion .github/workflows/safe-apps-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
strategy:
fail-fast: false
matrix:
containers: [1, 2, 3]
containers: [1, 2]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another arbitrary change, what's the reasoning here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katspaugh there are not many tests in safe apps, that's why reducing number of machines makes sense. As you said, there's more overhead starting the machine than running the tests themselves.

steps:
- uses: actions/checkout@v3

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/e2e.yml → .github/workflows/smoke.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: e2e
name: Smoke tests

on:
pull_request:
Expand All @@ -14,7 +14,7 @@ jobs:
strategy:
fail-fast: false
matrix:
containers: [1, 2, 3]
containers: [1, 2, 3, 4, 5]
steps:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change? From what I see each container still runs for about 7 minutes. There's more overhead starting the machine than running the tests themselves.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katspaugh we will be adding more tests to the suite. Adding more machines will reduce time spent on tests going forward.

- uses: actions/checkout@v3

Expand Down
12 changes: 11 additions & 1 deletion cypress.config.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { defineConfig } from 'cypress'
import * as fs from 'fs'

export default defineConfig({
projectId: 'exhdra',
Expand All @@ -8,8 +9,17 @@ export default defineConfig({
runMode: 1,
openMode: 0,
},

e2e: {
setupNodeEvents(on, config) {
on('after:spec', (spec, results) => {
if (results && results.video) {
const failures = results.tests.some((test) => test.attempts.some((attempt) => attempt.state === 'failed'))
if (!failures) {
fs.unlinkSync(results.video)
}
}
})
},
baseUrl: 'http://localhost:3000',
testIsolation: false,
hideXHR: true,
Expand Down
21 changes: 0 additions & 21 deletions cypress/e2e/create_safe.cy.js

This file was deleted.

29 changes: 0 additions & 29 deletions cypress/e2e/custom_apps.cy.js

This file was deleted.

44 changes: 0 additions & 44 deletions cypress/e2e/intercom.cy.js

This file was deleted.

54 changes: 0 additions & 54 deletions cypress/e2e/non_owner_spending_limit.cy.js

This file was deleted.

2 changes: 1 addition & 1 deletion cypress/e2e/pages/balances.pages.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ const hiddenTokenIcon = 'svg[data-testid="VisibilityOffOutlinedIcon"]'
const hideTokenDefaultString = 'Hide tokens'
const assetNameSortBtnStr = 'Asset'
const assetBalanceSortBtnStr = 'Balance'
const sendBtnStr = 'Send'
export const sendBtnStr = 'Send'
const sendTokensStr = 'Send tokens'

const pageRowsDefault = '25'
Expand Down
11 changes: 8 additions & 3 deletions cypress/e2e/pages/create_wallet.pages.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@ const createNewSafeBtn = 'span[data-track="create-safe: Continue to creation"]'
const connectWalletBtn = 'Connect wallet'
const googleConnectBtn = '[data-testid="google-connect-btn"]'
const googleSignedinBtn = '[data-testid="signed-in-account-btn"]'
const googleAccountInfoHeader = '[data-testid="open-account-center"]'
export const accountInfoHeader = '[data-testid="open-account-center"]'
const reviewStepOwnerInfo = '[data-testid="review-step-owner-info"]'
const reviewStepNextBtn = '[data-testid="review-step-next-btn"]'
const safeCreationStatusInfo = '[data-testid="safe-status-info"]'
const startUsingSafeBtn = '[data-testid="start-using-safe-btn"]'
const sponsorIcon = '[data-testid="sponsor-icon"]'
const networkFeeSection = '[data-tetid="network-fee-section"]'
const nextBtn = '[data-testid="next-btn"]'
const backBtn = '[data-testid="back-btn"]'

const sponsorStr = 'Your account is sponsored by Goerli'
const safeCreationProcessing = 'Transaction is being executed'
Expand All @@ -32,6 +34,9 @@ export const walletName = 'test1-sepolia-safe'
export const defaltSepoliaPlaceholder = 'Sepolia Safe'
const welcomeToSafeStr = 'Welcome to Safe'

export function clickOnBackBtn() {
cy.get(backBtn).should('be.enabled').click()
}
export function verifySafeIsBeingCreated() {
cy.get(safeCreationStatusInfo).should('have.text', safeCreationProcessing)
}
Expand Down Expand Up @@ -68,7 +73,7 @@ export function verifyGoogleSignin() {
}

export function verifyGoogleAccountInfoInHeader() {
return cy.get(googleAccountInfoHeader).should('exist')
return cy.get(accountInfoHeader).should('exist')
}

export function verifyPolicy1_1() {
Expand Down Expand Up @@ -128,7 +133,7 @@ export function selectNetwork(network, regex = false) {
}

export function clickOnNextBtn() {
cy.contains('button', 'Next').click()
cy.get(nextBtn).should('be.enabled').click()
}

export function verifyOwnerName(name, index) {
Expand Down
10 changes: 10 additions & 0 deletions cypress/e2e/pages/main.page.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,16 @@ export function clickOnSideMenuItem(item) {
cy.get('p').contains(item).click()
}

export function waitForTrnsactionHistoryToComplete() {
cy.intercept('GET', constants.transactionHistoryEndpoint).as('History')
cy.wait('@History')
}

export function waitForSafeListRequestToComplete() {
cy.intercept('GET', constants.safeListEndpoint).as('Safes')
cy.wait('@Safes')
}

export function acceptCookies(index = 0) {
cy.wait(1000)

Expand Down
2 changes: 2 additions & 0 deletions cypress/e2e/pages/navigation.page.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
export const sideNavSettingsIcon = '[data-testid="settings-nav-icon"]'
export const setupSection = '[data-testid="setup-section"]'
export const modalBackBtn = '[data-testid="modal-back-btn"]'

export function clickOnSideNavigation(option) {
cy.get(option).should('exist').click()
Expand Down
Loading
Loading