From 191ab10d2569ca2bf0f75c63fe2f389b9a4371d5 Mon Sep 17 00:00:00 2001 From: seaona <54408225+seaona@users.noreply.github.com> Date: Fri, 14 Jun 2024 14:09:59 +0200 Subject: [PATCH] fix: flaky test `should not prevent network requests to basic functionality endpoints when the basica functionality toggle is on` (#25316) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## **Description** The test fails because it expects a request to the autodetect token endpoint but this doesn't happen before the assertion is made. If we look at the artifacts we can see how after the network switch, MM is still loading. ci: https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/87560/workflows/e7f9a8fd-c4e5-4f08-a8b2-9be559d3ad4c/jobs/3208109/tests#failed-test-0 ![Screenshot from 2024-06-14 13-08-27](https://github.com/MetaMask/metamask-extension/assets/54408225/394c8fe1-5110-4faa-8d35-425a3313b034) ![image](https://github.com/MetaMask/metamask-extension/assets/54408225/a38f1e94-db2f-4f98-9c2b-f2c59585fece) I see that there is a delay after switching networks. Possibly the flakiness was already found at the moment the test was written and this delay intended to mitigate it, but the result of this is not deterministic, since the delay is not enough sometimes and makes the test flaky `await driver.delay(tinyDelayMs);` The fix is to wait deterministically until the network switch has happened. As an addition, to add more certainty, we click on refresh tokens, and we make the assertion of the requests after that. With these 2 actions we increase the certainty that the request will be made before the assertion, however is not 100% deterministic, as we cannot know for sure when the API request is made in the background. [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/MetaMask/metamask-extension/pull/25316?quickstart=1) ## **Related issues** Fixes: ## **Manual testing steps** 1. Go to this page... 2. 3. ## **Screenshots/Recordings** ### **Before** ### **After** ## **Pre-merge author checklist** - [ ] I've followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Extension Coding Standards](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/CODING_GUIDELINES.md). - [ ] I've completed the PR template to the best of my ability - [ ] I’ve included tests if applicable - [ ] I’ve documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [ ] I’ve applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots. --- test/e2e/tests/privacy/basic-functionality.spec.js | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/test/e2e/tests/privacy/basic-functionality.spec.js b/test/e2e/tests/privacy/basic-functionality.spec.js index d20aa0542285..b0f9b54e9928 100644 --- a/test/e2e/tests/privacy/basic-functionality.spec.js +++ b/test/e2e/tests/privacy/basic-functionality.spec.js @@ -66,6 +66,10 @@ describe('MetaMask onboarding @no-mmi', function () { await driver.clickElement({ text: 'Ethereum Mainnet', tag: 'p' }); await driver.delay(tinyDelayMs); + // Wait until network is fully switched and refresh tokens before asserting to mitigate flakiness + await driver.assertElementNotPresent('.loading-overlay'); + await driver.clickElement('[data-testid="refresh-list-button"]'); + for (let i = 0; i < mockedEndpoints.length; i += 1) { const requests = await mockedEndpoints[i].getSeenRequests(); @@ -104,9 +108,12 @@ describe('MetaMask onboarding @no-mmi', function () { await driver.clickElement({ text: 'Ethereum Mainnet', tag: 'p' }); await driver.delay(tinyDelayMs); + // Wait until network is fully switched and refresh tokens before asserting to mitigate flakiness + await driver.assertElementNotPresent('.loading-overlay'); + await driver.clickElement('[data-testid="refresh-list-button"]'); + for (let i = 0; i < mockedEndpoints.length; i += 1) { const requests = await mockedEndpoints[i].getSeenRequests(); - assert.equal( requests.length, 1,