Skip to content

Commit

Permalink
fix: flaky test `should not prevent network requests to basic functio…
Browse files Browse the repository at this point in the history
…nality endpoints when the basica functionality toggle is on` (#25316)

<!--
Please submit this PR as a draft initially.
Do not mark it as "Ready for review" until the template has been
completely filled out, and PR status checks have passed at least once.
-->

## **Description**

The test fails because it expects a request to the autodetect token
endpoint but this doesn't happen before the assertion is made.
If we look at the artifacts we can see how after the network switch, MM
is still loading.

ci:
https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/87560/workflows/e7f9a8fd-c4e5-4f08-a8b2-9be559d3ad4c/jobs/3208109/tests#failed-test-0

![Screenshot from 2024-06-14
13-08-27](https://github.com/MetaMask/metamask-extension/assets/54408225/394c8fe1-5110-4faa-8d35-425a3313b034)


![image](https://github.com/MetaMask/metamask-extension/assets/54408225/a38f1e94-db2f-4f98-9c2b-f2c59585fece)

I see that there is a delay after switching networks. Possibly the
flakiness was already found at the moment the test was written and this
delay intended to mitigate it, but the result of this is not
deterministic, since the delay is not enough sometimes and makes the
test flaky

`await driver.delay(tinyDelayMs);`

The fix is to wait deterministically until the network switch has
happened.
As an addition, to add more certainty, we click on refresh tokens, and
we make the assertion of the requests after that.

With these 2 actions we increase the certainty that the request will be
made before the assertion, however is not 100% deterministic, as we
cannot know for sure when the API request is made in the background.


[![Open in GitHub
Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/MetaMask/metamask-extension/pull/25316?quickstart=1)

## **Related issues**

Fixes:

## **Manual testing steps**

1. Go to this page...
2.
3.

## **Screenshots/Recordings**

<!-- If applicable, add screenshots and/or recordings to visualize the
before and after of your change. -->

### **Before**

<!-- [screenshots/recordings] -->

### **After**

<!-- [screenshots/recordings] -->

## **Pre-merge author checklist**

- [ ] I've followed [MetaMask Contributor
Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask
Extension Coding
Standards](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/CODING_GUIDELINES.md).
- [ ] I've completed the PR template to the best of my ability
- [ ] I’ve included tests if applicable
- [ ] I’ve documented my code using [JSDoc](https://jsdoc.app/) format
if applicable
- [ ] I’ve applied the right labels on the PR (see [labeling
guidelines](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/LABELING_GUIDELINES.md)).
Not required for external contributors.

## **Pre-merge reviewer checklist**

- [ ] I've manually tested the PR (e.g. pull and build branch, run the
app, test code being changed).
- [ ] I confirm that this PR addresses all acceptance criteria described
in the ticket it closes and includes the necessary testing evidence such
as recordings and or screenshots.
  • Loading branch information
seaona authored Jun 14, 2024
1 parent 0d32814 commit 191ab10
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion test/e2e/tests/privacy/basic-functionality.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ describe('MetaMask onboarding @no-mmi', function () {
await driver.clickElement({ text: 'Ethereum Mainnet', tag: 'p' });
await driver.delay(tinyDelayMs);

// Wait until network is fully switched and refresh tokens before asserting to mitigate flakiness
await driver.assertElementNotPresent('.loading-overlay');
await driver.clickElement('[data-testid="refresh-list-button"]');

for (let i = 0; i < mockedEndpoints.length; i += 1) {
const requests = await mockedEndpoints[i].getSeenRequests();

Expand Down Expand Up @@ -104,9 +108,12 @@ describe('MetaMask onboarding @no-mmi', function () {
await driver.clickElement({ text: 'Ethereum Mainnet', tag: 'p' });
await driver.delay(tinyDelayMs);

// Wait until network is fully switched and refresh tokens before asserting to mitigate flakiness
await driver.assertElementNotPresent('.loading-overlay');
await driver.clickElement('[data-testid="refresh-list-button"]');

for (let i = 0; i < mockedEndpoints.length; i += 1) {
const requests = await mockedEndpoints[i].getSeenRequests();

assert.equal(
requests.length,
1,
Expand Down

0 comments on commit 191ab10

Please sign in to comment.