Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: flaky test should not prevent network requests to basic functionality endpoints when the basica functionality toggle is on #25316

Merged
merged 3 commits into from
Jun 14, 2024

Conversation

seaona
Copy link
Contributor

@seaona seaona commented Jun 14, 2024

Description

The test fails because it expects a request to the autodetect token endpoint but this doesn't happen before the assertion is made.
If we look at the artifacts we can see how after the network switch, MM is still loading.

ci: https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/87560/workflows/e7f9a8fd-c4e5-4f08-a8b2-9be559d3ad4c/jobs/3208109/tests#failed-test-0

Screenshot from 2024-06-14 13-08-27

image

I see that there is a delay after switching networks. Possibly the flakiness was already found at the moment the test was written and this delay intended to mitigate it, but the result of this is not deterministic, since the delay is not enough sometimes and makes the test flaky

await driver.delay(tinyDelayMs);

The fix is to wait deterministically until the network switch has happened.
As an addition, to add more certainty, we click on refresh tokens, and we make the assertion of the requests after that.

With these 2 actions we increase the certainty that the request will be made before the assertion, however is not 100% deterministic, as we cannot know for sure when the API request is made in the background.

Open in GitHub Codespaces

Related issues

Fixes: #25305

Manual testing steps

  1. Go to this page...

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

Copy link

codecov bot commented Jun 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 65.67%. Comparing base (95b10da) to head (eb5b82c).
Report is 3 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #25316   +/-   ##
========================================
  Coverage    65.67%   65.67%           
========================================
  Files         1376     1376           
  Lines        54546    54546           
  Branches     14298    14298           
========================================
  Hits         35819    35819           
  Misses       18727    18727           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@seaona seaona marked this pull request as ready for review June 14, 2024 12:03
@seaona seaona requested a review from a team as a code owner June 14, 2024 12:03
@seaona seaona self-assigned this Jun 14, 2024
@DDDDDanica DDDDDanica merged commit 191ab10 into develop Jun 14, 2024
83 of 84 checks passed
@DDDDDanica DDDDDanica deleted the flaky-network-requests branch June 14, 2024 12:10
@github-actions github-actions bot locked and limited conversation to collaborators Jun 14, 2024
@metamaskbot
Copy link
Collaborator

Builds ready [eb5b82c]
Page Load Metrics (59 ± 17 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint69228913316
domContentLoaded8371263
load40205593617
domInteractive8371263
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

@metamaskbot metamaskbot added the release-12.1.0 Issue or pull request that will be included in release 12.1.0 label Jun 14, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
flaky tests release-12.1.0 Issue or pull request that will be included in release 12.1.0 team-extension-platform
Projects
None yet
4 participants