Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: flaky test vault-decrypt by removing driver navigate to open a new window #25443

Merged
merged 3 commits into from
Jun 21, 2024

Conversation

seaona
Copy link
Contributor

@seaona seaona commented Jun 20, 2024

Description

The vault decryption test opens 2 MetaMask windows whenever we start the test, causing flakiness in which window is currently selected and where the actions are performed.

Having 2 MM windows, together with the fact that the Chrome active window can sometimes not be the same as the Webdriver active window (see @HowardBraham 's work on that area), causes unpredictable effects - see video below.

For fixing it, we now remove the step of driver.navigate which will open a MM window, and caused to have 2 windows instead of 1.

However, this lead to a new problem. Lavamoat presents a race condition, which now is more present making the test fail in almost all runs. From @naugtur :

It is indeed a race condition, because scuttling can only destroy the global references after they've been successfully copied and used for establishing the root compartment of the application.

For fixing this, we add Proxy in the exceptions, following Lavamoat's team suggestion as a temporary solution.

mm-vault-decrypt-error.mp4

Open in GitHub Codespaces

Related issues

Fixes:

Manual testing steps

See successful run here

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@metamaskbot metamaskbot added the team-confirmations Push issues to confirmations team label Jun 20, 2024
.circleci/config.yml Outdated Show resolved Hide resolved
@seaona seaona self-assigned this Jun 21, 2024
@seaona seaona marked this pull request as ready for review June 21, 2024 10:22
@seaona seaona requested review from kumavis and a team as code owners June 21, 2024 10:22
Copy link
Member

@weizman weizman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

Copy link

codecov bot commented Jun 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 65.39%. Comparing base (2115516) to head (265d1e3).
Report is 22 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #25443   +/-   ##
========================================
  Coverage    65.39%   65.39%           
========================================
  Files         1382     1382           
  Lines        54764    54764           
  Branches     14369    14369           
========================================
  Hits         35809    35809           
  Misses       18955    18955           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@metamaskbot
Copy link
Collaborator

Builds ready [265d1e3]
Page Load Metrics (49 ± 3 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint64927873
domContentLoaded9131010
load40674973
domInteractive9131010
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

Copy link
Member

@Gudahtt Gudahtt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@seaona seaona merged commit 03b7dec into develop Jun 21, 2024
77 checks passed
@seaona seaona deleted the vault-decrypt-windows branch June 21, 2024 11:22
@github-actions github-actions bot locked and limited conversation to collaborators Jun 21, 2024
Copy link
Contributor

@naugtur naugtur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proxy exception is harmless.

@metamaskbot metamaskbot added the release-12.1.0 Issue or pull request that will be included in release 12.1.0 label Jun 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
flaky tests release-12.1.0 Issue or pull request that will be included in release 12.1.0 team-confirmations Push issues to confirmations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants