Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: flaky test Swap-Send ETH to non-contract address with data that matches swap data signature submits a transaction successfully #25545

Merged
merged 2 commits into from
Jun 27, 2024

Conversation

seaona
Copy link
Contributor

@seaona seaona commented Jun 27, 2024

Description

The problem is that we are looking for a pending transaction and then a confirmed transaction. If the transaction is confirmed very fast, the pending transaction is never found, thus making the test fail.
This is a bad pattern that introduces flakiness, as we are trying to find an element by its transient state, which is variable. What we really want to assert is that the transaction is in the end confirmed, so we shouldn't try to look for the pending transaction (something that disappears fast) in the first place.

Error:

TimeoutError: Waiting for element to be located By(css selector, .transaction-status-label--pending)
Wait timed out after 10081ms

ci error

Open in GitHub Codespaces

Related issues

Fixes: #25546

Manual testing steps

  1. Check ci

Screenshots/Recordings

See the screenshot in ci, how the expected transaction is indeed there (confirmed). However, on the error we couldn't locate the element when it was pending (possibly due to being really fast at occasions)

image

Screenshot from 2024-06-27 09-51-55

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@seaona seaona requested a review from a team as a code owner June 27, 2024 07:52
Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@seaona seaona self-assigned this Jun 27, 2024
@metamaskbot metamaskbot added the team-confirmations Push issues to confirmations team label Jun 27, 2024
@seaona seaona removed the team-confirmations Push issues to confirmations team label Jun 27, 2024
'Pending',
'-1 ETH',
'',
);
await swapSendPage.verifyHistoryEntry(
'Send ETH as TST',
'Confirmed',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are asserting that the expected tx is there and confirmed, which is what we really want. If we wanted to test transaction states (from unapproved, to pending to confirmed) this would need to be done in a separate test, and have a different approach (lowering the gas, to make sure that the pending tx stays there until speed up/cancel, for example)

Copy link

codecov bot commented Jun 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.68%. Comparing base (f82a04a) to head (d2f8b4c).
Report is 3 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #25545   +/-   ##
========================================
  Coverage    69.68%   69.68%           
========================================
  Files         1349     1349           
  Lines        47852    47852           
  Branches     13195    13195           
========================================
  Hits         33342    33342           
  Misses       14510    14510           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@metamaskbot
Copy link
Collaborator

Builds ready [d2f8b4c]
Page Load Metrics (240 ± 277 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint6613585157
domContentLoaded9441273
load402169240577277
domInteractive9431273
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

Copy link
Contributor

@chloeYue chloeYue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@hjetpoluru hjetpoluru self-requested a review June 27, 2024 16:47
Copy link
Contributor

@hjetpoluru hjetpoluru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hjetpoluru hjetpoluru merged commit 1d33d6f into develop Jun 27, 2024
70 checks passed
@hjetpoluru hjetpoluru deleted the flaky-swap-send-pending branch June 27, 2024 16:48
@github-actions github-actions bot locked and limited conversation to collaborators Jun 27, 2024
@metamaskbot metamaskbot added the release-12.1.0 Issue or pull request that will be included in release 12.1.0 label Jun 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
flaky tests release-12.1.0 Issue or pull request that will be included in release 12.1.0 team-extension-platform
Projects
Archived in project
4 participants