Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry flaky tests #449

Merged
merged 2 commits into from
Jan 25, 2024
Merged

Conversation

owaiskazi19
Copy link
Member

Description

Retry flaky tests using org.gradle.test-retry plugin.

Issues Resolved

Closes #442

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions bot added the backport 2.x backport PRs to 2.x branch label Jan 24, 2024
Signed-off-by: Owais Kazi <[email protected]>
Copy link

codecov bot commented Jan 24, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (b84e9aa) 71.88% compared to head (5843aec) 71.88%.

Additional details and impacted files
@@            Coverage Diff            @@
##               main     #449   +/-   ##
=========================================
  Coverage     71.88%   71.88%           
  Complexity      620      620           
=========================================
  Files            78       78           
  Lines          3126     3126           
  Branches        236      236           
=========================================
  Hits           2247     2247           
  Misses          772      772           
  Partials        107      107           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@dbwiddis dbwiddis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool app.

We do not need to apply it to all tests. Just IT. And let’s limit to one retry.

CHANGELOG.md Outdated Show resolved Hide resolved
build.gradle Outdated Show resolved Hide resolved
@owaiskazi19 owaiskazi19 marked this pull request as ready for review January 24, 2024 23:20
@owaiskazi19 owaiskazi19 changed the title [DRAFT] Retry flaky tests Retry flaky tests Jan 24, 2024
@owaiskazi19 owaiskazi19 added backport 2.x backport PRs to 2.x branch and removed backport 2.x backport PRs to 2.x branch labels Jan 24, 2024
Copy link
Member

@joshpalis joshpalis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a small comment, but looks got to me. I'm still not sure why we're encountering these multi-node failures on JDK 21.0.2 but not JDK 21.0.1

Copy link
Member

@joshpalis joshpalis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a small comment, but looks got to me. I'm still not sure why we're encountering these multi-node failures on JDK 21.0.2 but not JDK 21.0.1

build.gradle Show resolved Hide resolved
@amitgalitz
Copy link
Member

looks good to me, but I agree with Josh that we need another issue to solve these failures, the multi node flakiness is still a little concerning

Signed-off-by: Owais Kazi <[email protected]>
@owaiskazi19 owaiskazi19 merged commit 03cdb99 into opensearch-project:main Jan 25, 2024
26 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 25, 2024
* Retry flaky tests

Signed-off-by: Owais Kazi <[email protected]>

* Addressed PR Comments

Signed-off-by: Owais Kazi <[email protected]>

---------

Signed-off-by: Owais Kazi <[email protected]>
(cherry picked from commit 03cdb99)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
owaiskazi19 pushed a commit that referenced this pull request Jan 25, 2024
Retry flaky tests (#449)

* Retry flaky tests



* Addressed PR Comments



---------


(cherry picked from commit 03cdb99)

Signed-off-by: Owais Kazi <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x backport PRs to 2.x branch skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] macOS Integ Test Flakiness
4 participants