client/retry: only return the latest error in backoffer #8227

JmPotato · 2024-05-30T06:42:32Z

What problem does this PR solve?

Issue Number: ref #8142.

What is changed and how does it work?

Due to the return of historical errors causing the client's retry logic to fail,
and since we currently do not need to obtain all errors during retries, this PR
removes `multierr` from backoffer and add tests to ensure the correctness of the retry logic.

Check List

Tests

Unit test
Integration test

Release note

Fix the bug where HTTP client retry logic is ineffective.

ti-chi-bot · 2024-05-30T06:42:34Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

HuSharp
rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

okJiang · 2024-05-30T09:37:01Z

client/retry/backoff.go

-// SetRetryableChecker sets the retryable checker.
-func (bo *Backoffer) SetRetryableChecker(checker func(err error) bool) {
+// SetRetryableChecker sets the retryable checker, `overwrite` flag is used to indicate whether to overwrite the existing checker.
+func (bo *Backoffer) SetRetryableChecker(checker func(err error) bool, overwrite bool) {


Isn't overwrite over-designed? Because now there is only one call

The SetRetryableChecker function might be invoked by higher-level callers such as BR, Lightning, or TiDB. I added this flag to ensure the caller-set checker is not overwritten unexpectedly.

client/retry/backoff.go

Signed-off-by: JmPotato <[email protected]>

codecov · 2024-05-30T12:10:09Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.38%. Comparing base (632cda4) to head (cbbf78b).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8227      +/-   ##
==========================================
+ Coverage   77.32%   77.38%   +0.06%     
==========================================
  Files         470      471       +1     
  Lines       61340    61346       +6     
==========================================
+ Hits        47431    47475      +44     
+ Misses      10331    10303      -28     
+ Partials     3578     3568      -10

Flag	Coverage Δ
unittests	`77.38% <100.00%> (+0.06%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

ti-chi-bot · 2024-05-31T05:26:53Z

@okJiang: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

JmPotato · 2024-05-31T07:57:27Z

/merge

ti-chi-bot · 2024-05-31T07:57:29Z

@JmPotato: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2024-05-31T07:57:31Z

This pull request has been accepted and is ready to merge.

Commit hash: 5126a32

ref tikv#8142 Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot · 2024-07-31T08:17:44Z

In response to a cherrypick label: new pull request created to branch release-8.1: #8465.

Signed-off-by: JmPotato <[email protected]>

ref #8142, close #8499 Due to the return of historical errors causing the client's retry logic to fail, and since we currently do not need to obtain all errors during retries, this PR removes `multierr` from backoffer and add tests to ensure the correctness of the retry logic. Signed-off-by: JmPotato <[email protected]> Co-authored-by: JmPotato <[email protected]>

JmPotato added the component/client Client logic. label May 30, 2024

JmPotato requested review from rleungx and HuSharp May 30, 2024 06:42

ti-chi-bot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 30, 2024

JmPotato force-pushed the remove_multierr branch from 980dabc to 3fc87d0 Compare May 30, 2024 06:44

JmPotato requested a review from nolouch May 30, 2024 06:50

JmPotato force-pushed the remove_multierr branch from 3fc87d0 to d156d59 Compare May 30, 2024 06:58

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 30, 2024

JmPotato force-pushed the remove_multierr branch 2 times, most recently from 612870a to ce72132 Compare May 30, 2024 07:21

HuSharp approved these changes May 30, 2024

View reviewed changes

ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 30, 2024

okJiang reviewed May 30, 2024

View reviewed changes

JmPotato added 2 commits May 30, 2024 20:00

Only return the latest error in backoffer

4667cb7

Signed-off-by: JmPotato <[email protected]>

Remove defaultRetryableChecker

5126a32

Signed-off-by: JmPotato <[email protected]>

JmPotato force-pushed the remove_multierr branch from ce72132 to 5126a32 Compare May 30, 2024 12:00

JmPotato requested a review from okJiang May 30, 2024 12:04

okJiang approved these changes May 31, 2024

View reviewed changes

rleungx approved these changes May 31, 2024

View reviewed changes

ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 31, 2024

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label May 31, 2024

Merge branch 'master' into remove_multierr

cbbf78b

ti-chi-bot bot merged commit 199b017 into tikv:master May 31, 2024
15 checks passed

JmPotato deleted the remove_multierr branch May 31, 2024 08:30

JmPotato mentioned this pull request May 31, 2024

go.mod: update PD client to fix the backoffer retry pingcap/tidb#53718

Merged

2 tasks

ti-chi-bot mentioned this pull request May 31, 2024

go.mod: update PD client to fix the backoffer retry (#53718) pingcap/tidb#53722

Merged

2 tasks

JmPotato mentioned this pull request Jun 3, 2024

Update the PD client to adopt the latest fix/optimization tikv/client-go#1360

Merged

ti-chi-bot added the needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. label Jul 31, 2024

ti-chi-bot pushed a commit to ti-chi-bot/pd that referenced this pull request Jul 31, 2024

This is an automated cherry-pick of tikv#8227

a2b8e77

ref tikv#8142 Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Jul 31, 2024

client/retry: only return the latest error in backoffer (#8227) #8465

Merged

JmPotato added a commit to ti-chi-bot/pd that referenced this pull request Aug 6, 2024

This is an automated cherry-pick of tikv#8227

be99a3a

Signed-off-by: JmPotato <[email protected]>

JmPotato added a commit to ti-chi-bot/pd that referenced this pull request Aug 6, 2024

This is an automated cherry-pick of tikv#8227

5bbab09

Signed-off-by: JmPotato <[email protected]>

JmPotato added a commit to ti-chi-bot/pd that referenced this pull request Aug 6, 2024

This is an automated cherry-pick of tikv#8227

7da1420

Signed-off-by: JmPotato <[email protected]>

JmPotato added a commit to ti-chi-bot/pd that referenced this pull request Aug 6, 2024

This is an automated cherry-pick of tikv#8227

a0cc33b

Signed-off-by: JmPotato <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

client/retry: only return the latest error in backoffer #8227

client/retry: only return the latest error in backoffer #8227

JmPotato commented May 30, 2024 •

edited

Loading

ti-chi-bot bot commented May 30, 2024 •

edited

Loading

okJiang May 30, 2024 •

edited

Loading

JmPotato May 30, 2024

codecov bot commented May 30, 2024 •

edited

Loading

ti-chi-bot bot commented May 31, 2024

JmPotato commented May 31, 2024

ti-chi-bot bot commented May 31, 2024

ti-chi-bot bot commented May 31, 2024

ti-chi-bot commented Jul 31, 2024

client/retry: only return the latest error in backoffer #8227

client/retry: only return the latest error in backoffer #8227

Conversation

JmPotato commented May 30, 2024 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

ti-chi-bot bot commented May 30, 2024 • edited Loading

okJiang May 30, 2024 • edited Loading

Choose a reason for hiding this comment

JmPotato May 30, 2024

Choose a reason for hiding this comment

codecov bot commented May 30, 2024 • edited Loading

Codecov Report

ti-chi-bot bot commented May 31, 2024

JmPotato commented May 31, 2024

ti-chi-bot bot commented May 31, 2024

ti-chi-bot bot commented May 31, 2024

ti-chi-bot commented Jul 31, 2024

JmPotato commented May 30, 2024 •

edited

Loading

ti-chi-bot bot commented May 30, 2024 •

edited

Loading

okJiang May 30, 2024 •

edited

Loading

codecov bot commented May 30, 2024 •

edited

Loading