[refine](rf) Adjust the calculation logic for rf's always_true. #41477
+101
−61
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
TPC-H: Total hot run time: 40965 ms |
|
TeamCity be ut coverage result: |
TPC-DS: Total hot run time: 192403 ms |
ClickBench: Total hot run time: 33.21 s |
HappenLee
reviewed
Sep 30, 2024
| @@ -72,34 +72,58 @@ class VRuntimeFilterWrapper final : public VExpr { | |||
| _always_true_counter = always_true_counter; | |||
| } | |||
|
|
|||
| template <typename T, typename TT> | |||
| void update_counters(int64_t filter_rows, int64_t input_rows) { | |||
| if (_expr_filtered_rows_counter) { | |||
There was a problem hiding this comment.
what cast the counter is nullptr
Mryange
force-pushed
the
rf-always_true_counter
branch
from
3e49928
to
d86c1b5
Compare
|
run buildall |
TPC-H: Total hot run time: 41471 ms |
|
TeamCity be ut coverage result: |
TPC-DS: Total hot run time: 193117 ms |
ClickBench: Total hot run time: 32.74 s |
|
run external |
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
|
run buildall |
|
TeamCity be ut coverage result: |
TPC-H: Total hot run time: 41103 ms |
TPC-DS: Total hot run time: 191891 ms |
ClickBench: Total hot run time: 32.82 s |
|
run buildall |
|
TeamCity be ut coverage result: |
TPC-H: Total hot run time: 42633 ms |
TPC-DS: Total hot run time: 192631 ms |
|
run p0 |
ClickBench: Total hot run time: 33.2 s |
|
run external |
Mryange
force-pushed
the
rf-always_true_counter
branch
from
d86c1b5
to
6caf134
Compare
|
run buildall |
github-actions
bot
removed
the
approved
|
clang-tidy review says "All clean, LGTM! 👍" |
|
TeamCity be ut coverage result: |
|
run p0 |
|
run buildall |
|
TeamCity be ut coverage result: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Adjust the calculation logic for rf's always_true.
The previous approach was to sample one block and determine whether the always_true logic
would be applied for the subsequent 64 blocks.
The new approach divides the execution into independent periods of 64 blocks each.
At the beginning of each period, the filter rate is accumulated. If always_true is determined,
the remaining blocks in that period will follow the always_true logic.