-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[opt](coord) use single instance only with small limit #33888
Conversation
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
run buildall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
TPC-H: Total hot run time: 38386 ms
|
TPC-DS: Total hot run time: 184048 ms
|
ClickBench: Total hot run time: 29.97 s
|
run buildall |
TPC-H: Total hot run time: 38498 ms
|
TPC-DS: Total hot run time: 184285 ms
|
ClickBench: Total hot run time: 30.07 s
|
PR approved by at least one committer and no changes requested. |
… limit involved (#39927) For queries like `select * from tbl limit 100;`, we should limit the parallelism to 1. We already have strategies in our code, but we can not control its behavior. So add session variables to control them. Previous pr like #33888 #36535 already has done something similar, this pr integrates them.
… limit involved (apache#39927) For queries like `select * from tbl limit 100;`, we should limit the parallelism to 1. We already have strategies in our code, but we can not control its behavior. So add session variables to control them. Previous pr like apache#33888 apache#36535 already has done something similar, this pr integrates them.
Proposed changes
Followup #31342
When executing simple query like
select * from tbl limit 1
.Doris will use only one instance for each scan node, to reduce the fan-out of simple queries.
But if the limit number is large, eg, 1 billion, this logic is not suitable.
So I add a new session variable
limit_rows_for_single_instance
, default is 10000.Only if the limit number is lower than this value, Doris will use one instance for scan node.
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...