Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](inverted index) add inlist condition handling to compound #34375

Merged
merged 1 commit into from
May 1, 2024
Merged

[opt](inverted index) add inlist condition handling to compound #34375

merged 1 commit into from
May 1, 2024

Conversation

zzzxl1993
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zzzxl1993
Copy link
Contributor Author

run buildall

github-actions[bot]
github-actions bot reviewed May 1, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/olap/rowset/segment_v2/segment_iterator.h
return ss.str();
}

bool is_empty() const { return column_name.empty() && query_value.empty() && query_op.empty(); }
bool is_empty() const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'is_empty' should be marked [[nodiscard]] [modernize-use-nodiscard]

Suggested change
bool is_empty() const {
[[nodiscard]] bool is_empty() const {

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.79% (8069/21350)
Line Coverage: 29.45% (65899/223797)
Region Coverage: 28.91% (33927/117339)
Branch Coverage: 24.78% (17415/70282)
Coverage Report: http://coverage.selectdb-in.cc/coverage/11298c60fa7e70995ea2bafef16ad2fe0eda8eaf_11298c60fa7e70995ea2bafef16ad2fe0eda8eaf/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49735 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 11298c60fa7e70995ea2bafef16ad2fe0eda8eaf, data reload: false

------ Round 1 ----------------------------------
q1	17662	4454	4401	4401
q2	2034	151	142	142
q3	10499	1884	1931	1884
q4	10363	1276	1325	1276
q5	8445	3892	3878	3878
q6	234	122	123	122
q7	2007	1594	1602	1594
q8	9275	2731	2731	2731
q9	10725	10367	10300	10300
q10	8601	3530	3495	3495
q11	419	253	248	248
q12	475	301	307	301
q13	18343	3927	4018	3927
q14	356	320	314	314
q15	519	449	454	449
q16	667	580	577	577
q17	1134	980	971	971
q18	7255	6744	6863	6744
q19	1716	1592	1591	1591
q20	548	317	295	295
q21	4470	4169	4115	4115
q22	488	380	385	380
Total cold run time: 116235 ms
Total hot run time: 49735 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4365	4347	4328	4328
q2	320	222	228	222
q3	4147	4139	4138	4138
q4	2747	2757	2748	2748
q5	7208	7146	7143	7143
q6	237	116	119	116
q7	3246	2859	2800	2800
q8	4401	4469	4461	4461
q9	17001	16699	16718	16699
q10	4253	4278	4260	4260
q11	752	726	716	716
q12	1047	846	845	845
q13	7644	3740	3747	3740
q14	468	424	428	424
q15	496	469	454	454
q16	736	689	684	684
q17	3856	3879	3872	3872
q18	8698	8801	8809	8801
q19	1708	1727	1683	1683
q20	2382	2165	2106	2106
q21	8474	8452	8507	8452
q22	1000	956	947	947
Total cold run time: 85186 ms
Total hot run time: 79639 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204039 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 11298c60fa7e70995ea2bafef16ad2fe0eda8eaf, data reload: false

query1	913	387	380	380
query2	6522	2753	2700	2700
query3	6915	201	194	194
query4	20036	18229	18050	18050
query5	19714	6547	6472	6472
query6	284	217	228	217
query7	4154	291	299	291
query8	261	265	232	232
query9	3145	2695	2631	2631
query10	420	281	295	281
query11	11329	10723	10694	10694
query12	124	69	72	69
query13	5572	715	670	670
query14	17767	14001	13708	13708
query15	398	222	228	222
query16	6464	268	253	253
query17	1776	1455	870	870
query18	2316	412	410	410
query19	207	149	145	145
query20	71	75	76	75
query21	193	98	99	98
query22	5266	4882	4949	4882
query23	32607	31866	31996	31866
query24	6873	6529	6515	6515
query25	536	414	423	414
query26	528	166	161	161
query27	1881	296	294	294
query28	6131	2420	2349	2349
query29	2998	2798	2853	2798
query30	245	159	160	159
query31	943	743	741	741
query32	70	62	56	56
query33	398	240	248	240
query34	848	468	489	468
query35	1119	940	826	826
query36	1397	1261	1236	1236
query37	97	62	59	59
query38	3101	2921	2944	2921
query39	1380	1319	1332	1319
query40	211	91	98	91
query41	37	35	34	34
query42	95	79	77	77
query43	771	665	735	665
query44	1123	746	733	733
query45	240	230	229	229
query46	1226	954	920	920
query47	1899	1716	1698	1698
query48	997	718	720	718
query49	623	374	371	371
query50	868	591	618	591
query51	4837	4679	4680	4679
query52	88	81	82	81
query53	445	319	312	312
query54	2659	2456	2462	2456
query55	88	82	79	79
query56	206	206	188	188
query57	1182	1115	1127	1115
query58	218	186	195	186
query59	4245	4060	4012	4012
query60	206	191	212	191
query61	88	83	84	83
query62	798	494	466	466
query63	470	339	331	331
query64	2561	1498	1474	1474
query65	3629	3554	3588	3554
query66	768	377	383	377
query67	16326	15687	16459	15687
query68	10382	686	682	682
query69	578	362	361	361
query70	2009	1330	1386	1330
query71	420	305	304	304
query72	6429	3447	3439	3439
query73	740	331	315	315
query74	6388	5828	5916	5828
query75	5433	3710	3728	3710
query76	6702	1167	1217	1167
query77	1171	255	257	255
query78	13097	11741	11783	11741
query79	8067	657	639	639
query80	721	400	387	387
query81	453	230	235	230
query82	670	98	93	93
query83	166	142	131	131
query84	255	68	69	68
query85	849	308	301	301
query86	327	319	290	290
query87	3254	2992	3019	2992
query88	4200	2319	2322	2319
query89	437	284	297	284
query90	1953	189	193	189
query91	163	134	134	134
query92	56	51	53	51
query93	6278	533	575	533
query94	653	202	199	199
query95	1129	1081	1071	1071
query96	642	324	321	321
query97	6457	6276	6336	6276
query98	199	179	169	169
query99	3098	936	866	866
Total cold run time: 315279 ms
Total hot run time: 204039 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.44 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 11298c60fa7e70995ea2bafef16ad2fe0eda8eaf, data reload: false

query1	0.02	0.02	0.01
query2	0.06	0.03	0.02
query3	0.24	0.05	0.04
query4	1.80	0.07	0.06
query5	0.53	0.52	0.52
query6	1.23	0.62	0.61
query7	0.01	0.01	0.01
query8	0.02	0.02	0.02
query9	0.52	0.48	0.47
query10	0.53	0.54	0.52
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.62	0.62	0.61
query14	0.80	0.78	0.81
query15	0.79	0.75	0.78
query16	0.38	0.39	0.36
query17	1.00	0.97	0.99
query18	0.23	0.26	0.25
query19	1.85	1.85	1.88
query20	0.02	0.00	0.00
query21	15.46	0.55	0.55
query22	1.84	2.45	1.55
query23	16.98	1.08	0.80
query24	4.27	3.33	0.91
query25	0.36	0.13	0.06
query26	0.68	0.16	0.16
query27	0.04	0.05	0.04
query28	6.67	0.73	0.70
query29	12.75	2.37	2.40
query30	0.60	0.53	0.51
query31	2.82	0.40	0.37
query32	3.38	0.49	0.49
query33	3.07	3.06	3.09
query34	15.29	4.79	4.79
query35	4.88	4.82	4.84
query36	1.09	1.01	1.00
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.13	0.13
query41	0.07	0.01	0.01
query42	0.02	0.02	0.01
query43	0.02	0.01	0.02
Total cold run time: 101.46 s
Total hot run time: 30.44 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 11298c60fa7e70995ea2bafef16ad2fe0eda8eaf with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@xiaokang xiaokang merged commit f31458e into apache:branch-2.0 May 1, 2024
24 of 26 checks passed
@xiaokang
Copy link
Contributor

xiaokang commented May 1, 2024

#34134

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@zzzxl1993
[opt](inverted index) add inlist condition handling to compound apach…
397043b 
…e#34134 (apache#34375)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@zzzxl1993 @doris-robot @xiaokang