Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](inverted index)Change index_id from int32 to int64 to avoid overflow #35206

Merged
merged 1 commit into from
May 22, 2024
Merged

[fix](inverted index)Change index_id from int32 to int64 to avoid overflow #35206

merged 1 commit into from
May 22, 2024

Conversation

qidaye
Copy link
Contributor

@qidaye qidaye commented May 22, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@qidaye
Copy link
Contributor Author

qidaye commented May 22, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

dataroaring
dataroaring approved these changes May 22, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 22, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 40182 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3947005b8002bb8d31b1b211d7468c7e943aaa8d, data reload: false

------ Round 1 ----------------------------------
q1	18135	4482	4409	4409
q2	2643	197	202	197
q3	11280	1232	1180	1180
q4	10478	834	751	751
q5	7542	2709	2641	2641
q6	218	135	138	135
q7	967	609	609	609
q8	9455	2112	2088	2088
q9	8955	6501	6467	6467
q10	8893	3727	3713	3713
q11	439	251	243	243
q12	505	216	232	216
q13	17763	3027	3005	3005
q14	275	221	233	221
q15	514	474	472	472
q16	520	392	378	378
q17	962	720	710	710
q18	8185	7651	7409	7409
q19	6059	1567	1534	1534
q20	686	327	310	310
q21	4863	3218	3915	3218
q22	366	282	276	276
Total cold run time: 119703 ms
Total hot run time: 40182 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4355	4210	4211	4210
q2	382	267	265	265
q3	2975	2712	2730	2712
q4	1874	1602	1677	1602
q5	5236	5312	5290	5290
q6	207	124	127	124
q7	2188	1808	1768	1768
q8	3215	3325	3345	3325
q9	8351	8333	8339	8333
q10	3886	3630	3733	3630
q11	581	467	479	467
q12	752	584	624	584
q13	16365	3005	3026	3005
q14	291	270	270	270
q15	509	473	466	466
q16	472	440	405	405
q17	1784	1484	1481	1481
q18	7594	7619	7454	7454
q19	1651	1571	1567	1567
q20	1982	1822	1785	1785
q21	4942	4756	4756	4756
q22	556	504	511	504
Total cold run time: 70148 ms
Total hot run time: 54003 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170770 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3947005b8002bb8d31b1b211d7468c7e943aaa8d, data reload: false

query1	927	393	374	374
query2	6469	2451	2335	2335
query3	6655	205	211	205
query4	19370	17154	16937	16937
query5	4139	438	427	427
query6	248	175	155	155
query7	4582	302	287	287
query8	239	192	191	191
query9	8736	2427	2398	2398
query10	462	276	275	275
query11	10777	9986	9884	9884
query12	145	94	90	90
query13	1650	370	373	370
query14	9373	6841	7485	6841
query15	238	168	166	166
query16	8051	258	254	254
query17	1854	526	513	513
query18	2075	284	271	271
query19	213	158	154	154
query20	95	91	86	86
query21	202	133	129	129
query22	4033	3981	3803	3803
query23	33627	33160	32872	32872
query24	12209	2881	2818	2818
query25	646	347	363	347
query26	1828	161	163	161
query27	3085	324	321	321
query28	7545	2055	2055	2055
query29	1042	624	589	589
query30	311	168	175	168
query31	970	777	754	754
query32	93	53	53	53
query33	752	267	258	258
query34	1027	488	471	471
query35	733	596	596	596
query36	1049	913	905	905
query37	276	73	71	71
query38	2910	2757	2751	2751
query39	869	776	793	776
query40	280	125	126	125
query41	47	44	46	44
query42	105	96	97	96
query43	616	552	570	552
query44	1259	726	759	726
query45	177	194	156	156
query46	1070	719	724	719
query47	1837	1751	1768	1751
query48	368	296	296	296
query49	1184	378	397	378
query50	776	396	380	380
query51	6821	6920	6763	6763
query52	114	93	89	89
query53	349	279	294	279
query54	997	427	426	426
query55	74	77	71	71
query56	261	242	245	242
query57	1135	1065	1011	1011
query58	242	214	214	214
query59	3533	3374	3215	3215
query60	271	251	253	251
query61	89	88	89	88
query62	670	459	448	448
query63	303	277	278	277
query64	9773	2191	1791	1791
query65	3177	3080	3106	3080
query66	1374	345	379	345
query67	15240	14624	14625	14624
query68	4557	528	541	528
query69	440	263	268	263
query70	1189	1064	1037	1037
query71	398	268	281	268
query72	7664	5058	5266	5058
query73	713	321	316	316
query74	5953	5530	5529	5529
query75	3491	2721	2648	2648
query76	2699	969	1029	969
query77	429	265	265	265
query78	10287	9830	9956	9830
query79	2415	512	514	512
query80	837	438	470	438
query81	538	243	247	243
query82	1070	99	96	96
query83	278	169	165	165
query84	229	85	81	81
query85	1499	281	257	257
query86	466	325	310	310
query87	3287	3129	3107	3107
query88	4210	2316	2311	2311
query89	475	385	384	384
query90	1946	180	181	180
query91	124	98	96	96
query92	63	49	47	47
query93	1911	514	501	501
query94	1174	194	182	182
query95	399	308	303	303
query96	588	261	271	261
query97	3173	3071	3008	3008
query98	247	229	216	216
query99	1119	813	840	813
Total cold run time: 275515 ms
Total hot run time: 170770 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.66% (9013/25277)
Line Coverage: 27.30% (74510/272954)
Region Coverage: 26.53% (38564/145355)
Branch Coverage: 23.38% (19668/84116)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3947005b8002bb8d31b1b211d7468c7e943aaa8d_3947005b8002bb8d31b1b211d7468c7e943aaa8d/report/index.html

@doris-robot
Copy link

ClickBench: Total hot run time: 30.56 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3947005b8002bb8d31b1b211d7468c7e943aaa8d, data reload: false

query1	0.04	0.03	0.04
query2	0.09	0.04	0.04
query3	0.24	0.05	0.05
query4	1.66	0.08	0.07
query5	0.48	0.49	0.55
query6	1.13	0.72	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.52	0.48	0.49
query10	0.54	0.55	0.53
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.62
query14	0.76	0.77	0.79
query15	0.82	0.80	0.82
query16	0.37	0.36	0.37
query17	1.02	0.97	0.95
query18	0.20	0.27	0.21
query19	1.78	1.70	1.66
query20	0.02	0.01	0.01
query21	15.40	0.74	0.70
query22	4.04	7.02	1.98
query23	18.26	1.36	1.27
query24	1.36	0.42	0.22
query25	0.14	0.09	0.09
query26	0.27	0.18	0.16
query27	0.07	0.08	0.08
query28	13.40	1.02	0.99
query29	12.68	3.28	3.31
query30	0.25	0.06	0.06
query31	2.85	0.37	0.38
query32	3.28	0.47	0.47
query33	2.87	2.86	2.86
query34	17.06	4.42	4.44
query35	4.49	4.50	4.49
query36	0.65	0.46	0.45
query37	0.18	0.16	0.17
query38	0.17	0.15	0.15
query39	0.05	0.03	0.04
query40	0.16	0.15	0.15
query41	0.09	0.05	0.05
query42	0.07	0.06	0.05
query43	0.04	0.04	0.04
Total cold run time: 108.46 s
Total hot run time: 30.56 s

airborne12
airborne12 approved these changes May 22, 2024
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 119db75 into apache:master May 22, 2024
27 of 29 checks passed
@qidaye qidaye deleted the indexid_int32 branch May 23, 2024 02:21
yiguolei pushed a commit that referenced this pull request May 23, 2024
@qidaye @luennng @Doris-Extras
[fix](inverted index)Change index_id from int32 to int64 to avoid ove…
4b7608c 
…rflow (#35206)

Co-authored-by: Luennng <[email protected]>
dataroaring pushed a commit that referenced this pull request May 26, 2024
@qidaye @luennng @dataroaring
[fix](inverted index)Change index_id from int32 to int64 to avoid ove…
ca62ac6 
…rflow (#35206)

Co-authored-by: Luennng <[email protected]>
w41ter pushed a commit to w41ter/incubator-doris that referenced this pull request May 29, 2024
@qidaye @luennng @w41ter
[fix](inverted index)Change index_id from int32 to int64 to avoid ove…
4b2b6ca 
…rflow

Cherry-pick apache#35206

Co-authored-by: Luennng <[email protected]>
@w41ter w41ter mentioned this pull request May 29, 2024
Merged
w41ter pushed a commit to w41ter/incubator-doris that referenced this pull request May 29, 2024
@qidaye @luennng @w41ter
[fix](inverted index)Change index_id from int32 to int64 to avoid ove…
e769b9d 
…rflow

Cherry-pick apache#35206

Co-authored-by: Luennng <[email protected]>
xiaokang pushed a commit that referenced this pull request May 30, 2024
@w41ter @qidaye
[fix](inverted index)Change index_id from int32 to int64 to avoid ove…
8321a3f 
…rflow #35206 (#35573)

Co-authored-by: qiye <[email protected]>
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@w41ter @qidaye
[fix](inverted index)Change index_id from int32 to int64 to avoid ove…
7b3cb4a 
…rflow apache#35206 (apache#35573)

Co-authored-by: qiye <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@airborne12 airborne12 airborne12 approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.11-merged dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@qidaye @doris-robot @airborne12 @dataroaring @xiaokang @luennng