Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[branch-2.0](timezone) make TimeUtils formatter use correct time_zone (#37465) #37653

Merged
merged 1 commit into from
Aug 1, 2024

Conversation

zclllyybb
Copy link
Contributor

All timestamp/datetime parsing in Doris is controlled by the session variable time_zone.
Apply it also to interface of TimeUtils in FE.

pick #37465

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybb
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added area/load Issues or PRs related to all kinds of load area/nereids area/planner Issues or PRs related to the query planner labels Jul 11, 2024
@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49960 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 05cda028ebc79548aa017251d101471f20ae4113, data reload: false

------ Round 1 ----------------------------------
q1	18184	4393	4432	4393
q2	2075	160	152	152
q3	10263	1911	1906	1906
q4	10107	1257	1312	1257
q5	8670	3972	3875	3875
q6	226	126	127	126
q7	2021	1626	1631	1626
q8	9537	2759	2708	2708
q9	13655	10248	10391	10248
q10	8632	3605	3535	3535
q11	416	252	245	245
q12	470	311	302	302
q13	18363	3981	4077	3981
q14	355	335	319	319
q15	508	466	464	464
q16	691	575	582	575
q17	1144	958	917	917
q18	7287	6887	6975	6887
q19	1785	1714	1617	1617
q20	553	331	306	306
q21	4426	4209	4072	4072
q22	535	449	462	449
Total cold run time: 119903 ms
Total hot run time: 49960 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4405	4310	4356	4310
q2	326	229	222	222
q3	4191	4159	4176	4159
q4	2763	2745	2738	2738
q5	7164	7084	7155	7084
q6	239	119	123	119
q7	3230	2914	2882	2882
q8	4337	4442	4471	4442
q9	16851	16819	16784	16784
q10	4252	4247	4246	4246
q11	756	690	702	690
q12	1038	866	860	860
q13	6966	3745	3779	3745
q14	457	428	434	428
q15	516	466	459	459
q16	736	681	674	674
q17	3791	3862	3880	3862
q18	8721	8733	8830	8733
q19	1717	1611	1691	1611
q20	2350	2160	2108	2108
q21	8474	8594	8554	8554
q22	1064	992	1001	992
Total cold run time: 84344 ms
Total hot run time: 79702 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204518 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 05cda028ebc79548aa017251d101471f20ae4113, data reload: false

query1	913	416	382	382
query2	6559	3032	2630	2630
query3	6919	211	202	202
query4	20173	18005	18068	18005
query5	19722	6520	6568	6520
query6	303	220	242	220
query7	4181	303	305	303
query8	403	481	443	443
query9	3138	2694	2632	2632
query10	415	310	307	307
query11	11442	10735	10770	10735
query12	120	79	76	76
query13	5601	686	682	682
query14	17668	13305	13728	13305
query15	398	246	238	238
query16	6471	292	268	268
query17	1685	1453	903	903
query18	2291	419	419	419
query19	221	154	153	153
query20	82	74	85	74
query21	189	103	94	94
query22	5321	5225	5162	5162
query23	32647	31792	31886	31792
query24	6948	6538	6445	6445
query25	537	431	423	423
query26	544	161	160	160
query27	1892	296	305	296
query28	6276	2381	2365	2365
query29	2945	2827	2748	2748
query30	247	163	170	163
query31	924	750	749	749
query32	72	64	60	60
query33	408	263	267	263
query34	869	476	478	476
query35	1114	936	921	921
query36	1326	1169	1294	1169
query37	95	60	63	60
query38	3058	2941	2948	2941
query39	1388	1337	1341	1337
query40	212	99	97	97
query41	46	45	47	45
query42	84	85	80	80
query43	772	760	802	760
query44	1154	738	740	738
query45	247	237	231	231
query46	1221	966	960	960
query47	1970	1780	1791	1780
query48	1012	729	713	713
query49	619	389	369	369
query50	849	587	571	571
query51	4816	4685	4612	4612
query52	93	93	82	82
query53	446	319	329	319
query54	2639	2460	2473	2460
query55	95	87	85	85
query56	241	225	217	217
query57	1140	1140	1128	1128
query58	229	203	197	197
query59	4284	4380	3984	3984
query60	207	211	209	209
query61	95	93	93	93
query62	822	471	472	471
query63	487	342	340	340
query64	2575	1542	1484	1484
query65	3621	3589	3609	3589
query66	816	374	376	374
query67	15956	15628	15541	15541
query68	9351	664	679	664
query69	567	346	345	345
query70	1651	1381	1466	1381
query71	411	313	325	313
query72	6566	3529	3499	3499
query73	731	334	329	329
query74	6368	5939	5837	5837
query75	5316	3764	3716	3716
query76	5728	1146	1182	1146
query77	1004	271	266	266
query78	13750	12291	11628	11628
query79	10796	648	636	636
query80	849	403	407	403
query81	494	234	233	233
query82	1322	95	94	94
query83	169	131	132	131
query84	256	72	71	71
query85	718	340	336	336
query86	330	332	309	309
query87	3301	3005	3040	3005
query88	5031	2389	2375	2375
query89	435	282	300	282
query90	1924	214	207	207
query91	174	140	140	140
query92	58	60	55	55
query93	6383	599	571	571
query94	669	213	207	207
query95	1106	1041	1052	1041
query96	660	338	334	334
query97	6440	6440	6361	6361
query98	183	190	172	172
query99	2843	781	866	781
Total cold run time: 317837 ms
Total hot run time: 204518 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.89 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 05cda028ebc79548aa017251d101471f20ae4113, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.03
query3	0.24	0.05	0.05
query4	1.78	0.07	0.08
query5	0.53	0.52	0.53
query6	1.27	0.61	0.62
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.47	0.47
query10	0.53	0.54	0.53
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.62	0.62	0.62
query14	0.77	0.77	0.79
query15	0.78	0.77	0.75
query16	0.38	0.39	0.38
query17	1.02	1.01	1.03
query18	0.23	0.24	0.24
query19	1.94	1.80	1.85
query20	0.01	0.00	0.01
query21	15.47	0.56	0.55
query22	2.16	2.00	1.61
query23	17.30	1.09	0.90
query24	6.50	1.21	1.16
query25	0.39	0.07	0.06
query26	0.69	0.16	0.15
query27	0.05	0.05	0.04
query28	6.34	0.78	0.72
query29	12.61	2.32	2.23
query30	0.57	0.55	0.57
query31	2.80	0.38	0.37
query32	3.41	0.51	0.50
query33	3.15	3.10	3.08
query34	15.26	4.80	4.80
query35	4.85	4.81	4.85
query36	1.06	1.02	1.02
query37	0.05	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.15	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 103.97 s
Total hot run time: 30.89 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 05cda028ebc79548aa017251d101471f20ae4113 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.1 seconds inserted 10000000 Rows, about 473K ops/s

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50534 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2e3c68e97f1410b6557de348eb6dee3b42c689ae, data reload: false

------ Round 1 ----------------------------------
q1	17541	4340	4323	4323
q2	2077	158	151	151
q3	10254	1921	1921	1921
q4	10100	1227	1333	1227
q5	8530	3946	3901	3901
q6	234	125	124	124
q7	2041	1619	1612	1612
q8	9275	2732	2697	2697
q9	11032	10908	11042	10908
q10	8740	3543	3472	3472
q11	436	244	251	244
q12	472	304	305	304
q13	18393	3999	4057	3999
q14	352	342	325	325
q15	524	453	455	453
q16	675	568	574	568
q17	1128	961	981	961
q18	7290	6967	6887	6887
q19	1790	1650	1597	1597
q20	552	315	314	314
q21	4470	4105	4099	4099
q22	521	450	447	447
Total cold run time: 116427 ms
Total hot run time: 50534 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4334	4311	4302	4302
q2	322	229	223	223
q3	4162	4139	4124	4124
q4	2763	2723	2743	2723
q5	7188	7140	7170	7140
q6	241	121	119	119
q7	3229	2829	2819	2819
q8	4318	4444	4500	4444
q9	17401	17211	16975	16975
q10	4199	4295	4264	4264
q11	773	693	679	679
q12	1066	860	856	856
q13	6807	3727	3755	3727
q14	462	436	427	427
q15	507	454	459	454
q16	737	679	666	666
q17	3873	3823	3802	3802
q18	8820	8661	8736	8661
q19	1732	1743	1646	1646
q20	2413	2175	2109	2109
q21	8530	8516	8479	8479
q22	1060	992	999	992
Total cold run time: 84937 ms
Total hot run time: 79631 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204742 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2e3c68e97f1410b6557de348eb6dee3b42c689ae, data reload: false

query1	952	392	386	386
query2	6524	2821	2428	2428
query3	6923	206	200	200
query4	20545	17955	18145	17955
query5	19764	6495	6501	6495
query6	276	213	225	213
query7	4157	311	313	311
query8	454	424	380	380
query9	3110	2638	2581	2581
query10	418	295	310	295
query11	11371	10784	10767	10767
query12	131	79	77	77
query13	5615	707	713	707
query14	17829	13275	13429	13275
query15	365	240	257	240
query16	6478	286	267	267
query17	1732	1446	895	895
query18	2324	425	417	417
query19	225	151	154	151
query20	82	75	84	75
query21	196	95	99	95
query22	5224	5022	5051	5022
query23	32741	31818	32063	31818
query24	6929	6455	6499	6455
query25	526	440	432	432
query26	539	166	165	165
query27	1874	293	299	293
query28	6169	2377	2347	2347
query29	2977	2662	2825	2662
query30	241	167	171	167
query31	934	740	747	740
query32	72	63	61	61
query33	396	269	265	265
query34	855	471	486	471
query35	1133	922	927	922
query36	1318	1252	1070	1070
query37	92	68	65	65
query38	3074	2957	2941	2941
query39	1365	1364	1313	1313
query40	203	93	100	93
query41	47	45	45	45
query42	85	92	81	81
query43	693	712	690	690
query44	1122	727	743	727
query45	251	238	239	238
query46	1234	950	1018	950
query47	1912	1626	1858	1626
query48	1023	724	695	695
query49	633	385	383	383
query50	859	629	677	629
query51	4839	4675	4687	4675
query52	96	72	88	72
query53	450	327	340	327
query54	2696	2467	2429	2429
query55	98	75	82	75
query56	243	226	216	216
query57	1133	1156	1046	1046
query58	238	202	198	198
query59	4120	4050	3831	3831
query60	217	198	207	198
query61	108	97	100	97
query62	838	458	470	458
query63	498	342	351	342
query64	2384	1498	1501	1498
query65	3633	3536	3568	3536
query66	839	383	370	370
query67	16254	16879	17868	16879
query68	8763	639	649	639
query69	586	360	350	350
query70	1515	1567	1421	1421
query71	406	300	306	300
query72	6607	3487	3503	3487
query73	734	327	318	318
query74	6310	5821	5938	5821
query75	5220	3707	3664	3664
query76	5363	1125	1157	1125
query77	913	254	250	250
query78	12882	11838	11744	11744
query79	7830	629	636	629
query80	1104	410	403	403
query81	495	234	236	234
query82	1489	107	99	99
query83	170	135	131	131
query84	256	71	72	71
query85	893	335	337	335
query86	333	298	296	296
query87	3272	3056	3027	3027
query88	4899	2346	2346	2346
query89	383	285	312	285
query90	1971	210	223	210
query91	176	150	141	141
query92	60	53	58	53
query93	5079	565	594	565
query94	718	207	205	205
query95	1123	1044	1074	1044
query96	649	333	332	332
query97	6606	6384	6460	6384
query98	193	184	182	182
query99	2896	928	828	828
Total cold run time: 312570 ms
Total hot run time: 204742 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.66 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2e3c68e97f1410b6557de348eb6dee3b42c689ae, data reload: false

query1	0.02	0.03	0.02
query2	0.07	0.03	0.02
query3	0.25	0.04	0.04
query4	1.78	0.07	0.07
query5	0.54	0.53	0.52
query6	1.24	0.64	0.62
query7	0.02	0.01	0.01
query8	0.03	0.02	0.03
query9	0.52	0.48	0.48
query10	0.53	0.53	0.53
query11	0.12	0.10	0.08
query12	0.12	0.10	0.09
query13	0.64	0.62	0.62
query14	0.77	0.79	0.80
query15	0.78	0.76	0.76
query16	0.36	0.38	0.35
query17	1.01	1.02	0.95
query18	0.21	0.26	0.23
query19	1.92	1.83	1.86
query20	0.02	0.01	0.01
query21	15.47	0.57	0.55
query22	2.04	2.53	1.46
query23	17.04	1.13	0.83
query24	5.31	2.87	1.17
query25	0.40	0.13	0.04
query26	0.70	0.16	0.17
query27	0.05	0.04	0.04
query28	5.81	0.74	0.80
query29	12.76	2.29	2.34
query30	0.63	0.59	0.52
query31	2.80	0.39	0.36
query32	3.37	0.50	0.50
query33	3.12	3.07	3.06
query34	15.26	4.80	4.83
query35	4.89	4.89	4.84
query36	1.07	1.01	1.01
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 102.06 s
Total hot run time: 30.66 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 2e3c68e97f1410b6557de348eb6dee3b42c689ae with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50081 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f523cf42f584e33c1f5fd577b053fb0a845053c5, data reload: false

------ Round 1 ----------------------------------
q1	17631	4397	4361	4361
q2	2224	156	147	147
q3	10481	1910	1956	1910
q4	10361	1233	1335	1233
q5	8478	3869	3945	3869
q6	261	125	123	123
q7	2063	1611	1626	1611
q8	9807	2747	2714	2714
q9	11376	10261	10209	10209
q10	8679	3541	3510	3510
q11	414	235	249	235
q12	469	306	311	306
q13	18358	3965	4156	3965
q14	378	346	346	346
q15	528	471	474	471
q16	685	606	611	606
q17	1157	947	982	947
q18	7654	7368	7054	7054
q19	1790	1661	1620	1620
q20	755	300	298	298
q21	4520	4162	4109	4109
q22	530	457	437	437
Total cold run time: 118599 ms
Total hot run time: 50081 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4371	4320	4293	4293
q2	326	227	220	220
q3	4217	4142	4141	4141
q4	2736	2784	2746	2746
q5	7160	7103	7051	7051
q6	238	119	121	119
q7	3233	2868	2778	2778
q8	4351	4482	4448	4448
q9	16868	16721	16618	16618
q10	4227	4271	4234	4234
q11	750	742	693	693
q12	1025	876	849	849
q13	7346	3731	3749	3731
q14	457	423	422	422
q15	510	452	468	452
q16	727	680	674	674
q17	3921	3833	3829	3829
q18	8794	8819	8809	8809
q19	1714	1718	1647	1647
q20	2346	2119	2091	2091
q21	8445	8436	8480	8436
q22	1060	950	1010	950
Total cold run time: 84822 ms
Total hot run time: 79231 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203227 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f523cf42f584e33c1f5fd577b053fb0a845053c5, data reload: false

query1	939	415	378	378
query2	6556	2906	2687	2687
query3	6923	204	199	199
query4	20305	18018	18087	18018
query5	19733	6498	6525	6498
query6	293	220	235	220
query7	4157	298	304	298
query8	439	421	433	421
query9	3143	2734	2657	2657
query10	413	308	297	297
query11	11336	10707	10687	10687
query12	119	76	83	76
query13	5610	684	688	684
query14	17995	13162	13146	13146
query15	362	243	251	243
query16	6453	281	265	265
query17	1709	1456	882	882
query18	2302	416	412	412
query19	224	151	156	151
query20	80	77	82	77
query21	193	98	100	98
query22	5168	4931	4934	4931
query23	32662	31880	31915	31880
query24	6968	6586	6473	6473
query25	520	436	411	411
query26	526	166	160	160
query27	1796	297	304	297
query28	6140	2414	2370	2370
query29	2931	2825	2774	2774
query30	242	167	169	167
query31	905	729	742	729
query32	69	64	62	62
query33	405	263	253	253
query34	868	476	488	476
query35	1120	930	898	898
query36	1263	1130	1212	1130
query37	90	60	59	59
query38	3132	2944	2945	2944
query39	1388	1336	1323	1323
query40	209	96	89	89
query41	46	44	44	44
query42	84	81	84	81
query43	752	660	793	660
query44	1133	724	720	720
query45	246	236	236	236
query46	1235	959	979	959
query47	1997	1714	1697	1697
query48	1042	737	723	723
query49	630	369	375	369
query50	875	602	621	602
query51	4745	4648	4591	4591
query52	95	86	81	81
query53	449	324	325	324
query54	2623	2461	2485	2461
query55	84	91	80	80
query56	238	212	212	212
query57	1096	1276	1148	1148
query58	218	193	210	193
query59	4088	3814	3675	3675
query60	205	198	196	196
query61	98	94	102	94
query62	840	487	471	471
query63	491	344	348	344
query64	2521	1553	1331	1331
query65	3621	3540	3563	3540
query66	769	378	379	378
query67	16206	15410	15446	15410
query68	10465	654	644	644
query69	595	347	355	347
query70	1948	1449	1571	1449
query71	423	300	329	300
query72	6500	3491	3505	3491
query73	752	334	316	316
query74	6306	5918	5958	5918
query75	5402	3668	3752	3668
query76	6663	1171	1204	1171
query77	1162	250	249	249
query78	12659	11527	11566	11527
query79	7601	642	640	640
query80	970	403	396	396
query81	470	235	239	235
query82	873	103	100	100
query83	173	133	130	130
query84	249	70	71	70
query85	885	335	333	333
query86	327	303	297	297
query87	3250	3056	3098	3056
query88	4107	2280	2318	2280
query89	480	305	285	285
query90	1929	211	211	211
query91	187	153	141	141
query92	58	54	51	51
query93	6392	582	576	576
query94	702	201	211	201
query95	1107	1037	1045	1037
query96	640	330	318	318
query97	6429	6284	6318	6284
query98	188	187	176	176
query99	3002	900	890	890
Total cold run time: 315007 ms
Total hot run time: 203227 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.49 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f523cf42f584e33c1f5fd577b053fb0a845053c5, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.03
query3	0.25	0.04	0.04
query4	1.80	0.07	0.07
query5	0.55	0.53	0.53
query6	1.24	0.61	0.62
query7	0.01	0.01	0.01
query8	0.04	0.03	0.02
query9	0.52	0.48	0.50
query10	0.54	0.54	0.55
query11	0.12	0.08	0.08
query12	0.11	0.09	0.10
query13	0.62	0.61	0.61
query14	0.80	0.77	0.80
query15	0.78	0.75	0.77
query16	0.38	0.35	0.37
query17	1.04	1.00	0.98
query18	0.24	0.25	0.23
query19	1.92	1.78	1.81
query20	0.02	0.01	0.01
query21	15.52	0.55	0.54
query22	2.02	1.90	2.60
query23	17.46	0.94	1.07
query24	5.88	0.66	1.26
query25	0.41	0.06	0.05
query26	0.61	0.16	0.15
query27	0.05	0.04	0.04
query28	7.33	0.73	0.73
query29	12.65	2.26	2.30
query30	0.59	0.46	0.49
query31	2.80	0.38	0.38
query32	3.41	0.49	0.50
query33	3.05	3.03	3.06
query34	15.27	4.79	4.77
query35	4.87	4.83	4.83
query36	1.04	1.01	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.02	0.02
query43	0.02	0.01	0.01
Total cold run time: 104.43 s
Total hot run time: 30.49 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f523cf42f584e33c1f5fd577b053fb0a845053c5 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50161 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ff60f45116293221f57f64d130c1af1acc70cb31, data reload: false

------ Round 1 ----------------------------------
q1	17620	4453	4355	4355
q2	2093	154	149	149
q3	10467	1928	1950	1928
q4	10343	1225	1315	1225
q5	8672	3949	3973	3949
q6	254	125	126	125
q7	2053	1604	1582	1582
q8	9290	2721	2713	2713
q9	10677	10503	10542	10503
q10	8635	3473	3515	3473
q11	425	251	253	251
q12	474	300	302	300
q13	18380	3932	4038	3932
q14	352	341	339	339
q15	514	468	465	465
q16	710	564	576	564
q17	1133	947	954	947
q18	7280	6896	6923	6896
q19	1790	1679	1668	1668
q20	557	303	287	287
q21	4445	4108	4081	4081
q22	525	443	429	429
Total cold run time: 116689 ms
Total hot run time: 50161 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4335	4326	4321	4321
q2	325	222	225	222
q3	4201	4146	4136	4136
q4	2748	2732	2760	2732
q5	7181	7086	7082	7082
q6	239	120	119	119
q7	3219	2859	2853	2853
q8	4339	4483	4487	4483
q9	16881	16792	16782	16782
q10	4233	4224	4258	4224
q11	789	672	741	672
q12	1045	856	869	856
q13	7297	3732	3730	3730
q14	449	421	428	421
q15	501	454	450	450
q16	752	692	671	671
q17	3840	3890	3907	3890
q18	8883	8705	8847	8705
q19	1716	1710	1608	1608
q20	2405	2146	2152	2146
q21	8543	8522	8461	8461
q22	1049	985	972	972
Total cold run time: 84970 ms
Total hot run time: 79536 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204584 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ff60f45116293221f57f64d130c1af1acc70cb31, data reload: false

query1	917	428	379	379
query2	6528	2932	2715	2715
query3	6924	206	203	203
query4	20335	18003	18106	18003
query5	19727	6509	6570	6509
query6	287	212	228	212
query7	4163	301	307	301
query8	404	422	394	394
query9	3090	2643	2577	2577
query10	437	293	308	293
query11	11427	10728	10738	10728
query12	120	77	80	77
query13	5602	717	698	698
query14	17861	13125	13601	13125
query15	377	258	248	248
query16	6476	305	262	262
query17	1706	1435	883	883
query18	2306	413	411	411
query19	209	156	156	156
query20	82	78	77	77
query21	185	93	92	92
query22	5255	5027	5106	5027
query23	32666	32111	32066	32066
query24	6970	6601	6569	6569
query25	534	434	434	434
query26	526	167	157	157
query27	1899	297	291	291
query28	6060	2374	2341	2341
query29	2908	2790	2593	2593
query30	253	168	167	167
query31	931	754	739	739
query32	73	63	61	61
query33	411	271	276	271
query34	853	475	486	475
query35	1106	887	901	887
query36	1334	1524	1061	1061
query37	87	63	60	60
query38	3075	2943	2954	2943
query39	1402	1327	1349	1327
query40	212	95	94	94
query41	45	42	42	42
query42	81	80	89	80
query43	777	695	727	695
query44	1139	725	734	725
query45	250	232	234	232
query46	1222	951	964	951
query47	1842	1820	1833	1820
query48	1015	721	722	721
query49	619	373	385	373
query50	854	637	624	624
query51	4721	4634	4645	4634
query52	98	77	85	77
query53	453	319	320	319
query54	2655	2451	2487	2451
query55	83	91	84	84
query56	249	229	220	220
query57	1195	1099	1123	1099
query58	223	199	197	197
query59	4274	3980	3875	3875
query60	208	206	210	206
query61	97	94	101	94
query62	841	449	475	449
query63	483	349	344	344
query64	2465	1550	1479	1479
query65	3612	3564	3597	3564
query66	811	393	377	377
query67	15987	15326	15523	15326
query68	8820	670	645	645
query69	561	346	346	346
query70	1501	1443	1387	1387
query71	399	315	310	310
query72	6543	3474	3499	3474
query73	732	318	321	318
query74	6299	5840	5802	5802
query75	5234	3833	3711	3711
query76	5268	1148	1296	1148
query77	895	257	250	250
query78	12496	12509	12429	12429
query79	12702	673	640	640
query80	861	409	419	409
query81	490	230	234	230
query82	587	100	94	94
query83	162	138	131	131
query84	257	70	71	70
query85	843	339	332	332
query86	338	292	300	292
query87	3227	3040	3033	3033
query88	5577	2337	2337	2337
query89	409	290	275	275
query90	1956	200	210	200
query91	173	139	139	139
query92	57	51	58	51
query93	4721	557	562	557
query94	751	213	208	208
query95	1103	1081	1054	1054
query96	634	327	328	327
query97	6458	6390	6351	6351
query98	193	178	173	173
query99	2895	897	935	897
Total cold run time: 315159 ms
Total hot run time: 204584 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.95 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ff60f45116293221f57f64d130c1af1acc70cb31, data reload: false

query1	0.02	0.03	0.02
query2	0.07	0.02	0.03
query3	0.26	0.05	0.06
query4	1.78	0.07	0.07
query5	0.54	0.53	0.52
query6	1.27	0.64	0.62
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.54	0.49	0.47
query10	0.54	0.53	0.54
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.63	0.61	0.60
query14	0.77	0.80	0.79
query15	0.80	0.75	0.76
query16	0.39	0.37	0.40
query17	0.99	0.98	1.01
query18	0.23	0.25	0.24
query19	1.89	1.86	1.86
query20	0.01	0.01	0.01
query21	15.50	0.54	0.56
query22	2.09	2.40	1.89
query23	17.49	1.01	0.86
query24	6.00	0.77	1.32
query25	0.36	0.08	0.07
query26	0.67	0.16	0.17
query27	0.04	0.04	0.04
query28	7.04	0.77	0.80
query29	12.73	2.39	2.40
query30	0.63	0.53	0.52
query31	2.82	0.39	0.37
query32	3.37	0.51	0.51
query33	3.08	3.05	3.05
query34	15.27	4.80	4.79
query35	4.84	4.82	4.85
query36	1.03	1.02	1.01
query37	0.05	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.15	0.15	0.14
query41	0.07	0.02	0.02
query42	0.02	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.34 s
Total hot run time: 30.95 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ff60f45116293221f57f64d130c1af1acc70cb31 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@xiaokang xiaokang merged commit c3be8ec into apache:branch-2.0 Aug 1, 2024
22 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/load Issues or PRs related to all kinds of load area/nereids area/planner Issues or PRs related to the query planner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants