Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](statistics)Reduce partition column sample BE memory consumption. (#41203) #41359

Merged
merged 1 commit into from
Oct 8, 2024
Merged

[improvement](statistics)Reduce partition column sample BE memory consumption. (#41203) #41359

merged 1 commit into from
Oct 8, 2024

Conversation

Jibing-Li
Copy link
Contributor

backport: #41203

@Jibing-Li Jibing-Li marked this pull request as ready for review September 26, 2024 10:09
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49003 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e4a4b2f215a14f626f118a9ddade716698cb0c3e, data reload: false

------ Round 1 ----------------------------------
q1	17688	4365	4356	4356
q2	2108	154	148	148
q3	10494	1866	1887	1866
q4	10309	1234	1331	1234
q5	8442	3904	3885	3885
q6	222	125	122	122
q7	2028	1610	1598	1598
q8	9312	2730	2691	2691
q9	10161	9785	9689	9689
q10	8653	3540	3564	3540
q11	423	251	246	246
q12	462	300	293	293
q13	18336	3976	4053	3976
q14	350	329	332	329
q15	536	464	464	464
q16	532	454	460	454
q17	1147	963	955	955
q18	7227	6954	6833	6833
q19	1728	1563	1545	1545
q20	521	327	316	316
q21	4436	4153	4080	4080
q22	500	383	399	383
Total cold run time: 115615 ms
Total hot run time: 49003 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4337	4356	4355	4355
q2	325	232	221	221
q3	4165	4169	4124	4124
q4	2755	2753	2759	2753
q5	7198	7102	7192	7102
q6	235	120	119	119
q7	3215	2852	2829	2829
q8	4396	4500	4465	4465
q9	13948	13933	14143	13933
q10	4431	4368	4282	4282
q11	748	675	677	675
q12	1047	886	862	862
q13	8744	3802	3765	3765
q14	459	416	429	416
q15	512	463	461	461
q16	629	581	592	581
q17	3781	3811	3736	3736
q18	8793	8821	8805	8805
q19	1719	1651	1630	1630
q20	2392	2196	2108	2108
q21	8499	8396	8414	8396
q22	1043	953	910	910
Total cold run time: 83371 ms
Total hot run time: 76528 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212359 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e4a4b2f215a14f626f118a9ddade716698cb0c3e, data reload: false

query1	927	426	378	378
query2	6553	2188	2285	2188
query3	6923	203	199	199
query4	23298	21819	21301	21301
query5	19736	6519	6518	6518
query6	323	222	246	222
query7	4379	294	306	294
query8	268	271	240	240
query9	3099	2644	2591	2591
query10	427	299	306	299
query11	15563	15651	15405	15405
query12	121	74	73	73
query13	1018	433	432	432
query14	17879	13456	13452	13452
query15	384	214	226	214
query16	6448	280	257	257
query17	1684	928	891	891
query18	883	333	308	308
query19	212	157	153	153
query20	106	99	100	99
query21	196	96	94	94
query22	5354	5109	5103	5103
query23	34207	33385	33582	33385
query24	6756	6321	6339	6321
query25	529	431	414	414
query26	801	158	159	158
query27	2374	293	292	292
query28	6057	2250	2211	2211
query29	2941	2712	2678	2678
query30	248	173	166	166
query31	969	744	732	732
query32	68	65	58	58
query33	448	264	262	262
query34	862	500	488	488
query35	1115	925	886	886
query36	1387	1278	1125	1125
query37	92	58	61	58
query38	3099	2943	2875	2875
query39	1370	1316	1321	1316
query40	207	94	90	90
query41	39	36	37	36
query42	87	82	90	82
query43	619	577	625	577
query44	1189	703	711	703
query45	250	231	225	225
query46	1236	968	955	955
query47	1930	1675	2122	1675
query48	518	421	402	402
query49	616	361	363	361
query50	853	603	606	603
query51	4766	4647	4687	4647
query52	97	84	80	80
query53	246	197	197	197
query54	2640	2590	2474	2474
query55	96	80	83	80
query56	216	200	199	199
query57	1264	1186	1124	1124
query58	204	210	215	210
query59	3570	3310	3300	3300
query60	214	193	194	193
query61	95	97	94	94
query62	734	465	443	443
query63	200	172	171	171
query64	4985	1644	1513	1513
query65	3624	3545	3534	3534
query66	638	418	388	388
query67	15435	15852	15109	15109
query68	10516	680	625	625
query69	498	257	272	257
query70	1744	1346	1607	1346
query71	412	312	306	306
query72	6898	4813	4709	4709
query73	765	331	323	323
query74	6398	5807	5832	5807
query75	5303	3643	3732	3643
query76	6195	1138	1185	1138
query77	1059	252	250	250
query78	12659	11896	11851	11851
query79	8046	643	658	643
query80	1388	395	372	372
query81	487	237	232	232
query82	1696	96	93	93
query83	171	132	129	129
query84	259	70	68	68
query85	872	312	316	312
query86	341	288	293	288
query87	3217	3029	3045	3029
query88	4501	2282	2278	2278
query89	478	273	291	273
query90	1993	206	216	206
query91	169	130	124	124
query92	59	49	49	49
query93	7207	570	555	555
query94	676	212	208	208
query95	1975	1944	1948	1944
query96	641	321	323	321
query97	6456	6301	6366	6301
query98	234	209	206	206
query99	3041	931	872	872
Total cold run time: 322026 ms
Total hot run time: 212359 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.47 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e4a4b2f215a14f626f118a9ddade716698cb0c3e, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.04	0.04
query4	1.83	0.07	0.06
query5	0.54	0.53	0.53
query6	1.24	0.62	0.62
query7	0.01	0.02	0.01
query8	0.04	0.03	0.02
query9	0.52	0.47	0.46
query10	0.52	0.54	0.53
query11	0.12	0.08	0.08
query12	0.11	0.08	0.09
query13	0.62	0.61	0.61
query14	0.78	0.79	0.76
query15	0.77	0.75	0.76
query16	0.36	0.36	0.36
query17	1.03	1.01	1.01
query18	0.22	0.26	0.24
query19	1.96	1.88	1.86
query20	0.02	0.02	0.01
query21	15.46	0.54	0.54
query22	2.05	2.34	1.77
query23	17.13	0.96	0.95
query24	5.69	0.75	0.95
query25	0.40	0.14	0.04
query26	0.55	0.15	0.15
query27	0.04	0.04	0.02
query28	7.73	0.78	0.70
query29	12.62	2.21	2.28
query30	0.56	0.52	0.54
query31	2.80	0.38	0.38
query32	3.40	0.51	0.49
query33	3.11	3.06	3.04
query34	15.23	4.77	4.79
query35	4.90	4.83	4.81
query36	1.04	1.01	1.01
query37	0.05	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.02	0.01
query43	0.03	0.02	0.02
Total cold run time: 104.13 s
Total hot run time: 30.47 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit e4a4b2f215a14f626f118a9ddade716698cb0c3e with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s

@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 48916 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 79700f3ccffc58e62d051e3f103c4a6545ff5ae7, data reload: false

------ Round 1 ----------------------------------
q1	17907	4336	4316	4316
q2	2088	160	137	137
q3	10310	1946	1940	1940
q4	10364	1236	1341	1236
q5	8807	3852	3892	3852
q6	237	123	121	121
q7	2039	1602	1575	1575
q8	9351	2728	2694	2694
q9	10223	9935	9659	9659
q10	8604	3561	3495	3495
q11	431	249	249	249
q12	460	302	300	300
q13	18356	4007	4037	4007
q14	355	332	340	332
q15	512	455	458	455
q16	560	460	469	460
q17	1133	949	952	949
q18	7153	6883	6884	6883
q19	1706	1580	1517	1517
q20	525	315	304	304
q21	4399	4192	4043	4043
q22	497	405	392	392
Total cold run time: 116017 ms
Total hot run time: 48916 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4302	4287	4329	4287
q2	326	227	220	220
q3	4152	4123	4096	4096
q4	2757	2760	2745	2745
q5	7114	7086	7120	7086
q6	234	119	118	118
q7	3217	2897	2874	2874
q8	4369	4448	4528	4448
q9	13590	13663	13612	13612
q10	4248	4252	4273	4252
q11	774	685	683	683
q12	1024	819	860	819
q13	6799	3728	3788	3728
q14	454	431	428	428
q15	507	468	447	447
q16	641	586	598	586
q17	3796	3879	3866	3866
q18	8888	8824	8810	8810
q19	1735	1650	1665	1650
q20	2388	2109	2101	2101
q21	8454	8481	8462	8462
q22	1026	929	949	929
Total cold run time: 80795 ms
Total hot run time: 76247 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211911 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 79700f3ccffc58e62d051e3f103c4a6545ff5ae7, data reload: false

query1	933	390	502	390
query2	6558	2190	2010	2010
query3	6923	202	202	202
query4	23139	21640	21660	21640
query5	19731	6473	6464	6464
query6	298	243	238	238
query7	4330	297	305	297
query8	262	261	235	235
query9	3064	2644	2561	2561
query10	462	305	303	303
query11	15801	15393	15241	15241
query12	132	72	72	72
query13	1040	446	434	434
query14	17240	13493	13290	13290
query15	366	218	225	218
query16	6491	281	258	258
query17	1769	920	895	895
query18	890	320	310	310
query19	221	147	153	147
query20	99	103	97	97
query21	193	95	96	95
query22	5214	5077	5025	5025
query23	34297	33421	33479	33421
query24	7877	6312	6406	6312
query25	524	431	428	428
query26	1277	161	160	160
query27	2377	298	290	290
query28	6087	2262	2213	2213
query29	2987	2689	2778	2689
query30	248	173	167	167
query31	942	737	742	737
query32	73	67	61	61
query33	455	253	259	253
query34	879	476	488	476
query35	1134	956	879	879
query36	1201	1242	1302	1242
query37	168	59	60	59
query38	3095	2934	2914	2914
query39	1364	1321	1301	1301
query40	309	95	96	95
query41	40	37	37	37
query42	86	84	86	84
query43	725	631	610	610
query44	1144	712	713	712
query45	256	229	225	225
query46	1233	957	948	948
query47	1908	1805	1906	1805
query48	494	407	406	406
query49	646	383	372	372
query50	851	603	608	603
query51	4776	4658	4668	4658
query52	95	85	79	79
query53	234	192	187	187
query54	2662	2477	2484	2477
query55	88	91	94	91
query56	231	209	201	201
query57	1252	1205	1203	1203
query58	218	203	211	203
query59	3589	3391	3187	3187
query60	223	197	218	197
query61	93	92	101	92
query62	814	534	443	443
query63	202	172	175	172
query64	3495	1586	1493	1493
query65	3582	3552	3588	3552
query66	767	436	417	417
query67	15468	15401	15137	15137
query68	9694	637	635	635
query69	490	278	280	278
query70	1650	1335	1437	1335
query71	411	304	317	304
query72	6830	4755	4336	4336
query73	765	319	325	319
query74	6330	5905	5858	5858
query75	5366	3606	3609	3606
query76	5470	1139	1179	1139
query77	942	266	252	252
query78	12528	11642	11885	11642
query79	8412	643	659	643
query80	1625	380	403	380
query81	483	242	235	235
query82	1627	97	99	97
query83	168	132	133	132
query84	260	72	70	70
query85	884	317	305	305
query86	336	297	298	297
query87	3189	3031	3096	3031
query88	4610	2274	2260	2260
query89	483	300	296	296
query90	1962	212	210	210
query91	157	135	125	125
query92	59	51	52	51
query93	6699	569	594	569
query94	692	214	209	209
query95	1955	1979	1937	1937
query96	644	320	323	320
query97	6468	6407	6320	6320
query98	242	207	207	207
query99	2949	877	803	803
Total cold run time: 320002 ms
Total hot run time: 211911 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.3 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 79700f3ccffc58e62d051e3f103c4a6545ff5ae7, data reload: false

query1	0.03	0.03	0.02
query2	0.08	0.03	0.02
query3	0.25	0.05	0.04
query4	1.81	0.06	0.06
query5	0.54	0.54	0.53
query6	1.32	0.62	0.62
query7	0.01	0.02	0.02
query8	0.03	0.02	0.02
query9	0.52	0.51	0.47
query10	0.52	0.53	0.53
query11	0.13	0.08	0.08
query12	0.12	0.09	0.09
query13	0.63	0.60	0.60
query14	0.78	0.77	0.78
query15	0.78	0.76	0.76
query16	0.37	0.37	0.39
query17	0.99	1.03	1.02
query18	0.24	0.24	0.25
query19	1.95	1.79	1.84
query20	0.01	0.01	0.01
query21	15.45	0.55	0.53
query22	2.01	2.08	1.75
query23	17.28	0.97	1.12
query24	5.48	2.39	1.39
query25	0.35	0.11	0.05
query26	0.85	0.16	0.15
query27	0.04	0.05	0.04
query28	5.89	0.73	0.72
query29	12.62	2.29	2.21
query30	0.56	0.53	0.54
query31	2.80	0.38	0.37
query32	3.40	0.50	0.51
query33	3.09	3.14	3.14
query34	15.29	4.82	4.79
query35	4.82	4.84	4.85
query36	1.07	1.01	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 102.48 s
Total hot run time: 31.3 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 79700f3ccffc58e62d051e3f103c4a6545ff5ae7 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.1 seconds inserted 10000000 Rows, about 452K ops/s

@Jibing-Li
Copy link
Contributor Author

run external

4 similar comments
@Jibing-Li
Copy link
Contributor Author

run external

@Jibing-Li
Copy link
Contributor Author

run external

@morrySnow
Copy link
Contributor

run external

@Jibing-Li
Copy link
Contributor Author

run external

@Jibing-Li
[improvement](statistics)Reduce partition column sample BE memory con…
2790511 
…sumption. (apache#41203)

For string type columns, use xxhash_64 to transfer column value to an
integer, and then calculate the NDV based on the integer hash value. In
this case, we can reduce the memory cost of sample analyze and improve
the performance.
For example, l_comment column of TPCH 100G lineitem table. The memory
cost to calculate its NDV is reduced to 8GB from 22GB
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49176 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 27905118f6f9e5db65ad94638a30f5bd3a60e08c, data reload: false

------ Round 1 ----------------------------------
q1	17632	4341	4362	4341
q2	2061	152	144	144
q3	10457	1895	1970	1895
q4	10351	1242	1326	1242
q5	8451	3889	3892	3889
q6	233	125	124	124
q7	2014	1635	1658	1635
q8	9344	2723	2721	2721
q9	10129	9758	9792	9758
q10	8629	3532	3512	3512
q11	422	255	237	237
q12	471	309	306	306
q13	18350	3972	4049	3972
q14	355	339	326	326
q15	521	472	468	468
q16	549	451	463	451
q17	1133	968	916	916
q18	7298	6913	6996	6913
q19	1679	1543	1558	1543
q20	523	325	313	313
q21	4390	4157	4077	4077
q22	504	402	393	393
Total cold run time: 115496 ms
Total hot run time: 49176 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4314	4304	4343	4304
q2	326	225	222	222
q3	4137	4111	4144	4111
q4	2788	2745	2764	2745
q5	7134	7121	7075	7075
q6	236	121	119	119
q7	3252	2852	2871	2852
q8	4361	4523	4499	4499
q9	13578	13416	13430	13416
q10	4281	4264	4257	4257
q11	779	722	706	706
q12	1024	854	845	845
q13	6855	3699	3754	3699
q14	462	426	431	426
q15	499	450	464	450
q16	621	588	573	573
q17	3811	3827	3840	3827
q18	8655	8683	8668	8668
q19	1714	1681	1650	1650
q20	2407	2132	2074	2074
q21	8458	8450	8448	8448
q22	1032	932	954	932
Total cold run time: 80724 ms
Total hot run time: 75898 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211951 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 27905118f6f9e5db65ad94638a30f5bd3a60e08c, data reload: false

query1	954	394	408	394
query2	6529	2196	2130	2130
query3	6920	204	203	203
query4	23965	21627	21288	21288
query5	19747	6519	6464	6464
query6	285	225	246	225
query7	4329	302	304	302
query8	254	283	231	231
query9	3033	2634	2559	2559
query10	451	309	295	295
query11	15676	15413	14866	14866
query12	125	89	69	69
query13	1034	438	424	424
query14	17041	13648	13115	13115
query15	386	229	229	229
query16	6486	277	256	256
query17	1759	919	914	914
query18	917	322	309	309
query19	219	150	147	147
query20	105	97	98	97
query21	190	95	93	93
query22	5336	5248	5013	5013
query23	33942	33440	33657	33440
query24	7856	6355	6291	6291
query25	523	429	425	425
query26	1264	162	162	162
query27	2393	295	293	293
query28	6090	2259	2217	2217
query29	2912	2738	2627	2627
query30	238	168	164	164
query31	953	736	749	736
query32	68	62	60	60
query33	448	265	270	265
query34	861	462	474	462
query35	1115	909	936	909
query36	1402	1358	1013	1013
query37	168	58	59	58
query38	3090	2942	2898	2898
query39	1392	1329	1324	1324
query40	302	96	95	95
query41	38	36	36	36
query42	94	87	89	87
query43	598	660	583	583
query44	1140	718	708	708
query45	238	237	228	228
query46	1227	963	949	949
query47	1856	1940	1849	1849
query48	498	415	402	402
query49	645	379	383	379
query50	863	585	629	585
query51	4771	4746	4658	4658
query52	95	82	75	75
query53	239	183	187	183
query54	2680	2478	2512	2478
query55	93	79	87	79
query56	235	196	203	196
query57	1215	1216	1078	1078
query58	220	205	204	204
query59	3551	3338	3363	3338
query60	211	192	195	192
query61	93	93	96	93
query62	744	481	535	481
query63	209	181	176	176
query64	3518	1590	1437	1437
query65	3628	3570	3557	3557
query66	763	413	388	388
query67	18080	15441	15339	15339
query68	11139	642	660	642
query69	511	283	276	276
query70	2139	1380	1508	1380
query71	411	311	329	311
query72	6792	4824	4774	4774
query73	803	328	324	324
query74	6335	5808	5807	5807
query75	5452	3750	3640	3640
query76	6868	1150	1188	1150
query77	1201	258	253	253
query78	12529	21186	12015	12015
query79	7110	649	651	649
query80	599	388	407	388
query81	494	238	239	238
query82	266	102	98	98
query83	179	131	137	131
query84	262	75	74	74
query85	731	334	318	318
query86	323	296	301	296
query87	3261	3008	3005	3005
query88	4277	2281	2277	2277
query89	354	305	286	286
query90	2150	222	214	214
query91	159	126	125	125
query92	57	51	54	51
query93	952	556	546	546
query94	846	212	211	211
query95	2047	2030	1952	1952
query96	642	329	321	321
query97	6488	6333	6315	6315
query98	214	209	195	195
query99	2671	888	869	869
Total cold run time: 316562 ms
Total hot run time: 211951 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.9 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 27905118f6f9e5db65ad94638a30f5bd3a60e08c, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.02	0.02
query3	0.24	0.05	0.04
query4	1.82	0.07	0.07
query5	0.54	0.53	0.52
query6	1.24	0.66	0.61
query7	0.01	0.00	0.01
query8	0.04	0.02	0.02
query9	0.53	0.49	0.47
query10	0.54	0.54	0.53
query11	0.12	0.09	0.09
query12	0.12	0.10	0.09
query13	0.62	0.62	0.62
query14	0.78	0.80	0.79
query15	0.78	0.76	0.76
query16	0.36	0.39	0.37
query17	1.05	0.98	1.01
query18	0.23	0.25	0.26
query19	1.93	1.86	1.89
query20	0.02	0.01	0.01
query21	15.45	0.56	0.57
query22	2.09	1.92	1.42
query23	17.72	1.04	0.93
query24	5.93	1.25	1.35
query25	0.36	0.05	0.07
query26	0.71	0.15	0.15
query27	0.05	0.05	0.04
query28	6.63	0.74	0.73
query29	12.63	2.40	2.36
query30	0.56	0.53	0.52
query31	2.82	0.39	0.38
query32	3.38	0.50	0.49
query33	3.09	3.07	3.06
query34	15.26	4.79	4.81
query35	4.85	4.85	4.80
query36	1.07	1.01	1.02
query37	0.06	0.05	0.04
query38	0.04	0.03	0.02
query39	0.02	0.01	0.01
query40	0.16	0.15	0.14
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.01
Total cold run time: 104.06 s
Total hot run time: 30.9 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 27905118f6f9e5db65ad94638a30f5bd3a60e08c with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@Jibing-Li
Copy link
Contributor Author

run p0

@Jibing-Li Jibing-Li merged commit bf46281 into apache:branch-2.0 Oct 8, 2024
21 of 23 checks passed
@Jibing-Li Jibing-Li deleted the reduceMem2.0 branch October 8, 2024 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Jibing-Li @doris-robot @morrySnow