Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature](Nereids): add ColumnPruningPostProcessor. (#32800) #34077

Merged
merged 1 commit into from
Apr 25, 2024
Merged

[feature](Nereids): add ColumnPruningPostProcessor. (#32800) #34077

merged 1 commit into from
Apr 25, 2024

Conversation

jackwener
Copy link
Member

Proposed changes

pick #32800

(cherry picked from commit 5970f98)

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@jackwener
Copy link
Member Author

run buildall

@jackwener
Copy link
Member Author

run buildall

@jackwener
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49380 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5161bc7a1e878a569703f0ab41d258c798afb56b, data reload: false

------ Round 1 ----------------------------------
q1	18157	4419	4341	4341
q2	2029	148	140	140
q3	10473	1884	1930	1884
q4	10307	1239	1322	1239
q5	8511	3879	3895	3879
q6	224	123	122	122
q7	2014	1577	1634	1577
q8	9520	2691	2695	2691
q9	13485	10193	10177	10177
q10	8643	3532	3486	3486
q11	406	245	239	239
q12	463	309	313	309
q13	18369	3977	4000	3977
q14	356	326	330	326
q15	516	459	462	459
q16	679	573	578	573
q17	1110	927	915	915
q18	7182	6919	6752	6752
q19	1693	1559	1521	1521
q20	511	314	297	297
q21	4423	4093	4139	4093
q22	497	384	383	383
Total cold run time: 119568 ms
Total hot run time: 49380 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4340	4266	4359	4266
q2	315	220	224	220
q3	4122	4144	4121	4121
q4	2752	2751	2750	2750
q5	7206	7159	7106	7106
q6	233	118	116	116
q7	3268	2867	2873	2867
q8	4348	4453	4466	4453
q9	16855	16686	16613	16613
q10	4208	4246	4243	4243
q11	768	694	694	694
q12	1021	852	841	841
q13	6538	3741	3758	3741
q14	452	418	429	418
q15	499	448	451	448
q16	746	693	686	686
q17	3844	3833	3887	3833
q18	8778	8831	8830	8830
q19	1711	1692	1648	1648
q20	2361	2145	2108	2108
q21	8423	8510	8499	8499
q22	1006	925	923	923
Total cold run time: 83794 ms
Total hot run time: 79424 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203851 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5161bc7a1e878a569703f0ab41d258c798afb56b, data reload: false

query1	928	393	389	389
query2	6540	2786	2894	2786
query3	6918	207	197	197
query4	20971	18091	18068	18068
query5	19764	6513	6531	6513
query6	289	217	218	217
query7	4168	303	317	303
query8	250	283	265	265
query9	3174	2730	2664	2664
query10	407	297	309	297
query11	11320	10716	10763	10716
query12	118	80	66	66
query13	5578	698	671	671
query14	18384	13168	13272	13168
query15	362	215	229	215
query16	6455	271	251	251
query17	1754	1466	877	877
query18	2305	410	414	410
query19	203	148	150	148
query20	69	74	70	70
query21	191	94	93	93
query22	5334	5115	5049	5049
query23	32514	31858	31940	31858
query24	6906	6562	6482	6482
query25	512	411	393	393
query26	535	157	159	157
query27	1893	291	290	290
query28	6202	2417	2380	2380
query29	2873	2878	2894	2878
query30	241	161	157	157
query31	892	765	736	736
query32	71	59	54	54
query33	399	249	241	241
query34	840	486	488	486
query35	1141	935	923	923
query36	1230	1243	1117	1117
query37	90	60	58	58
query38	3085	2985	2890	2890
query39	1362	1315	1311	1311
query40	203	93	93	93
query41	44	33	33	33
query42	80	84	79	79
query43	797	766	717	717
query44	1145	736	729	729
query45	238	220	229	220
query46	1226	943	944	943
query47	1859	1746	1811	1746
query48	986	715	724	715
query49	618	368	352	352
query50	862	638	635	635
query51	4742	4713	4613	4613
query52	96	75	80	75
query53	442	324	319	319
query54	2671	2492	2452	2452
query55	86	80	76	76
query56	214	194	207	194
query57	1198	1080	1182	1080
query58	224	179	209	179
query59	4383	4199	4180	4180
query60	215	205	207	205
query61	87	82	80	80
query62	889	482	484	482
query63	468	339	345	339
query64	2491	1566	1430	1430
query65	3601	3534	3575	3534
query66	794	372	387	372
query67	15530	15547	16389	15547
query68	9621	649	683	649
query69	570	333	344	333
query70	1832	1504	1536	1504
query71	418	303	302	302
query72	6403	3409	3415	3409
query73	733	326	316	316
query74	6299	5863	5924	5863
query75	5508	3641	3655	3641
query76	6195	1162	1161	1161
query77	1033	254	245	245
query78	12435	11718	11583	11583
query79	7379	629	668	629
query80	946	402	385	385
query81	456	227	238	227
query82	835	99	97	97
query83	167	134	135	134
query84	257	72	69	69
query85	841	297	291	291
query86	333	289	290	289
query87	3324	3037	3013	3013
query88	4372	2343	2327	2327
query89	397	286	280	280
query90	1916	204	213	204
query91	164	133	130	130
query92	55	51	53	51
query93	6359	559	558	558
query94	689	202	203	202
query95	1088	1047	1056	1047
query96	648	334	321	321
query97	6479	6291	6273	6273
query98	184	180	170	170
query99	2917	885	921	885
Total cold run time: 313310 ms
Total hot run time: 203851 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.73 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5161bc7a1e878a569703f0ab41d258c798afb56b, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.02
query3	0.25	0.05	0.04
query4	1.81	0.06	0.07
query5	0.53	0.53	0.53
query6	1.24	0.63	0.62
query7	0.02	0.02	0.02
query8	0.04	0.02	0.02
query9	0.51	0.49	0.49
query10	0.54	0.53	0.54
query11	0.12	0.08	0.09
query12	0.11	0.08	0.09
query13	0.62	0.61	0.61
query14	0.78	0.78	0.78
query15	0.79	0.76	0.77
query16	0.37	0.37	0.36
query17	0.98	0.99	0.98
query18	0.23	0.25	0.26
query19	1.93	1.84	1.86
query20	0.02	0.02	0.02
query21	15.48	0.55	0.55
query22	2.34	2.14	1.43
query23	17.44	1.08	0.92
query24	5.08	1.10	1.06
query25	0.42	0.10	0.06
query26	0.54	0.17	0.15
query27	0.04	0.04	0.04
query28	7.89	0.77	0.73
query29	12.67	2.36	2.33
query30	0.59	0.54	0.50
query31	2.81	0.38	0.38
query32	3.38	0.50	0.49
query33	3.04	3.11	3.10
query34	15.23	4.83	4.80
query35	4.86	4.81	4.83
query36	1.04	1.02	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.21 s
Total hot run time: 30.73 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 5161bc7a1e878a569703f0ab41d258c798afb56b with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@jackwener
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49280 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 106a8c0db30111391610ba9ea8f208ee4611071f, data reload: false

------ Round 1 ----------------------------------
q1	17651	4364	4343	4343
q2	2043	153	137	137
q3	10441	1891	1945	1891
q4	10360	1245	1325	1245
q5	8498	4318	3848	3848
q6	225	123	121	121
q7	2014	1586	1557	1557
q8	9509	2705	2702	2702
q9	13795	10423	10312	10312
q10	8640	3530	3509	3509
q11	417	229	248	229
q12	466	300	300	300
q13	18364	3933	4000	3933
q14	346	330	326	326
q15	502	456	448	448
q16	673	563	558	558
q17	1112	929	915	915
q18	7278	6723	6829	6723
q19	1674	1521	1429	1429
q20	535	296	308	296
q21	4389	4110	4064	4064
q22	509	394	396	394
Total cold run time: 119441 ms
Total hot run time: 49280 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4319	4304	4302	4302
q2	316	220	213	213
q3	4139	4170	4110	4110
q4	2745	2735	2749	2735
q5	7235	7097	7097	7097
q6	233	116	118	116
q7	3242	2826	2822	2822
q8	4360	4434	4472	4434
q9	17447	17199	17042	17042
q10	4268	4268	4302	4268
q11	722	684	700	684
q12	1041	858	861	858
q13	7321	3753	3751	3751
q14	462	420	423	420
q15	505	449	457	449
q16	748	690	669	669
q17	3908	3873	3973	3873
q18	9269	9229	8836	8836
q19	1748	1699	1648	1648
q20	2444	2135	2157	2135
q21	8504	8492	8389	8389
q22	1001	926	934	926
Total cold run time: 85977 ms
Total hot run time: 79777 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 201324 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 106a8c0db30111391610ba9ea8f208ee4611071f, data reload: false

query1	909	383	377	377
query2	6526	2733	2659	2659
query3	6926	200	196	196
query4	20944	17933	17876	17876
query5	19746	6516	6446	6446
query6	293	211	234	211
query7	4149	290	292	290
query8	268	255	222	222
query9	3125	2698	2597	2597
query10	432	297	305	297
query11	11352	10721	10707	10707
query12	119	71	70	70
query13	5588	671	664	664
query14	18034	12920	13675	12920
query15	383	214	226	214
query16	6438	263	255	255
query17	1735	1452	865	865
query18	2311	409	396	396
query19	202	144	144	144
query20	76	79	79	79
query21	197	88	94	88
query22	5308	5083	4849	4849
query23	32217	31948	31896	31896
query24	6844	6463	6491	6463
query25	523	412	405	405
query26	526	158	154	154
query27	1850	292	287	287
query28	6217	2381	2342	2342
query29	3017	2843	2844	2843
query30	237	158	162	158
query31	903	751	722	722
query32	66	58	59	58
query33	397	245	253	245
query34	848	471	480	471
query35	1136	967	941	941
query36	1363	1213	1216	1213
query37	85	63	58	58
query38	3090	2919	3011	2919
query39	1373	1318	1314	1314
query40	204	92	95	92
query41	35	41	34	34
query42	89	85	84	84
query43	712	658	704	658
query44	1105	731	729	729
query45	240	221	231	221
query46	1208	964	961	961
query47	1800	1849	1583	1583
query48	1016	695	691	691
query49	611	356	367	356
query50	849	576	567	567
query51	4774	4621	4677	4621
query52	89	80	79	79
query53	439	310	300	300
query54	2627	2425	2459	2425
query55	79	82	75	75
query56	202	210	191	191
query57	1177	1080	1058	1058
query58	211	203	206	203
query59	3887	4269	3758	3758
query60	201	202	196	196
query61	89	82	86	82
query62	804	455	464	455
query63	472	329	334	329
query64	2274	1520	1453	1453
query65	3600	3830	3531	3531
query66	777	361	377	361
query67	16087	16554	14627	14627
query68	8717	645	677	645
query69	560	331	333	331
query70	1592	1325	1393	1325
query71	418	304	307	304
query72	6485	3428	3447	3428
query73	736	319	310	310
query74	6405	5929	5873	5873
query75	5194	3725	3731	3725
query76	5193	1140	1184	1140
query77	896	258	249	249
query78	12832	11842	17100	11842
query79	5018	627	635	627
query80	827	399	390	390
query81	492	232	226	226
query82	257	91	94	91
query83	173	140	128	128
query84	257	68	68	68
query85	848	314	301	301
query86	317	282	295	282
query87	3229	3044	3004	3004
query88	3575	2292	2307	2292
query89	337	287	306	287
query90	1947	204	216	204
query91	163	130	136	130
query92	59	52	53	52
query93	915	547	556	547
query94	701	196	197	196
query95	1095	1058	1043	1043
query96	632	327	320	320
query97	6544	6324	6378	6324
query98	192	176	173	173
query99	2946	903	836	836
Total cold run time: 300963 ms
Total hot run time: 201324 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 106a8c0db30111391610ba9ea8f208ee4611071f, data reload: false

query1	0.02	0.02	0.01
query2	0.06	0.02	0.02
query3	0.24	0.05	0.04
query4	1.82	0.07	0.06
query5	0.53	0.52	0.52
query6	1.24	0.63	0.62
query7	0.02	0.01	0.01
query8	0.03	0.02	0.03
query9	0.52	0.48	0.46
query10	0.53	0.53	0.54
query11	0.12	0.08	0.08
query12	0.12	0.10	0.09
query13	0.61	0.60	0.63
query14	0.78	0.77	0.78
query15	0.77	0.74	0.75
query16	0.37	0.36	0.37
query17	0.99	0.98	0.99
query18	0.19	0.28	0.24
query19	1.93	1.86	1.82
query20	0.01	0.01	0.01
query21	15.45	0.56	0.56
query22	2.16	2.70	1.50
query23	17.00	1.00	0.95
query24	5.51	1.29	1.33
query25	0.36	0.06	0.07
query26	0.72	0.16	0.14
query27	0.04	0.05	0.03
query28	7.05	0.72	0.73
query29	12.95	2.33	2.61
query30	0.63	0.54	0.51
query31	2.80	0.39	0.37
query32	3.40	0.49	0.49
query33	3.03	3.11	3.08
query34	15.26	4.83	4.80
query35	4.86	4.86	4.87
query36	1.04	1.00	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.03
query39	0.02	0.01	0.01
query40	0.15	0.14	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.54 s
Total hot run time: 30.91 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 106a8c0db30111391610ba9ea8f208ee4611071f with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.7 seconds inserted 10000000 Rows, about 460K ops/s

@jackwener
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49533 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e616e2ad77e6927790476db42ae98dc2e2ab0990, data reload: false

------ Round 1 ----------------------------------
q1	18027	4343	4305	4305
q2	2018	148	140	140
q3	10612	1897	1941	1897
q4	10309	1225	1307	1225
q5	8956	3849	3848	3848
q6	221	123	123	123
q7	2023	1565	1593	1565
q8	9348	2708	2712	2708
q9	10594	10334	10207	10207
q10	8638	3502	3512	3502
q11	429	232	251	232
q12	473	306	323	306
q13	18351	3960	4064	3960
q14	360	320	318	318
q15	507	453	455	453
q16	694	572	576	572
q17	1122	959	971	959
q18	7268	6894	6988	6894
q19	1706	1598	1576	1576
q20	540	319	290	290
q21	4490	4112	4060	4060
q22	490	393	403	393
Total cold run time: 117176 ms
Total hot run time: 49533 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4280	4299	4298	4298
q2	317	217	224	217
q3	4141	4145	4111	4111
q4	2741	2722	2753	2722
q5	7173	7051	7068	7051
q6	237	118	119	118
q7	3165	2782	2898	2782
q8	4322	4448	4471	4448
q9	16924	16846	16690	16690
q10	4155	4256	4258	4256
q11	760	657	664	657
q12	1011	851	844	844
q13	5361	3676	3760	3676
q14	446	425	426	425
q15	493	455	458	455
q16	719	681	684	681
q17	3795	3830	3903	3830
q18	8661	8787	8828	8787
q19	1711	1701	1646	1646
q20	2394	2145	2121	2121
q21	8519	8441	8485	8441
q22	1031	971	923	923
Total cold run time: 82356 ms
Total hot run time: 79179 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203923 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e616e2ad77e6927790476db42ae98dc2e2ab0990, data reload: false

query1	920	381	393	381
query2	6564	2867	2844	2844
query3	6923	195	194	194
query4	21161	18065	18007	18007
query5	19730	6546	6497	6497
query6	282	213	232	213
query7	4161	302	304	302
query8	267	239	259	239
query9	3122	2718	2635	2635
query10	514	300	316	300
query11	11263	10793	10697	10697
query12	123	80	68	68
query13	5575	684	687	684
query14	17471	13518	13686	13518
query15	381	234	231	231
query16	6461	271	261	261
query17	1745	1460	862	862
query18	2344	403	405	403
query19	195	147	146	146
query20	77	77	73	73
query21	186	95	96	95
query22	5060	4937	4941	4937
query23	32589	31909	31891	31891
query24	7032	6527	6575	6527
query25	514	443	420	420
query26	527	160	160	160
query27	1842	292	297	292
query28	6167	2374	2358	2358
query29	2997	2744	2805	2744
query30	239	160	160	160
query31	914	718	712	712
query32	67	57	59	57
query33	390	250	247	247
query34	869	462	477	462
query35	1105	855	917	855
query36	1666	1325	1319	1319
query37	86	65	61	61
query38	3075	2920	2948	2920
query39	1375	1311	1328	1311
query40	201	97	93	93
query41	41	35	34	34
query42	93	85	79	79
query43	718	776	646	646
query44	1133	730	742	730
query45	244	228	227	227
query46	1224	942	979	942
query47	1929	1682	2017	1682
query48	1008	706	687	687
query49	612	370	371	370
query50	876	576	573	573
query51	4773	4681	4642	4642
query52	83	88	72	72
query53	441	323	319	319
query54	2652	2477	2480	2477
query55	80	83	84	83
query56	203	197	189	189
query57	1223	1227	1120	1120
query58	219	211	199	199
query59	4220	3915	3938	3915
query60	203	197	197	197
query61	89	82	83	82
query62	852	514	505	505
query63	468	335	345	335
query64	2345	1604	1460	1460
query65	3598	3571	3552	3552
query66	771	369	373	369
query67	17591	15779	16745	15779
query68	7156	654	686	654
query69	562	333	330	330
query70	1593	1545	1342	1342
query71	398	304	300	300
query72	6533	3478	3459	3459
query73	720	324	312	312
query74	6303	5887	5782	5782
query75	4567	3710	3654	3654
query76	4513	1161	1199	1161
query77	536	249	247	247
query78	12385	11603	11682	11603
query79	7103	661	645	645
query80	2715	394	400	394
query81	532	227	227	227
query82	1553	96	94	94
query83	203	136	136	136
query84	260	71	67	67
query85	1385	297	288	288
query86	364	314	294	294
query87	3287	2995	3046	2995
query88	4934	2361	2344	2344
query89	363	308	311	308
query90	1669	213	214	213
query91	166	134	144	134
query92	58	52	54	52
query93	2804	584	582	582
query94	897	203	203	203
query95	1132	1037	1044	1037
query96	646	343	324	324
query97	6401	6259	6380	6259
query98	198	181	171	171
query99	3041	871	871	871
Total cold run time: 308846 ms
Total hot run time: 203923 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.76 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e616e2ad77e6927790476db42ae98dc2e2ab0990, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.04	0.06
query4	1.80	0.07	0.06
query5	0.54	0.53	0.51
query6	1.24	0.62	0.62
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.51	0.47	0.48
query10	0.53	0.53	0.54
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.62	0.63	0.62
query14	0.79	0.78	0.81
query15	0.78	0.75	0.78
query16	0.36	0.36	0.39
query17	1.03	1.02	0.99
query18	0.22	0.27	0.23
query19	1.87	1.86	1.86
query20	0.02	0.01	0.01
query21	15.48	0.56	0.55
query22	2.06	2.12	1.48
query23	17.22	1.03	0.96
query24	5.45	1.12	1.50
query25	0.36	0.14	0.03
query26	0.67	0.16	0.15
query27	0.04	0.04	0.04
query28	7.27	0.72	0.71
query29	12.60	2.36	2.33
query30	0.61	0.50	0.53
query31	2.81	0.39	0.37
query32	3.39	0.50	0.50
query33	3.10	3.06	3.08
query34	15.24	4.83	4.80
query35	4.85	4.83	4.84
query36	1.06	1.02	1.02
query37	0.05	0.04	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.01	0.01
query43	0.03	0.01	0.02
Total cold run time: 103.51 s
Total hot run time: 30.76 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit e616e2ad77e6927790476db42ae98dc2e2ab0990 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@jackwener jackwener merged commit 991b6a7 into apache:branch-2.0 Apr 25, 2024
24 of 25 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@jackwener
[feature](Nereids): add ColumnPruningPostProcessor. (apache#32800) (a…
ac5ec1f 
…pache#34077)

(cherry picked from commit 5970f98)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/nereids
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jackwener @doris-robot