Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feat](nereids) adjust stats derive by delta row #39185

Merged
merged 1 commit into from
Aug 14, 2024
Merged

[feat](nereids) adjust stats derive by delta row #39185

merged 1 commit into from
Aug 14, 2024

Conversation

englefly
Copy link
Contributor

@englefly englefly commented Aug 10, 2024 •
edited
Loading

#39222
After analyzing, user may insert new rows.
analyzed rows: the rows have been analyzed
delta row: rows inserted after analyze job

if analyzed rows are filtered out, then we try to estimate filter result by delta row with unknown column stats.

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49647 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6b3689ceb59bff5d2e0d5a91be409ecf8dc23d21, data reload: false

------ Round 1 ----------------------------------
q1	18007	4364	4379	4364
q2	2029	157	154	154
q3	10254	1933	1875	1875
q4	10350	1240	1273	1240
q5	8827	3848	3886	3848
q6	234	125	124	124
q7	2059	1609	1622	1609
q8	9278	2714	2713	2713
q9	10573	10337	10149	10149
q10	8658	3485	3475	3475
q11	428	250	244	244
q12	470	299	297	297
q13	18365	3955	4007	3955
q14	345	323	328	323
q15	498	453	456	453
q16	658	586	561	561
q17	1128	954	961	954
q18	7229	6891	6883	6883
q19	1771	1690	1641	1641
q20	552	289	322	289
q21	4384	4136	4048	4048
q22	540	449	448	448
Total cold run time: 116637 ms
Total hot run time: 49647 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4298	4277	4280	4277
q2	322	237	225	225
q3	4201	4111	4157	4111
q4	2740	2730	2740	2730
q5	7131	7111	7117	7111
q6	232	118	119	118
q7	3237	2822	2868	2822
q8	4341	4471	4522	4471
q9	16866	16775	16849	16775
q10	4224	4270	4224	4224
q11	770	711	688	688
q12	1036	845	844	844
q13	7197	3746	3721	3721
q14	444	417	414	414
q15	490	459	462	459
q16	734	697	671	671
q17	3828	3808	3836	3808
q18	8683	8790	8668	8668
q19	1716	1686	1646	1646
q20	2386	2160	2094	2094
q21	8519	8592	8489	8489
q22	1063	1042	1000	1000
Total cold run time: 84458 ms
Total hot run time: 79366 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204857 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6b3689ceb59bff5d2e0d5a91be409ecf8dc23d21, data reload: false

query1	918	390	432	390
query2	6527	2768	2736	2736
query3	6919	212	203	203
query4	19991	17920	17940	17920
query5	19731	6518	6475	6475
query6	283	222	242	222
query7	4155	304	304	304
query8	442	418	404	404
query9	3127	2726	2690	2690
query10	419	303	285	285
query11	11403	10799	10830	10799
query12	131	78	75	75
query13	5755	696	732	696
query14	19412	13829	13737	13737
query15	405	243	244	243
query16	6525	301	261	261
query17	1786	1455	869	869
query18	2551	427	428	427
query19	557	158	168	158
query20	87	81	86	81
query21	532	105	98	98
query22	5750	5494	5587	5494
query23	33218	31976	31882	31882
query24	6988	6463	6518	6463
query25	509	438	410	410
query26	523	160	157	157
query27	1851	291	293	291
query28	6249	2402	2356	2356
query29	2951	2854	2867	2854
query30	240	169	171	169
query31	895	747	755	747
query32	70	62	61	61
query33	409	244	259	244
query34	845	465	473	465
query35	1154	929	935	929
query36	1480	1173	1148	1148
query37	92	60	64	60
query38	3051	2928	2953	2928
query39	1372	1313	1332	1313
query40	207	94	93	93
query41	46	44	44	44
query42	84	85	80	80
query43	669	718	689	689
query44	1225	727	737	727
query45	243	238	235	235
query46	1264	1004	1004	1004
query47	1777	1760	1861	1760
query48	1005	706	708	706
query49	623	376	371	371
query50	867	610	663	610
query51	4771	4595	4652	4595
query52	89	84	88	84
query53	450	319	328	319
query54	2645	2447	2459	2447
query55	91	73	85	73
query56	242	206	198	198
query57	1254	1088	1119	1088
query58	217	223	212	212
query59	4116	3950	3848	3848
query60	223	208	203	203
query61	100	93	100	93
query62	878	494	467	467
query63	495	335	340	335
query64	2539	1538	1390	1390
query65	3641	3577	3579	3577
query66	826	375	374	374
query67	17681	16494	15731	15731
query68	9509	661	667	661
query69	566	339	332	332
query70	1685	1293	1323	1293
query71	427	313	313	313
query72	6517	3482	3451	3451
query73	730	328	322	322
query74	6375	5912	5917	5912
query75	5348	3718	3708	3708
query76	5744	1150	1206	1150
query77	990	266	256	256
query78	12426	11452	11587	11452
query79	6739	618	632	618
query80	978	417	402	402
query81	487	234	233	233
query82	1434	103	93	93
query83	177	135	135	135
query84	264	72	71	71
query85	894	332	327	327
query86	333	307	317	307
query87	3253	3003	2993	2993
query88	4840	2304	2295	2295
query89	350	291	287	287
query90	2059	202	203	202
query91	186	150	139	139
query92	59	52	54	52
query93	4253	619	578	578
query94	739	208	211	208
query95	1104	1063	1040	1040
query96	643	329	319	319
query97	6464	6387	6301	6301
query98	186	172	188	172
query99	2930	965	838	838
Total cold run time: 316260 ms
Total hot run time: 204857 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.82 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6b3689ceb59bff5d2e0d5a91be409ecf8dc23d21, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.02
query3	0.26	0.04	0.05
query4	1.80	0.07	0.07
query5	0.53	0.55	0.53
query6	1.23	0.62	0.60
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.49	0.47
query10	0.54	0.55	0.53
query11	0.13	0.09	0.08
query12	0.11	0.10	0.09
query13	0.62	0.61	0.61
query14	0.78	0.78	0.80
query15	0.77	0.77	0.76
query16	0.36	0.38	0.36
query17	1.01	1.03	1.01
query18	0.24	0.24	0.26
query19	1.89	1.85	1.84
query20	0.02	0.01	0.01
query21	15.43	0.56	0.57
query22	1.95	1.93	1.92
query23	17.56	1.03	0.98
query24	4.05	0.76	1.48
query25	0.36	0.09	0.05
query26	0.51	0.15	0.15
query27	0.04	0.04	0.04
query28	8.95	0.76	0.72
query29	12.60	2.23	2.25
query30	0.59	0.53	0.54
query31	2.80	0.38	0.38
query32	3.37	0.49	0.50
query33	3.04	3.06	3.08
query34	15.25	4.77	4.79
query35	4.87	4.83	4.84
query36	1.06	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.74 s
Total hot run time: 30.82 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 6b3689ceb59bff5d2e0d5a91be409ecf8dc23d21 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@englefly
Copy link
Contributor Author

run buildall

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49927 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4f105fd0d894f36c7268fc6e6e0e8f420f2726e5, data reload: false

------ Round 1 ----------------------------------
q1	18055	4438	4348	4348
q2	2070	161	154	154
q3	10419	2019	1933	1933
q4	10305	1273	1344	1273
q5	8552	3898	3946	3898
q6	235	126	127	126
q7	2093	1582	1619	1582
q8	9301	2768	2726	2726
q9	10601	10423	10358	10358
q10	8628	3571	3462	3462
q11	420	251	254	251
q12	472	306	308	306
q13	18343	3978	4027	3978
q14	357	330	334	330
q15	510	464	458	458
q16	682	573	563	563
q17	1144	979	976	976
q18	7274	6794	6829	6794
q19	1799	1674	1620	1620
q20	528	321	313	313
q21	4473	4105	4043	4043
q22	537	461	435	435
Total cold run time: 116798 ms
Total hot run time: 49927 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4304	4316	4309	4309
q2	320	221	214	214
q3	4166	4125	4132	4125
q4	2752	2748	2754	2748
q5	7222	7157	7108	7108
q6	240	122	125	122
q7	3241	2906	2907	2906
q8	4377	4456	4533	4456
q9	16966	16746	16762	16746
q10	4255	4288	4291	4288
q11	738	665	708	665
q12	1022	868	893	868
q13	6806	3771	3773	3771
q14	449	430	432	430
q15	500	451	463	451
q16	748	684	691	684
q17	3823	3819	3806	3806
q18	8784	8856	9000	8856
q19	1736	1724	1723	1723
q20	2415	2140	2101	2101
q21	8539	8532	8555	8532
q22	1020	1011	1035	1011
Total cold run time: 84423 ms
Total hot run time: 79920 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202951 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4f105fd0d894f36c7268fc6e6e0e8f420f2726e5, data reload: false

query1	919	390	418	390
query2	6559	2827	2627	2627
query3	6919	205	205	205
query4	19914	17901	17969	17901
query5	19736	6460	6485	6460
query6	284	211	228	211
query7	4154	305	324	305
query8	420	447	391	391
query9	3187	2747	2653	2653
query10	414	279	314	279
query11	11302	10657	10773	10657
query12	122	76	84	76
query13	5598	696	701	696
query14	18111	13312	13543	13312
query15	358	250	245	245
query16	6476	283	270	270
query17	1702	1469	879	879
query18	2316	416	410	410
query19	209	154	151	151
query20	81	78	78	78
query21	198	99	90	90
query22	5180	4886	4985	4886
query23	32535	31811	31940	31811
query24	6990	6676	6519	6519
query25	548	423	444	423
query26	530	169	164	164
query27	1881	296	291	291
query28	6205	2367	2358	2358
query29	2943	2750	2756	2750
query30	236	171	169	169
query31	914	747	734	734
query32	73	64	60	60
query33	398	262	269	262
query34	840	484	462	462
query35	1135	930	891	891
query36	1348	1201	1315	1201
query37	88	65	61	61
query38	3037	2891	2981	2891
query39	1383	1312	1328	1312
query40	200	103	91	91
query41	46	44	45	44
query42	88	92	95	92
query43	730	654	755	654
query44	1119	729	734	729
query45	249	237	236	236
query46	1239	960	990	960
query47	1962	2017	1744	1744
query48	1011	719	699	699
query49	624	363	370	363
query50	866	586	620	586
query51	4730	4632	4703	4632
query52	94	87	87	87
query53	445	319	315	315
query54	2678	2460	2483	2460
query55	82	76	84	76
query56	250	227	213	213
query57	1140	1204	1041	1041
query58	215	216	195	195
query59	4312	4162	3853	3853
query60	222	220	223	220
query61	99	96	98	96
query62	779	456	484	456
query63	494	347	344	344
query64	2470	1534	1513	1513
query65	3636	3578	3588	3578
query66	803	383	379	379
query67	15747	15436	14828	14828
query68	10233	669	654	654
query69	592	382	363	363
query70	1850	1574	1423	1423
query71	426	302	321	302
query72	6630	3499	3514	3499
query73	739	323	323	323
query74	6249	5843	5881	5843
query75	5267	3722	3665	3665
query76	6417	1176	1156	1156
query77	1102	278	258	258
query78	12702	11714	11720	11714
query79	7280	641	651	641
query80	1155	400	408	400
query81	483	235	236	235
query82	1385	102	100	100
query83	171	136	133	133
query84	256	73	72	72
query85	882	328	336	328
query86	339	312	297	297
query87	3237	3016	3036	3016
query88	4561	2326	2312	2312
query89	391	278	294	278
query90	1975	213	195	195
query91	177	137	144	137
query92	56	56	51	51
query93	6710	577	634	577
query94	698	214	211	211
query95	1166	1070	1072	1070
query96	644	346	339	339
query97	6546	6373	6454	6373
query98	209	198	165	165
query99	3020	861	806	806
Total cold run time: 315121 ms
Total hot run time: 202951 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.14 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4f105fd0d894f36c7268fc6e6e0e8f420f2726e5, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.02	0.02
query3	0.24	0.04	0.04
query4	1.80	0.07	0.07
query5	0.53	0.53	0.52
query6	1.28	0.61	0.61
query7	0.01	0.00	0.01
query8	0.04	0.02	0.02
query9	0.53	0.49	0.48
query10	0.54	0.55	0.54
query11	0.12	0.09	0.09
query12	0.11	0.09	0.10
query13	0.62	0.61	0.61
query14	0.79	0.78	0.79
query15	0.77	0.76	0.77
query16	0.39	0.39	0.37
query17	0.98	1.03	1.02
query18	0.22	0.27	0.24
query19	1.91	1.83	1.88
query20	0.02	0.01	0.01
query21	15.44	0.60	0.58
query22	1.95	2.27	2.09
query23	17.56	0.98	0.92
query24	5.91	0.88	5.01
query25	0.46	0.08	0.05
query26	0.94	0.15	0.14
query27	0.04	0.04	0.03
query28	3.46	0.77	0.75
query29	12.63	2.29	2.36
query30	0.57	0.53	0.52
query31	2.81	0.38	0.37
query32	3.40	0.50	0.51
query33	3.10	3.06	3.08
query34	15.27	4.80	4.77
query35	4.86	4.84	4.81
query36	1.06	1.00	1.00
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.17	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 100.84 s
Total hot run time: 31.14 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4f105fd0d894f36c7268fc6e6e0e8f420f2726e5 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49855 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 366fd2a8b0fff0b8084786e6413fedb8e669f665, data reload: false

------ Round 1 ----------------------------------
q1	17728	4398	4362	4362
q2	2067	153	147	147
q3	10289	1910	1962	1910
q4	10339	1207	1338	1207
q5	8465	3844	3933	3844
q6	233	125	125	125
q7	2069	1612	1619	1612
q8	9345	2735	2699	2699
q9	10950	10512	10484	10484
q10	8641	3492	3500	3492
q11	425	255	247	247
q12	468	314	315	314
q13	18362	3951	4043	3951
q14	359	327	330	327
q15	510	464	462	462
q16	681	570	566	566
q17	1127	962	977	962
q18	7329	6754	6872	6754
q19	1781	1590	1576	1576
q20	545	320	301	301
q21	4448	4072	4083	4072
q22	537	459	441	441
Total cold run time: 116698 ms
Total hot run time: 49855 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4328	4334	4309	4309
q2	326	227	223	223
q3	4192	4171	4099	4099
q4	2747	2768	2763	2763
q5	7262	7103	7132	7103
q6	233	120	122	120
q7	3186	2898	2849	2849
q8	4372	4458	4463	4458
q9	17368	17055	17107	17055
q10	4235	4271	4252	4252
q11	746	658	718	658
q12	1033	863	860	860
q13	7038	3762	3731	3731
q14	452	425	428	425
q15	496	457	452	452
q16	727	683	686	683
q17	3865	3788	3869	3788
q18	8682	8598	8747	8598
q19	1732	1713	1621	1621
q20	2365	2134	2110	2110
q21	8486	8521	8474	8474
q22	1009	944	965	944
Total cold run time: 84880 ms
Total hot run time: 79575 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 205517 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 366fd2a8b0fff0b8084786e6413fedb8e669f665, data reload: false

query1	942	395	419	395
query2	6574	2655	2610	2610
query3	6925	208	200	200
query4	21184	18066	18015	18015
query5	19726	6487	6519	6487
query6	292	221	233	221
query7	4162	309	318	309
query8	396	445	381	381
query9	3174	2732	2671	2671
query10	434	293	297	293
query11	11431	10784	10785	10784
query12	115	86	78	78
query13	5603	715	697	697
query14	18253	13860	13792	13792
query15	362	242	237	237
query16	6457	282	262	262
query17	1550	1457	880	880
query18	2271	419	415	415
query19	206	149	152	149
query20	78	80	82	80
query21	186	100	104	100
query22	5350	5040	5066	5040
query23	32646	32011	32095	32011
query24	6902	6475	6472	6472
query25	528	440	422	422
query26	531	168	169	168
query27	1865	306	303	303
query28	6103	2381	2343	2343
query29	2974	2825	2784	2784
query30	243	177	168	168
query31	920	745	788	745
query32	73	66	64	64
query33	401	261	241	241
query34	848	483	481	481
query35	1102	975	974	974
query36	1481	1123	1287	1123
query37	92	62	61	61
query38	3113	2914	2944	2914
query39	1374	1296	1326	1296
query40	205	93	101	93
query41	50	49	44	44
query42	83	78	86	78
query43	698	680	803	680
query44	1199	721	724	721
query45	254	242	240	240
query46	1238	991	964	964
query47	1947	1717	1600	1600
query48	1030	719	716	716
query49	613	369	377	369
query50	853	602	596	596
query51	4818	4722	4657	4657
query52	82	87	76	76
query53	459	335	330	330
query54	2672	2491	2505	2491
query55	91	94	89	89
query56	217	216	219	216
query57	1295	1202	1152	1152
query58	227	216	208	208
query59	3959	4011	3811	3811
query60	214	214	209	209
query61	101	99	99	99
query62	881	526	503	503
query63	485	345	352	345
query64	2575	1569	1486	1486
query65	3655	3555	3612	3555
query66	823	389	390	389
query67	16055	15807	15729	15729
query68	8518	674	657	657
query69	581	353	375	353
query70	1660	1551	1511	1511
query71	397	318	323	318
query72	6560	3536	3514	3514
query73	748	323	324	323
query74	6242	5836	5898	5836
query75	4660	3661	3712	3661
query76	4835	1140	1199	1140
query77	671	257	255	255
query78	12721	12780	11983	11983
query79	10004	658	681	658
query80	757	409	414	409
query81	498	240	233	233
query82	550	99	101	99
query83	168	133	134	133
query84	260	71	73	71
query85	792	338	344	338
query86	338	315	285	285
query87	3299	2990	3058	2990
query88	4450	2348	2350	2348
query89	427	282	305	282
query90	1959	213	226	213
query91	175	148	138	138
query92	63	54	54	54
query93	4950	553	625	553
query94	762	221	215	215
query95	1139	1099	1056	1056
query96	666	324	343	324
query97	6697	6502	6397	6397
query98	198	171	175	171
query99	2919	947	887	887
Total cold run time: 312310 ms
Total hot run time: 205517 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.13 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 366fd2a8b0fff0b8084786e6413fedb8e669f665, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.02	0.03
query3	0.24	0.04	0.04
query4	1.80	0.07	0.08
query5	0.54	0.52	0.52
query6	1.24	0.62	0.62
query7	0.01	0.01	0.01
query8	0.04	0.03	0.02
query9	0.53	0.49	0.49
query10	0.54	0.53	0.55
query11	0.12	0.09	0.10
query12	0.11	0.09	0.09
query13	0.63	0.60	0.61
query14	0.78	0.79	0.77
query15	0.77	0.76	0.78
query16	0.35	0.38	0.37
query17	1.02	1.04	1.01
query18	0.24	0.24	0.26
query19	1.94	1.84	1.84
query20	0.01	0.00	0.01
query21	15.43	0.58	0.55
query22	2.18	2.46	1.45
query23	17.48	1.06	0.95
query24	5.88	0.60	1.18
query25	0.37	0.09	0.07
query26	0.64	0.15	0.15
query27	0.05	0.05	0.03
query28	7.89	0.76	0.73
query29	12.78	2.36	2.11
query30	0.60	0.53	0.54
query31	2.80	0.39	0.37
query32	3.39	0.50	0.50
query33	3.07	3.06	3.04
query34	15.26	4.82	4.80
query35	4.89	4.87	4.89
query36	1.06	1.02	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.15	0.15
query41	0.06	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 105.14 s
Total hot run time: 30.13 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 366fd2a8b0fff0b8084786e6413fedb8e669f665 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@englefly englefly merged commit c9813c8 into apache:branch-2.0 Aug 14, 2024
23 of 25 checks passed
Jibing-Li pushed a commit to Jibing-Li/incubator-doris that referenced this pull request Aug 26, 2024
@englefly @Jibing-Li
[feat](nereids) adjust stats derive by delta row (apache#39185)
7062650 
apache#39222
After analyzing, user may insert new rows. 
analyzed rows: the rows have been analyzed
delta row: rows inserted after analyze job

if analyzed rows are filtered out, then we try to estimate filter result
by delta row with unknown column stats.
  

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Aug 27, 2024
@englefly
[feat](nereids) adjust stats derive by delta row (apache#39185)
6ec4343 
apache#39222
After analyzing, user may insert new rows.
analyzed rows: the rows have been analyzed
delta row: rows inserted after analyze job

if analyzed rows are filtered out, then we try to estimate filter result
by delta row with unknown column stats.

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/nereids dev/2.0.15-merged kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@englefly @doris-robot @morrySnow