Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](tvf) fix FE cannot start when replay alter view from tvf (#40872) #41159

Merged
merged 2 commits into from
Sep 24, 2024

Conversation

morningman
Copy link
Contributor

bp #40872

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49326 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c29e4145128b58ccc82b8187cb7c518de3192c2a, data reload: false

------ Round 1 ----------------------------------
q1	17517	4483	4395	4395
q2	1936	189	149	149
q3	10187	1904	1954	1904
q4	10070	1302	1350	1302
q5	8658	3854	3946	3854
q6	234	124	123	123
q7	2054	1589	1594	1589
q8	9333	2751	2736	2736
q9	10407	9916	9950	9916
q10	8627	3594	3574	3574
q11	423	258	255	255
q12	463	299	300	299
q13	18263	3939	4050	3939
q14	354	325	323	323
q15	521	467	462	462
q16	517	463	468	463
q17	1133	957	917	917
q18	7294	6830	6947	6830
q19	1713	1575	1542	1542
q20	532	299	306	299
q21	4505	4189	4067	4067
q22	514	409	388	388
Total cold run time: 115255 ms
Total hot run time: 49326 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4367	4301	4357	4301
q2	329	231	222	222
q3	4219	4170	4137	4137
q4	2748	2754	2769	2754
q5	7234	7142	7122	7122
q6	240	120	120	120
q7	3289	2889	2906	2889
q8	4369	4450	4537	4450
q9	13760	13604	13619	13604
q10	4247	4292	4246	4246
q11	792	683	682	682
q12	1031	860	863	860
q13	4209	3762	3749	3749
q14	464	428	425	425
q15	501	478	457	457
q16	650	596	585	585
q17	3869	3828	3836	3828
q18	8835	8879	8727	8727
q19	1740	1636	1694	1636
q20	2416	2168	2111	2111
q21	8461	8453	8460	8453
q22	1047	953	921	921
Total cold run time: 78817 ms
Total hot run time: 76279 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212658 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c29e4145128b58ccc82b8187cb7c518de3192c2a, data reload: false

query1	929	394	414	394
query2	6554	2165	2231	2165
query3	6927	210	199	199
query4	23321	21449	21860	21449
query5	19724	6568	6614	6568
query6	298	221	232	221
query7	4328	297	310	297
query8	248	280	236	236
query9	3121	2710	2618	2618
query10	469	326	307	307
query11	15736	15162	15000	15000
query12	125	78	71	71
query13	1049	439	438	438
query14	17256	13064	13492	13064
query15	379	226	232	226
query16	5698	270	260	260
query17	1764	910	906	906
query18	901	315	328	315
query19	202	151	152	151
query20	109	96	95	95
query21	192	98	95	95
query22	5211	5159	5072	5072
query23	34554	33691	33432	33432
query24	7882	6384	6300	6300
query25	529	440	436	436
query26	1376	167	169	167
query27	2536	298	295	295
query28	6007	2276	2259	2259
query29	2852	2673	2596	2596
query30	247	178	166	166
query31	974	769	743	743
query32	71	65	59	59
query33	453	257	261	257
query34	896	473	466	466
query35	1130	917	918	917
query36	1385	1128	1225	1128
query37	172	58	60	58
query38	3062	2909	2925	2909
query39	1374	1334	1350	1334
query40	311	99	97	97
query41	40	38	36	36
query42	82	93	90	90
query43	636	575	649	575
query44	1172	723	715	715
query45	244	235	233	233
query46	1247	982	957	957
query47	1785	1767	1777	1767
query48	508	422	412	412
query49	656	382	387	382
query50	879	622	640	622
query51	4793	4645	4713	4645
query52	89	84	74	74
query53	230	194	188	188
query54	2666	2476	2497	2476
query55	86	88	81	81
query56	230	203	203	203
query57	1318	1213	1114	1114
query58	224	219	223	219
query59	3518	3383	3418	3383
query60	212	205	212	205
query61	96	94	102	94
query62	883	459	457	457
query63	203	180	176	176
query64	3672	1598	1488	1488
query65	3665	3531	3559	3531
query66	794	419	408	408
query67	16358	15597	15423	15423
query68	10918	653	654	653
query69	498	265	268	265
query70	1696	1512	1578	1512
query71	394	319	309	309
query72	6887	4833	4787	4787
query73	764	341	341	341
query74	6282	5820	5738	5738
query75	5333	3719	3638	3638
query76	6630	1149	1199	1149
query77	1107	254	256	254
query78	12441	11645	11929	11645
query79	7669	648	651	648
query80	1311	389	387	387
query81	479	242	233	233
query82	1699	97	100	97
query83	174	134	133	133
query84	252	72	69	69
query85	873	319	311	311
query86	334	336	287	287
query87	3207	3034	3033	3033
query88	4346	2302	2310	2302
query89	476	289	281	281
query90	1969	213	217	213
query91	158	125	134	125
query92	65	54	50	50
query93	6884	591	574	574
query94	702	206	212	206
query95	2050	1930	1981	1930
query96	661	329	334	329
query97	6516	6443	6428	6428
query98	230	199	211	199
query99	2731	921	812	812
Total cold run time: 322374 ms
Total hot run time: 212658 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.41 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c29e4145128b58ccc82b8187cb7c518de3192c2a, data reload: false

query1	0.01	0.02	0.02
query2	0.07	0.03	0.03
query3	0.24	0.04	0.06
query4	1.78	0.10	0.07
query5	0.54	0.51	0.52
query6	1.26	0.61	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.02
query9	0.52	0.48	0.48
query10	0.53	0.53	0.53
query11	0.12	0.09	0.09
query12	0.12	0.09	0.10
query13	0.63	0.60	0.62
query14	0.80	0.79	0.79
query15	0.78	0.76	0.76
query16	0.36	0.35	0.35
query17	1.01	1.02	0.99
query18	0.23	0.27	0.21
query19	1.91	1.76	1.85
query20	0.02	0.01	0.01
query21	15.46	0.58	0.54
query22	2.37	2.23	1.78
query23	17.03	1.10	0.85
query24	7.51	1.34	0.68
query25	0.38	0.11	0.05
query26	0.81	0.18	0.16
query27	0.05	0.03	0.04
query28	5.60	0.76	0.76
query29	12.64	2.29	2.25
query30	0.62	0.53	0.53
query31	2.80	0.39	0.38
query32	3.37	0.50	0.49
query33	3.07	3.09	3.05
query34	15.26	4.80	4.78
query35	4.86	4.84	4.86
query36	1.04	1.02	1.04
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.15	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.24 s
Total hot run time: 30.41 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c29e4145128b58ccc82b8187cb7c518de3192c2a with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49106 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 68124b67e5b3d209637f1e0f342ebe791063a86c, data reload: false

------ Round 1 ----------------------------------
q1	17630	4406	4401	4401
q2	2037	158	149	149
q3	10446	1870	1933	1870
q4	10303	1265	1340	1265
q5	8659	3873	3936	3873
q6	226	145	127	127
q7	2064	1582	1578	1578
q8	9305	2736	2719	2719
q9	10164	9800	9897	9800
q10	8652	3556	3502	3502
q11	423	247	249	247
q12	469	294	296	294
q13	18383	3997	4055	3997
q14	345	322	331	322
q15	514	461	452	452
q16	548	475	460	460
q17	1115	926	962	926
q18	7273	6957	6870	6870
q19	1645	1516	1489	1489
q20	520	309	317	309
q21	4442	4181	4064	4064
q22	495	397	392	392
Total cold run time: 115658 ms
Total hot run time: 49106 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4354	4310	4258	4258
q2	322	229	224	224
q3	4171	4126	4153	4126
q4	2730	2718	2725	2718
q5	7206	7053	7067	7053
q6	237	118	116	116
q7	3237	2856	2882	2856
q8	4341	4486	4468	4468
q9	13575	13615	13495	13495
q10	4270	4250	4236	4236
q11	763	685	708	685
q12	1006	842	855	842
q13	6428	3733	3714	3714
q14	465	424	418	418
q15	503	462	464	462
q16	632	589	589	589
q17	3838	3836	3895	3836
q18	8690	8797	8757	8757
q19	1713	1650	1674	1650
q20	2404	2085	2095	2085
q21	8431	8428	8411	8411
q22	988	995	930	930
Total cold run time: 80304 ms
Total hot run time: 75929 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 214267 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 68124b67e5b3d209637f1e0f342ebe791063a86c, data reload: false

query1	933	394	412	394
query2	6527	2210	2092	2092
query3	6920	199	198	198
query4	23823	21875	21542	21542
query5	19730	6536	6397	6397
query6	286	218	224	218
query7	4342	302	311	302
query8	266	259	256	256
query9	3091	2655	2583	2583
query10	471	299	300	299
query11	15518	15005	15174	15005
query12	131	73	75	73
query13	1039	438	432	432
query14	17076	13419	13568	13419
query15	368	220	230	220
query16	6483	281	256	256
query17	1769	924	878	878
query18	897	314	314	314
query19	216	148	156	148
query20	107	101	105	101
query21	191	97	92	92
query22	5232	5084	5135	5084
query23	34318	33508	33415	33415
query24	7893	6334	6354	6334
query25	528	426	441	426
query26	1260	163	160	160
query27	2467	300	294	294
query28	6170	2284	2235	2235
query29	2871	2723	2829	2723
query30	245	170	166	166
query31	964	725	737	725
query32	76	62	55	55
query33	456	258	267	258
query34	856	480	481	480
query35	1150	895	879	879
query36	1411	1135	1204	1135
query37	172	59	61	59
query38	3085	2938	2901	2901
query39	1375	1327	1316	1316
query40	300	93	97	93
query41	39	36	38	36
query42	81	86	84	84
query43	606	629	556	556
query44	1158	722	716	716
query45	248	231	230	230
query46	1247	970	957	957
query47	1897	1880	1856	1856
query48	510	411	418	411
query49	646	398	378	378
query50	853	603	571	571
query51	4761	4681	4670	4670
query52	84	81	85	81
query53	232	187	193	187
query54	2638	2487	2506	2487
query55	89	84	86	84
query56	230	202	213	202
query57	1314	1099	1078	1078
query58	224	208	199	199
query59	3596	3252	3207	3207
query60	217	206	199	199
query61	96	95	96	95
query62	856	468	459	459
query63	200	180	177	177
query64	3531	1622	1493	1493
query65	3577	3593	3550	3550
query66	776	417	407	407
query67	16419	16952	18089	16952
query68	8386	661	674	661
query69	476	279	299	279
query70	1832	1352	1402	1352
query71	400	311	292	292
query72	6725	4811	4860	4811
query73	743	329	328	328
query74	6199	5869	5850	5850
query75	4783	3698	3625	3625
query76	4469	1075	1180	1075
query77	545	256	251	251
query78	12345	12248	11808	11808
query79	13079	634	637	634
query80	2069	387	373	373
query81	538	237	236	236
query82	875	97	96	96
query83	180	134	131	131
query84	258	70	72	70
query85	1225	322	322	322
query86	361	296	297	296
query87	3248	3068	3004	3004
query88	5428	2298	2305	2298
query89	473	297	303	297
query90	1749	218	212	212
query91	167	130	135	130
query92	61	50	51	50
query93	5864	583	569	569
query94	935	210	210	210
query95	1990	1985	2014	1985
query96	657	325	323	323
query97	6456	6411	6328	6328
query98	224	221	205	205
query99	2983	887	1000	887
Total cold run time: 322861 ms
Total hot run time: 214267 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.09 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 68124b67e5b3d209637f1e0f342ebe791063a86c, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.05	0.04
query4	1.79	0.08	0.06
query5	0.53	0.52	0.52
query6	1.23	0.62	0.66
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.51	0.49	0.49
query10	0.54	0.54	0.53
query11	0.12	0.09	0.08
query12	0.12	0.09	0.09
query13	0.62	0.60	0.61
query14	0.78	0.78	0.80
query15	0.78	0.76	0.75
query16	0.36	0.37	0.36
query17	1.00	0.97	1.02
query18	0.23	0.26	0.20
query19	1.91	1.77	1.89
query20	0.02	0.01	0.01
query21	15.45	0.55	0.55
query22	2.06	2.18	2.22
query23	17.32	1.13	0.84
query24	7.73	1.32	0.89
query25	0.38	0.13	0.04
query26	0.80	0.17	0.16
query27	0.05	0.04	0.03
query28	5.40	0.78	0.76
query29	12.69	2.36	2.39
query30	0.56	0.53	0.52
query31	2.80	0.40	0.38
query32	3.40	0.50	0.50
query33	3.08	3.05	3.04
query34	15.27	4.79	4.80
query35	4.86	4.82	4.86
query36	1.04	1.01	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.2 s
Total hot run time: 31.09 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 68124b67e5b3d209637f1e0f342ebe791063a86c with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@morningman morningman merged commit 4560ff7 into apache:branch-2.0 Sep 24, 2024
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants