Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[enhance](mtmv)support create mtmv with other mtmv #32984

Merged
merged 5 commits into from
Apr 3, 2024
Merged

[enhance](mtmv)support create mtmv with other mtmv #32984

merged 5 commits into from
Apr 3, 2024

Conversation

zddr
Copy link
Contributor

@zddr zddr commented Mar 28, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zddr
Copy link
Contributor Author

zddr commented Mar 28, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38046 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0f2aa930b3cb0100de895c1bf604080ef08f5d65, data reload: false

------ Round 1 ----------------------------------
q1	17911	4296	4276	4276
q2	2438	169	159	159
q3	11201	1124	1233	1124
q4	10556	800	795	795
q5	7453	2988	2946	2946
q6	207	125	124	124
q7	1041	583	590	583
q8	9340	1974	2015	1974
q9	6887	6351	6314	6314
q10	8368	3502	3574	3502
q11	422	227	213	213
q12	366	202	193	193
q13	17791	2885	2897	2885
q14	243	206	203	203
q15	515	473	464	464
q16	505	375	366	366
q17	959	537	605	537
q18	7688	6490	6576	6490
q19	1546	1469	1414	1414
q20	550	272	254	254
q21	3538	2939	2935	2935
q22	343	295	299	295
Total cold run time: 109868 ms
Total hot run time: 38046 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4072	4060	4074	4060
q2	332	233	231	231
q3	2995	2843	2860	2843
q4	1893	1589	1582	1582
q5	5188	5254	5261	5254
q6	199	116	119	116
q7	2235	1825	1851	1825
q8	3193	3327	3285	3285
q9	8484	8477	8495	8477
q10	3775	3834	3778	3778
q11	533	451	448	448
q12	750	560	554	554
q13	15333	2883	2841	2841
q14	292	259	252	252
q15	501	472	470	470
q16	481	428	422	422
q17	1726	1512	1472	1472
q18	7358	7198	7067	7067
q19	1616	1509	1517	1509
q20	1943	1744	1736	1736
q21	4700	4547	4706	4547
q22	522	459	471	459
Total cold run time: 68121 ms
Total hot run time: 53228 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 182357 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0f2aa930b3cb0100de895c1bf604080ef08f5d65, data reload: false

query1	929	390	364	364
query2	6538	1992	1907	1907
query3	6709	215	210	210
query4	32261	21819	21428	21428
query5	4328	420	397	397
query6	274	182	172	172
query7	4628	300	295	295
query8	224	171	172	171
query9	9031	2308	2311	2308
query10	558	246	273	246
query11	17421	14270	14271	14270
query12	144	94	90	90
query13	1643	431	437	431
query14	11260	7930	7807	7807
query15	261	192	190	190
query16	8264	258	260	258
query17	1989	554	537	537
query18	2112	296	290	290
query19	346	153	147	147
query20	90	86	87	86
query21	197	124	131	124
query22	4973	4780	4792	4780
query23	33451	32976	32723	32723
query24	11885	2869	2805	2805
query25	627	370	369	369
query26	1831	153	151	151
query27	2996	316	325	316
query28	7920	1895	1836	1836
query29	1058	656	640	640
query30	311	162	159	159
query31	961	739	735	735
query32	92	59	56	56
query33	778	259	261	259
query34	1131	500	497	497
query35	904	705	693	693
query36	1015	919	866	866
query37	272	67	68	67
query38	3531	3442	3427	3427
query39	1573	1568	1530	1530
query40	288	108	113	108
query41	50	50	48	48
query42	105	94	100	94
query43	499	462	446	446
query44	1155	719	741	719
query45	272	268	254	254
query46	1126	721	702	702
query47	1930	1828	1872	1828
query48	444	377	355	355
query49	1245	337	341	337
query50	764	375	380	375
query51	6740	6636	6618	6618
query52	107	91	90	90
query53	345	278	281	278
query54	333	253	235	235
query55	82	78	85	78
query56	274	237	235	235
query57	1229	1100	1112	1100
query58	244	211	210	210
query59	2915	2665	2627	2627
query60	264	246	255	246
query61	115	112	112	112
query62	651	448	443	443
query63	318	282	279	279
query64	6468	4072	4304	4072
query65	3174	3058	3040	3040
query66	1492	385	376	376
query67	15763	14963	14883	14883
query68	8454	541	546	541
query69	655	381	378	378
query70	1245	1203	1189	1189
query71	504	288	271	271
query72	6452	2512	2358	2358
query73	760	312	323	312
query74	8006	6444	6374	6374
query75	4160	2239	2212	2212
query76	5121	875	915	875
query77	648	266	265	265
query78	11319	10372	10369	10369
query79	11198	527	519	519
query80	1327	373	380	373
query81	496	227	226	226
query82	233	84	88	84
query83	219	149	144	144
query84	290	82	78	78
query85	1088	314	310	310
query86	358	318	274	274
query87	3779	3533	3560	3533
query88	5408	2312	2334	2312
query89	505	382	373	373
query90	2117	178	180	178
query91	169	141	132	132
query92	64	46	49	46
query93	6316	504	489	489
query94	1347	182	178	178
query95	413	312	311	311
query96	643	276	271	271
query97	2741	2510	2540	2510
query98	230	221	211	211
query99	1054	840	836	836
Total cold run time: 320227 ms
Total hot run time: 182357 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 0f2aa930b3cb0100de895c1bf604080ef08f5d65 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.9 seconds inserted 10000000 Rows, about 719K ops/s

@zddr
Copy link
Contributor Author

zddr commented Mar 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37352 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e6c0b34406a25bdd65aeb6ff56d7e3baffe56968, data reload: false

------ Round 1 ----------------------------------
q1	17636	4218	4084	4084
q2	2107	156	149	149
q3	10590	1109	1155	1109
q4	10218	746	850	746
q5	7474	2885	2897	2885
q6	204	125	124	124
q7	1027	568	576	568
q8	9339	1984	1963	1963
q9	7186	6372	6304	6304
q10	8455	3460	3532	3460
q11	428	224	225	224
q12	410	201	191	191
q13	17789	2858	2876	2858
q14	240	208	206	206
q15	514	475	454	454
q16	483	376	378	376
q17	939	627	552	552
q18	7013	6513	6333	6333
q19	3180	1418	1467	1418
q20	545	248	245	245
q21	3571	2898	2812	2812
q22	331	291	303	291
Total cold run time: 109679 ms
Total hot run time: 37352 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4127	4102	4048	4048
q2	325	235	237	235
q3	2953	2841	2811	2811
q4	1843	1586	1550	1550
q5	5219	5217	5229	5217
q6	198	116	117	116
q7	2199	1817	1797	1797
q8	3167	3300	3295	3295
q9	8305	8366	8369	8366
q10	3789	3755	3999	3755
q11	574	451	455	451
q12	769	596	588	588
q13	17172	3064	3024	3024
q14	320	312	285	285
q15	517	471	479	471
q16	518	466	458	458
q17	1785	1551	1498	1498
q18	7929	7547	7536	7536
q19	1727	1535	1582	1535
q20	2005	1850	1827	1827
q21	4941	4842	4749	4749
q22	532	446	437	437
Total cold run time: 70914 ms
Total hot run time: 54049 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 182859 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e6c0b34406a25bdd65aeb6ff56d7e3baffe56968, data reload: false

query1	934	358	340	340
query2	6363	2141	1956	1956
query3	6692	217	223	217
query4	31838	21359	21272	21272
query5	4257	402	400	400
query6	267	176	177	176
query7	4618	296	287	287
query8	236	176	179	176
query9	9042	2294	2300	2294
query10	431	257	273	257
query11	17131	14176	14243	14176
query12	132	88	83	83
query13	1618	436	414	414
query14	9806	7856	7673	7673
query15	236	177	195	177
query16	7899	262	261	261
query17	1951	586	560	560
query18	1593	290	291	290
query19	252	168	163	163
query20	95	94	89	89
query21	209	133	123	123
query22	4988	4782	4756	4756
query23	33503	32694	33122	32694
query24	11150	3009	2945	2945
query25	624	416	417	416
query26	1001	164	163	163
query27	2959	363	385	363
query28	7559	1971	1941	1941
query29	900	643	651	643
query30	311	159	161	159
query31	1046	786	747	747
query32	103	88	58	58
query33	796	270	256	256
query34	1331	518	536	518
query35	935	726	737	726
query36	1052	909	918	909
query37	124	69	70	69
query38	3667	3648	3598	3598
query39	1621	1657	1590	1590
query40	171	111	117	111
query41	50	48	49	48
query42	108	99	101	99
query43	485	472	458	458
query44	1277	772	764	764
query45	289	261	273	261
query46	1156	717	743	717
query47	2009	1941	1919	1919
query48	479	370	371	370
query49	987	356	361	356
query50	793	402	403	402
query51	6976	6739	6806	6739
query52	109	89	95	89
query53	356	276	291	276
query54	301	230	238	230
query55	89	79	87	79
query56	252	248	228	228
query57	1269	1231	1222	1222
query58	248	212	223	212
query59	2969	2651	2602	2602
query60	261	237	231	231
query61	95	93	92	92
query62	605	428	429	428
query63	303	271	277	271
query64	5255	4115	4007	4007
query65	3047	3041	3036	3036
query66	784	357	352	352
query67	15456	14795	14872	14795
query68	8718	527	553	527
query69	624	371	364	364
query70	1239	1126	1150	1126
query71	508	269	274	269
query72	6341	2543	2387	2387
query73	1064	316	323	316
query74	6783	6382	6381	6381
query75	3632	2200	2191	2191
query76	5391	881	928	881
query77	613	269	272	269
query78	11295	10480	10280	10280
query79	9885	533	542	533
query80	2086	376	381	376
query81	534	223	213	213
query82	764	90	79	79
query83	209	145	145	145
query84	279	76	76	76
query85	1382	341	302	302
query86	411	293	313	293
query87	3755	3567	3572	3567
query88	4763	2402	2398	2398
query89	488	370	375	370
query90	1907	174	175	174
query91	169	137	158	137
query92	62	46	46	46
query93	6490	506	489	489
query94	1079	179	176	176
query95	405	305	311	305
query96	604	273	275	273
query97	2705	2524	2508	2508
query98	238	230	221	221
query99	1163	858	867	858
Total cold run time: 311035 ms
Total hot run time: 182859 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit e6c0b34406a25bdd65aeb6ff56d7e3baffe56968 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.9 seconds inserted 10000000 Rows, about 719K ops/s

@zddr
Copy link
Contributor Author

zddr commented Mar 29, 2024

run p0

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 2, 2024

PR approved by anyone and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 2, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 2, 2024
@morrySnow morrySnow merged commit cc4f446 into apache:master Apr 3, 2024
29 of 30 checks passed
yiguolei pushed a commit that referenced this pull request Apr 10, 2024
@zddr @Doris-Extras
[enhance](mtmv)support create mtmv with other mtmv (#32984)
0ab8b57
@yiguolei yiguolei mentioned this pull request Apr 26, 2024
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@seawinde seawinde seawinde approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@zddr @doris-robot @morrySnow @seawinde @morningman