Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](restore) Fix clean restore with view #40620 #41185

Merged
merged 1 commit into from
Sep 24, 2024
Merged

[fix](restore) Fix clean restore with view #40620 #41185

merged 1 commit into from
Sep 24, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Sep 24, 2024

cherry pick from #40620

@w41ter
[fix](restore) Fix clean restore with view (apache#40620)
a43fcbd 
While the previous impl only considered OLAP tables, restore included
ODBC and Views, both of which were not recorded in
backupOlapTableObjects and were accidentally deleted.
@w41ter
Copy link
Contributor Author

w41ter commented Sep 24, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@doris-robot
Copy link

TPC-H: Total hot run time: 49312 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a43fcbd98981266e4bba2f64305341fafe8fed01, data reload: false

------ Round 1 ----------------------------------
q1	17857	4415	4399	4399
q2	2064	154	146	146
q3	10449	1906	1950	1906
q4	10331	1305	1339	1305
q5	8526	3890	3925	3890
q6	233	122	124	122
q7	2063	1620	1596	1596
q8	9336	2778	2731	2731
q9	10119	9950	9814	9814
q10	8605	3540	3554	3540
q11	421	238	247	238
q12	474	302	307	302
q13	18371	4011	4002	4002
q14	355	328	334	328
q15	518	462	456	456
q16	541	466	458	458
q17	1139	964	923	923
q18	7269	6926	6829	6829
q19	1716	1570	1522	1522
q20	523	324	307	307
q21	4481	4128	4119	4119
q22	508	413	379	379
Total cold run time: 115899 ms
Total hot run time: 49312 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4322	4303	4314	4303
q2	333	226	225	225
q3	4191	4119	4152	4119
q4	2756	2726	2739	2726
q5	7185	7149	7093	7093
q6	238	120	119	119
q7	3269	2888	2828	2828
q8	4392	4506	4541	4506
q9	13629	13655	13594	13594
q10	4257	4270	4256	4256
q11	769	693	700	693
q12	1028	843	849	843
q13	7043	3690	3779	3690
q14	459	430	424	424
q15	489	468	457	457
q16	644	585	599	585
q17	3813	3800	3878	3800
q18	8789	8705	8744	8705
q19	1720	1707	1690	1690
q20	2380	2122	2118	2118
q21	8667	8528	8444	8444
q22	1035	909	930	909
Total cold run time: 81408 ms
Total hot run time: 76127 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212028 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a43fcbd98981266e4bba2f64305341fafe8fed01, data reload: false

query1	932	397	406	397
query2	6550	2179	2062	2062
query3	6929	207	200	200
query4	23642	21806	21498	21498
query5	19764	6459	6513	6459
query6	294	218	224	218
query7	4334	298	307	298
query8	253	244	241	241
query9	3065	2665	2588	2588
query10	472	313	288	288
query11	15844	15063	15160	15063
query12	136	72	74	72
query13	1020	434	436	434
query14	17107	13293	13735	13293
query15	363	233	231	231
query16	6477	278	262	262
query17	1945	960	905	905
query18	893	319	311	311
query19	212	147	151	147
query20	105	98	98	98
query21	187	101	95	95
query22	5180	5047	4967	4967
query23	34280	33417	33526	33417
query24	7909	6324	6345	6324
query25	522	435	423	423
query26	1262	164	168	164
query27	2412	298	291	291
query28	6097	2293	2237	2237
query29	2912	2851	2800	2800
query30	247	169	168	168
query31	945	775	729	729
query32	69	63	55	55
query33	475	260	264	260
query34	860	475	478	475
query35	1123	952	942	942
query36	1270	1271	1141	1141
query37	178	60	59	59
query38	3100	2912	2939	2912
query39	1389	1326	1323	1323
query40	314	96	93	93
query41	41	36	37	36
query42	89	91	89	89
query43	640	554	578	554
query44	1165	729	718	718
query45	244	234	226	226
query46	1248	948	965	948
query47	2045	1835	1809	1809
query48	534	411	405	405
query49	659	384	366	366
query50	850	590	596	590
query51	4777	4719	4672	4672
query52	97	78	79	78
query53	237	190	195	190
query54	2665	2459	2446	2446
query55	94	84	83	83
query56	240	216	200	200
query57	1242	1165	1194	1165
query58	221	212	211	211
query59	3658	3180	3346	3180
query60	222	209	214	209
query61	95	95	94	94
query62	839	555	523	523
query63	201	171	176	171
query64	5751	1609	1388	1388
query65	3623	3564	3550	3550
query66	610	397	406	397
query67	15917	17157	15134	15134
query68	10611	667	671	667
query69	508	278	270	270
query70	1808	1348	1321	1321
query71	411	311	308	308
query72	6819	4651	4824	4651
query73	768	337	331	331
query74	6208	5867	5839	5839
query75	5391	3700	3691	3691
query76	6376	1120	1194	1120
query77	1059	252	257	252
query78	12608	12304	11560	11560
query79	11790	649	650	649
query80	795	385	405	385
query81	492	240	236	236
query82	1558	100	99	99
query83	180	137	131	131
query84	257	71	69	69
query85	883	318	321	318
query86	332	301	296	296
query87	3265	3035	3046	3035
query88	4966	2294	2293	2293
query89	483	310	306	306
query90	2206	215	215	215
query91	161	128	125	125
query92	55	53	60	53
query93	7098	628	534	534
query94	961	213	208	208
query95	1968	2047	2002	2002
query96	639	329	324	324
query97	6411	6313	6290	6290
query98	216	227	215	215
query99	2958	881	863	863
Total cold run time: 329313 ms
Total hot run time: 212028 ms

@w41ter w41ter merged commit d5d5f8f into apache:branch-2.0 Sep 24, 2024
21 of 24 checks passed
@w41ter w41ter deleted the pick_40620_to_origin_branch-2.0 branch September 24, 2024 03:14
@doris-robot
Copy link

ClickBench: Total hot run time: 30.86 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a43fcbd98981266e4bba2f64305341fafe8fed01, data reload: false

query1	0.02	0.02	0.01
query2	0.07	0.03	0.02
query3	0.25	0.04	0.04
query4	1.81	0.06	0.06
query5	0.54	0.53	0.52
query6	1.24	0.66	0.62
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.54	0.49	0.49
query10	0.54	0.54	0.53
query11	0.12	0.09	0.09
query12	0.12	0.09	0.10
query13	0.62	0.64	0.60
query14	0.77	0.79	0.79
query15	0.79	0.77	0.77
query16	0.39	0.36	0.37
query17	1.02	1.02	0.98
query18	0.22	0.24	0.25
query19	1.95	1.81	1.88
query20	0.01	0.01	0.02
query21	15.46	0.56	0.53
query22	2.15	2.14	1.93
query23	17.23	0.98	0.94
query24	3.62	0.89	1.84
query25	0.33	0.10	0.06
query26	0.52	0.15	0.15
query27	0.05	0.04	0.03
query28	9.12	0.72	0.71
query29	12.63	2.17	2.33
query30	0.59	0.53	0.53
query31	2.81	0.38	0.38
query32	3.38	0.51	0.50
query33	3.03	3.12	3.10
query34	15.25	4.77	4.80
query35	4.85	4.83	4.83
query36	1.05	1.02	1.01
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.15
query41	0.06	0.02	0.01
query42	0.02	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 103.53 s
Total hot run time: 30.86 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit a43fcbd98981266e4bba2f64305341fafe8fed01 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.1 seconds inserted 10000000 Rows, about 452K ops/s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@w41ter @doris-robot