Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bugfix](iceberg)Fixed missing type of iceberg table for timetravel #36047

Merged
merged 1 commit into from
Jun 12, 2024

Conversation

wuwenchi
Copy link
Contributor

@wuwenchi wuwenchi commented Jun 7, 2024

Proposed changes

Related: #34681

34681 supports the time function, but supports the type HMSExternalTable.

This PR add support for IcebergExternalTable type.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@wuwenchi
Copy link
Contributor Author

wuwenchi commented Jun 7, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40021 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 560cbff3ad276a94a184ed436185ef379f38c3c0, data reload: false

------ Round 1 ----------------------------------
q1	17662	4392	4241	4241
q2	2026	193	191	191
q3	10710	1336	1284	1284
q4	10588	786	859	786
q5	7646	2697	2684	2684
q6	225	134	134	134
q7	954	596	610	596
q8	9428	2096	2098	2096
q9	9320	6627	6617	6617
q10	9483	3699	3681	3681
q11	447	247	234	234
q12	434	228	221	221
q13	17783	2960	2963	2960
q14	266	222	221	221
q15	530	477	479	477
q16	497	385	379	379
q17	963	706	658	658
q18	8117	7441	7425	7425
q19	4374	1360	1455	1360
q20	665	310	304	304
q21	4907	3153	3295	3153
q22	377	327	319	319
Total cold run time: 117402 ms
Total hot run time: 40021 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4362	4240	4195	4195
q2	362	251	268	251
q3	2979	2707	2746	2707
q4	1867	1584	1562	1562
q5	5241	5248	5236	5236
q6	216	125	129	125
q7	2087	1681	1733	1681
q8	3182	3291	3280	3280
q9	8337	8326	8332	8326
q10	3844	3668	3635	3635
q11	593	477	480	477
q12	745	592	593	592
q13	16306	2978	2997	2978
q14	271	272	277	272
q15	518	473	480	473
q16	489	405	419	405
q17	1780	1492	1486	1486
q18	7586	7661	7430	7430
q19	2290	1543	1570	1543
q20	1949	1765	1738	1738
q21	4712	4760	4580	4580
q22	595	515	523	515
Total cold run time: 70311 ms
Total hot run time: 53487 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172408 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 560cbff3ad276a94a184ed436185ef379f38c3c0, data reload: false

query1	914	371	372	371
query2	6446	2325	2355	2325
query3	6650	206	213	206
query4	19095	17542	16976	16976
query5	4171	449	466	449
query6	248	160	166	160
query7	4596	293	294	293
query8	302	280	282	280
query9	8854	2427	2402	2402
query10	450	309	269	269
query11	10487	9963	9976	9963
query12	140	89	84	84
query13	1643	358	359	358
query14	9433	7623	7642	7623
query15	234	193	186	186
query16	7819	264	252	252
query17	1778	546	552	546
query18	1957	286	277	277
query19	192	152	151	151
query20	91	91	85	85
query21	216	133	129	129
query22	4199	4197	3968	3968
query23	33557	32918	32959	32918
query24	12012	2786	2837	2786
query25	675	356	354	354
query26	1766	155	162	155
query27	3015	320	334	320
query28	7607	2025	2026	2025
query29	1084	607	614	607
query30	275	150	147	147
query31	954	728	721	721
query32	92	55	55	55
query33	785	301	275	275
query34	966	483	463	463
query35	734	606	617	606
query36	1127	947	944	944
query37	186	70	74	70
query38	2866	2709	2713	2709
query39	876	771	815	771
query40	283	125	124	124
query41	56	52	50	50
query42	129	97	102	97
query43	601	534	551	534
query44	1191	725	744	725
query45	192	166	160	160
query46	1069	700	726	700
query47	1840	1746	1771	1746
query48	374	289	300	289
query49	1197	415	434	415
query50	789	422	391	391
query51	6775	6618	6570	6570
query52	107	90	97	90
query53	353	291	285	285
query54	958	452	432	432
query55	74	72	73	72
query56	277	254	252	252
query57	1140	1068	1065	1065
query58	257	252	273	252
query59	3471	3027	3249	3027
query60	300	301	279	279
query61	91	92	88	88
query62	637	450	478	450
query63	319	292	300	292
query64	9859	2206	1713	1713
query65	3196	3098	3159	3098
query66	1366	341	334	334
query67	15661	14869	15168	14869
query68	4634	549	536	536
query69	437	307	316	307
query70	1165	1163	1129	1129
query71	392	280	282	280
query72	7112	5524	6105	5524
query73	753	327	321	321
query74	5804	5445	5453	5445
query75	3380	2681	2682	2681
query76	2534	886	865	865
query77	439	303	306	303
query78	10289	10008	9779	9779
query79	2226	516	533	516
query80	2037	463	470	463
query81	563	219	217	217
query82	1390	102	104	102
query83	284	170	174	170
query84	265	87	87	87
query85	1583	272	263	263
query86	498	314	295	295
query87	3255	3096	3114	3096
query88	3971	2417	2333	2333
query89	479	391	385	385
query90	1808	192	195	192
query91	126	101	98	98
query92	63	50	52	50
query93	2243	510	498	498
query94	1225	192	183	183
query95	397	308	308	308
query96	596	263	266	263
query97	3176	3016	3045	3016
query98	229	205	197	197
query99	1117	828	840	828
Total cold run time: 275524 ms
Total hot run time: 172408 ms

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Jun 7, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 7, 2024
Copy link
Contributor

github-actions bot commented Jun 7, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

ClickBench: Total hot run time: 30.69 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 560cbff3ad276a94a184ed436185ef379f38c3c0, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.70	0.07	0.08
query5	0.51	0.49	0.49
query6	1.13	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.53	0.51	0.47
query10	0.54	0.57	0.54
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.58	0.59	0.59
query14	0.81	0.81	0.76
query15	0.84	0.82	0.82
query16	0.37	0.37	0.37
query17	0.98	0.97	1.00
query18	0.20	0.26	0.23
query19	1.90	1.83	1.71
query20	0.02	0.01	0.01
query21	15.45	0.73	0.67
query22	4.65	6.62	2.11
query23	18.31	1.38	1.24
query24	2.09	0.22	0.22
query25	0.16	0.09	0.09
query26	0.26	0.17	0.18
query27	0.08	0.08	0.09
query28	13.29	1.02	1.00
query29	12.76	3.30	3.27
query30	0.26	0.05	0.05
query31	2.89	0.39	0.38
query32	3.28	0.47	0.46
query33	2.86	2.92	2.89
query34	17.31	4.46	4.40
query35	4.58	4.46	4.54
query36	0.65	0.47	0.47
query37	0.18	0.15	0.15
query38	0.14	0.14	0.15
query39	0.04	0.04	0.03
query40	0.15	0.15	0.14
query41	0.08	0.05	0.04
query42	0.06	0.05	0.06
query43	0.03	0.03	0.04
Total cold run time: 110.38 s
Total hot run time: 30.69 s

@morrySnow
Copy link
Contributor

do we need to add test case?

@wuwenchi
Copy link
Contributor Author

do we need to add test case?

@morrySnow There is a related case in the regression test, which was previously in a failed state, but has been muted. After this PR is merged, the case can be run successfully.

@morrySnow morrySnow merged commit f0971c3 into apache:master Jun 12, 2024
28 of 30 checks passed
dataroaring pushed a commit that referenced this pull request Jun 13, 2024
…36047)

Related:  #34681

#34681 supports the time function, but supports the type
`HMSExternalTable`.

This PR add support for `IcebergExternalTable` type.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.4-merged dev/3.0.0-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants