Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improve](jdbc catalog) Remove all property checks during create #35194

Merged
merged 1 commit into from
May 24, 2024
Merged

[improve](jdbc catalog) Remove all property checks during create #35194

merged 1 commit into from
May 24, 2024

Conversation

zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented May 22, 2024

Proposed changes

Issue Number: close #xxx

Previously, in order to prevent users from writing wrong parameters when creating a jdbc catalog, we checked all properties. However, we found that when the parent class adds a new property, the subclass cannot sense the parent class's newly added property and can only manually add a check list. In order to avoid this trouble, I will delete this check for now and will check it in a better way later.

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zy-kkk
Copy link
Member Author

zy-kkk commented May 22, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41979 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 335ee118138ef826405656d3517c34e69d7f4fb9, data reload: false

------ Round 1 ----------------------------------
q1	17607	4458	4231	4231
q2	2022	192	187	187
q3	10470	1287	1210	1210
q4	10191	774	841	774
q5	7518	2727	2727	2727
q6	216	132	136	132
q7	970	595	594	594
q8	9221	2165	2108	2108
q9	9099	6736	6652	6652
q10	9463	3922	3874	3874
q11	467	245	243	243
q12	463	228	223	223
q13	17250	3278	3196	3196
q14	268	220	223	220
q15	520	471	473	471
q16	508	391	392	391
q17	991	726	762	726
q18	8426	7908	7963	7908
q19	6597	1557	1500	1500
q20	650	312	322	312
q21	5159	4060	4020	4020
q22	349	291	280	280
Total cold run time: 118425 ms
Total hot run time: 41979 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4530	4451	4444	4444
q2	377	267	294	267
q3	3202	2988	2889	2889
q4	2007	1618	1579	1579
q5	5336	5487	5471	5471
q6	215	124	127	124
q7	2230	1831	1840	1831
q8	3211	3420	3401	3401
q9	8722	8643	8773	8643
q10	4132	3787	3709	3709
q11	584	499	486	486
q12	806	612	628	612
q13	16102	3187	3133	3133
q14	288	282	276	276
q15	519	493	473	473
q16	502	450	442	442
q17	1857	1528	1538	1528
q18	7910	7574	7331	7331
q19	1664	1548	1535	1535
q20	2011	1790	1771	1771
q21	5583	4817	4661	4661
q22	589	481	481	481
Total cold run time: 72377 ms
Total hot run time: 55087 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169324 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 335ee118138ef826405656d3517c34e69d7f4fb9, data reload: false

query1	920	376	369	369
query2	6438	2373	2282	2282
query3	6662	202	203	202
query4	19344	17397	17318	17318
query5	4154	421	406	406
query6	244	150	152	150
query7	4587	286	280	280
query8	245	181	180	180
query9	8554	2390	2382	2382
query10	457	269	264	264
query11	10623	9997	9986	9986
query12	131	91	90	90
query13	1649	365	352	352
query14	9998	7495	7473	7473
query15	253	167	169	167
query16	8001	274	260	260
query17	1696	535	512	512
query18	2072	277	318	277
query19	203	159	165	159
query20	90	81	85	81
query21	194	126	125	125
query22	4186	3900	3912	3900
query23	33882	32889	32928	32889
query24	10594	2911	2835	2835
query25	561	346	347	346
query26	694	149	152	149
query27	2200	316	328	316
query28	5887	2050	2049	2049
query29	864	612	586	586
query30	258	170	176	170
query31	958	778	767	767
query32	94	52	52	52
query33	660	265	253	253
query34	891	457	484	457
query35	717	617	584	584
query36	1082	919	897	897
query37	108	68	66	66
query38	2857	2750	2749	2749
query39	844	770	813	770
query40	191	126	125	125
query41	44	44	43	43
query42	100	96	93	93
query43	606	527	547	527
query44	1089	715	733	715
query45	180	161	163	161
query46	1075	701	707	701
query47	1832	1796	1756	1756
query48	368	285	299	285
query49	842	368	380	368
query50	768	378	392	378
query51	6967	6839	6706	6706
query52	104	87	96	87
query53	344	282	286	282
query54	862	421	411	411
query55	81	68	71	68
query56	252	240	262	240
query57	1134	1033	1029	1029
query58	233	211	208	208
query59	3397	3079	3083	3079
query60	289	244	245	244
query61	86	90	85	85
query62	611	447	456	447
query63	306	283	291	283
query64	8455	2222	1767	1767
query65	3141	3089	3089	3089
query66	789	320	330	320
query67	15357	14944	14741	14741
query68	4492	514	531	514
query69	435	266	260	260
query70	1107	1110	1074	1074
query71	422	269	263	263
query72	7595	5312	2767	2767
query73	713	317	310	310
query74	6194	5618	5555	5555
query75	3317	2628	2613	2613
query76	2495	1018	1093	1018
query77	381	271	258	258
query78	10350	9848	9737	9737
query79	2187	505	507	505
query80	1225	432	427	427
query81	557	250	245	245
query82	666	94	95	94
query83	258	171	166	166
query84	255	82	86	82
query85	2021	272	266	266
query86	504	312	300	300
query87	3340	3293	3124	3124
query88	4211	2347	2351	2347
query89	498	398	381	381
query90	2064	183	188	183
query91	120	98	94	94
query92	62	48	49	48
query93	2574	505	484	484
query94	1283	183	181	181
query95	389	298	304	298
query96	591	264	264	264
query97	3169	3038	3008	3008
query98	246	226	219	219
query99	1111	875	851	851
Total cold run time: 268187 ms
Total hot run time: 169324 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.54 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 335ee118138ef826405656d3517c34e69d7f4fb9, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.03	0.04
query3	0.24	0.05	0.05
query4	1.66	0.06	0.07
query5	0.48	0.47	0.50
query6	1.12	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.48
query10	0.55	0.54	0.54
query11	0.15	0.11	0.12
query12	0.15	0.12	0.11
query13	0.60	0.60	0.60
query14	0.79	0.77	0.77
query15	0.84	0.81	0.81
query16	0.36	0.36	0.38
query17	0.97	0.95	0.94
query18	0.21	0.26	0.25
query19	1.86	1.68	1.70
query20	0.02	0.01	0.02
query21	15.60	0.66	0.64
query22	4.63	6.91	1.98
query23	18.31	1.36	1.25
query24	1.38	0.38	0.25
query25	0.15	0.08	0.10
query26	0.28	0.18	0.16
query27	0.08	0.08	0.08
query28	13.37	1.04	0.99
query29	13.76	3.30	3.26
query30	0.24	0.05	0.06
query31	2.85	0.38	0.37
query32	3.30	0.46	0.46
query33	2.84	2.92	2.90
query34	17.25	4.46	4.46
query35	4.66	4.50	4.68
query36	0.64	0.46	0.46
query37	0.18	0.16	0.15
query38	0.14	0.14	0.15
query39	0.04	0.03	0.04
query40	0.17	0.14	0.15
query41	0.09	0.05	0.05
query42	0.06	0.04	0.04
query43	0.05	0.03	0.04
Total cold run time: 110.8 s
Total hot run time: 30.54 s

morningman
morningman approved these changes May 23, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@zy-kkk zy-kkk merged commit 45a0d2f into apache:master May 24, 2024
29 of 31 checks passed
@zy-kkk zy-kkk deleted the jdbc_resources branch May 24, 2024 08:06
zy-kkk added a commit to zy-kkk/doris that referenced this pull request May 24, 2024
@zy-kkk
[improve](jdbc catalog) Remove all property checks during create (apa…
6ead165 
…che#35194)
@zy-kkk zy-kkk mentioned this pull request May 24, 2024
Merged
zy-kkk added a commit to zy-kkk/doris that referenced this pull request May 24, 2024
@zy-kkk
[improve](jdbc catalog) Remove all property checks during create (apa…
3f06f60 
…che#35194)
@zy-kkk zy-kkk mentioned this pull request May 24, 2024
Merged
zy-kkk added a commit that referenced this pull request May 24, 2024
@zy-kkk
[improve](jdbc catalog) Remove all property checks during create (#35194
cf46ebe 
) (#35354)
zy-kkk added a commit that referenced this pull request May 25, 2024
@zy-kkk
[improve](jdbc catalog) Remove all property checks during create (#35194
3375839 
) (#35353)
dataroaring pushed a commit that referenced this pull request May 26, 2024
@zy-kkk @dataroaring
[improve](jdbc catalog) Remove all property checks during create (#35194
d66c87f 
)
seawinde pushed a commit to seawinde/doris that referenced this pull request May 27, 2024
@zy-kkk @seawinde
[improve](jdbc catalog) Remove all property checks during create (apa…
7e35521 
…che#35194)
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@zy-kkk
[improve](jdbc catalog) Remove all property checks during create (apa…
c8de851 
…che#35194) (apache#35353)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morningman morningman morningman approved these changes

@wuwenchi wuwenchi wuwenchi approved these changes

@Jibing-Li Jibing-Li Jibing-Li approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.11-merged dev/2.1.4-merged dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@zy-kkk @doris-robot @morningman @wuwenchi @Jibing-Li @dataroaring