Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exception while truncate table. #40738

Merged
merged 1 commit into from
Sep 12, 2024
Merged

[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exception while truncate table. #40738

merged 1 commit into from
Sep 12, 2024
+235 −70

Conversation

Jibing-Li
Copy link
Contributor

Bug fix.

  1. Fix drop stats log editlog NPE bug.
  2. Catch drop stats exception while truncate table.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li Jibing-Li marked this pull request as ready for review September 12, 2024 08:20
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 12, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39912 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 65a88de8ac2f5bf4e5b22cc63a83655979f56fdd, data reload: false

------ Round 1 ----------------------------------
q1	18131	4373	4267	4267
q2	2581	191	213	191
q3	11616	1368	1339	1339
q4	10780	993	1078	993
q5	7840	3233	3172	3172
q6	229	141	141	141
q7	1044	648	630	630
q8	9474	1951	1995	1951
q9	6761	6309	6316	6309
q10	7039	2452	2469	2452
q11	444	259	251	251
q12	415	229	230	229
q13	17755	3044	3071	3044
q14	287	259	267	259
q15	560	493	492	492
q16	526	437	432	432
q17	983	934	935	934
q18	7591	6758	6862	6758
q19	1392	1250	1224	1224
q20	609	330	334	330
q21	3883	3515	3520	3515
q22	1106	1015	999	999
Total cold run time: 111046 ms
Total hot run time: 39912 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4243	4150	4217	4150
q2	343	232	241	232
q3	2940	2906	2923	2906
q4	1969	1956	1956	1956
q5	5446	5436	5462	5436
q6	218	132	133	132
q7	2062	1699	1668	1668
q8	3230	3331	3306	3306
q9	8428	8462	8458	8458
q10	3393	3425	3432	3425
q11	573	469	472	469
q12	760	590	592	590
q13	6388	3109	3119	3109
q14	293	270	277	270
q15	542	484	482	482
q16	504	475	460	460
q17	1765	1752	1714	1714
q18	8045	7629	7758	7629
q19	1711	1687	1681	1681
q20	2051	1782	1807	1782
q21	5576	5563	5466	5466
q22	1119	1024	1021	1021
Total cold run time: 61599 ms
Total hot run time: 56342 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194724 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 65a88de8ac2f5bf4e5b22cc63a83655979f56fdd, data reload: false

query1	923	390	387	387
query2	6499	1695	1689	1689
query3	6657	210	224	210
query4	26301	24183	24110	24110
query5	5419	545	533	533
query6	268	172	169	169
query7	4584	310	300	300
query8	281	222	244	222
query9	8511	2459	2460	2459
query10	515	292	291	291
query11	15981	15400	15551	15400
query12	165	102	101	101
query13	1685	400	386	386
query14	10807	6827	6507	6507
query15	219	184	180	180
query16	7574	516	441	441
query17	1438	588	586	586
query18	1940	306	304	304
query19	204	163	152	152
query20	125	113	115	113
query21	212	108	108	108
query22	4605	4225	4311	4225
query23	34310	33745	33693	33693
query24	10928	3167	3127	3127
query25	716	419	417	417
query26	1762	169	168	168
query27	2864	280	285	280
query28	6908	2031	2011	2011
query29	1106	449	436	436
query30	306	166	166	166
query31	983	786	789	786
query32	110	60	58	58
query33	732	310	301	301
query34	921	483	486	483
query35	897	772	744	744
query36	1060	930	912	912
query37	181	94	86	86
query38	4004	3933	3917	3917
query39	1465	1426	1407	1407
query40	288	121	122	121
query41	52	48	48	48
query42	121	101	100	100
query43	479	432	425	425
query44	1204	792	759	759
query45	203	172	177	172
query46	1128	818	821	818
query47	1895	1760	1790	1760
query48	377	291	294	291
query49	1226	467	458	458
query50	912	438	454	438
query51	7602	7012	6837	6837
query52	103	93	92	92
query53	266	190	190	190
query54	789	474	481	474
query55	80	80	79	79
query56	301	292	278	278
query57	1235	1091	1089	1089
query58	266	243	254	243
query59	2583	2671	2582	2582
query60	310	295	280	280
query61	124	235	104	104
query62	920	651	683	651
query63	230	194	191	191
query64	5146	704	672	672
query65	3356	3184	3171	3171
query66	1391	345	348	345
query67	15977	15496	15556	15496
query68	3210	594	588	588
query69	407	279	285	279
query70	1174	1104	1075	1075
query71	347	304	278	278
query72	5862	4221	4147	4147
query73	771	330	326	326
query74	9316	8871	9067	8871
query75	3368	2854	2790	2790
query76	1882	1306	1339	1306
query77	448	327	320	320
query78	9954	9693	9281	9281
query79	1146	910	882	882
query80	993	828	811	811
query81	508	268	272	268
query82	1235	255	265	255
query83	210	192	197	192
query84	244	110	107	107
query85	725	409	390	390
query86	461	315	319	315
query87	4436	4411	4369	4369
query88	4080	4010	4007	4007
query89	397	384	372	372
query90	1750	317	311	311
query91	124	122	176	122
query92	79	76	76	76
query93	1060	1066	1043	1043
query94	815	383	357	357
query95	540	422	423	422
query96	472	464	466	464
query97	3151	3135	3194	3135
query98	231	228	227	227
query99	1564	1327	1309	1309
Total cold run time: 283690 ms
Total hot run time: 194724 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.87 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 65a88de8ac2f5bf4e5b22cc63a83655979f56fdd, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.04	0.05
query4	1.69	0.07	0.07
query5	0.50	0.49	0.50
query6	1.15	0.74	0.74
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.52	0.52
query10	0.58	0.58	0.56
query11	0.16	0.11	0.12
query12	0.16	0.13	0.13
query13	0.63	0.61	0.62
query14	1.46	1.46	1.46
query15	0.92	0.88	0.89
query16	0.37	0.36	0.35
query17	1.04	1.04	1.06
query18	0.22	0.20	0.20
query19	1.91	1.82	1.85
query20	0.01	0.01	0.01
query21	15.40	0.70	0.68
query22	4.08	7.14	2.07
query23	18.15	1.25	1.30
query24	2.26	0.22	0.22
query25	0.18	0.09	0.08
query26	0.30	0.18	0.18
query27	0.08	0.08	0.08
query28	13.17	1.01	0.98
query29	12.54	3.36	3.40
query30	0.25	0.06	0.05
query31	2.88	0.42	0.42
query32	3.23	0.50	0.50
query33	2.99	3.12	3.03
query34	15.43	4.33	4.35
query35	4.36	4.38	4.38
query36	0.69	0.49	0.48
query37	0.19	0.17	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.19	0.14	0.14
query41	0.09	0.06	0.05
query42	0.05	0.05	0.04
query43	0.05	0.04	0.05
Total cold run time: 108.55 s
Total hot run time: 31.87 s

@Jibing-Li
Copy link
Contributor Author

run external

morningman
morningman approved these changes Sep 12, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Jibing-Li Jibing-Li merged commit 4466541 into apache:master Sep 12, 2024
26 of 29 checks passed
@Jibing-Li Jibing-Li deleted the fix branch September 13, 2024 02:42
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Sep 14, 2024
@Jibing-Li
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exc…
e7d9b50 
…eption while truncate table. (apache#40738)

Bug fix.
1. Fix drop stats log editlog NPE bug.
2. Catch drop stats exception while truncate table.
@Jibing-Li Jibing-Li mentioned this pull request Sep 14, 2024
Merged
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Sep 14, 2024
@Jibing-Li
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exc…
7c0e9f4 
…eption while truncate table. (apache#40738)

Bug fix.
1. Fix drop stats log editlog NPE bug.
2. Catch drop stats exception while truncate table.
Jibing-Li added a commit that referenced this pull request Sep 14, 2024
@Jibing-Li
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exc…
b7103bf 
…eption while truncate table. (#40738) (#40853)

backport: #40738
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Sep 19, 2024
@Jibing-Li
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exc…
79a282a 
…eption while truncate table. (apache#40738)

Bug fix.
1. Fix drop stats log editlog NPE bug.
2. Catch drop stats exception while truncate table.
@Jibing-Li Jibing-Li mentioned this pull request Sep 19, 2024
Merged
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Sep 19, 2024
@Jibing-Li
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exc…
817766f 
…eption while truncate table. (apache#40738)

Bug fix.
1. Fix drop stats log editlog NPE bug.
2. Catch drop stats exception while truncate table.
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Sep 26, 2024
@Jibing-Li
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exc…
c195ad2 
…eption while truncate table. (apache#40738)

Bug fix.
1. Fix drop stats log editlog NPE bug.
2. Catch drop stats exception while truncate table.
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Sep 26, 2024
@Jibing-Li
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exc…
32d4321 
…eption while truncate table. (apache#40738)

Bug fix.
1. Fix drop stats log editlog NPE bug.
2. Catch drop stats exception while truncate table.
Jibing-Li added a commit that referenced this pull request Sep 26, 2024
@Jibing-Li
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exc…
b7fc188 
…eption while truncate table. (#40738) (#40979)

backport: #40738
Jibing-Li added a commit that referenced this pull request Oct 8, 2024
@Jibing-Li
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exc…
1515e17 
…eption while truncate table. (#40738) (#40979)

backport: #40738
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morningman morningman morningman approved these changes

@morrySnow morrySnow morrySnow approved these changes

@dataroaring dataroaring Awaiting requested review from dataroaring dataroaring is a code owner

@CalvinKirs CalvinKirs Awaiting requested review from CalvinKirs CalvinKirs is a code owner

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.16-merged dev/2.1.7-merged dev/3.0.x-conflict reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Jibing-Li @doris-robot @morningman @morrySnow @dataroaring