[improve](binlog) Filter the truncated partitions #41611

[w41ter]

cherry pick from #41611

[w41ter]

run buildall

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[w41ter]

run buildall

[w41ter]

run buildall

[doris-robot]

TPC-H: Total hot run time: 48750 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fb110614d8a4de9e1f2a940e3a94436f4d7cbdde, data reload: false

------ Round 1 ----------------------------------
q1	17943	4380	4358	4358
q2	2085	150	146	146
q3	10315	1920	1928	1920
q4	10369	1234	1301	1234
q5	8828	3818	3896	3818
q6	238	125	125	125
q7	2063	1636	1607	1607
q8	9296	2713	2694	2694
q9	10125	9852	9730	9730
q10	8654	3536	3513	3513
q11	421	246	253	246
q12	468	295	301	295
q13	18350	3917	4049	3917
q14	348	317	322	317
q15	503	447	459	447
q16	541	458	461	458
q17	1116	947	930	930
q18	7202	6765	6902	6765
q19	1665	1531	1497	1497
q20	541	310	312	310
q21	4373	4154	4045	4045
q22	484	378	403	378
Total cold run time: 115928 ms
Total hot run time: 48750 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4303	4295	4335	4295
q2	334	227	225	225
q3	4158	4115	4121	4115
q4	2765	2749	2763	2749
q5	7173	7089	7039	7039
q6	238	117	117	117
q7	3204	2807	2849	2807
q8	4362	4472	4422	4422
q9	13626	13567	13421	13421
q10	4219	4254	4219	4219
q11	760	707	692	692
q12	1012	860	864	860
q13	6797	3742	3712	3712
q14	453	434	414	414
q15	501	463	450	450
q16	639	608	587	587
q17	3883	3816	3841	3816
q18	8785	8667	8722	8667
q19	1710	1680	1657	1657
q20	2386	2107	2113	2107
q21	8528	8528	8399	8399
q22	1004	971	975	971
Total cold run time: 80840 ms
Total hot run time: 75741 ms

[doris-robot]

TPC-DS: Total hot run time: 212256 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fb110614d8a4de9e1f2a940e3a94436f4d7cbdde, data reload: false

query1	929	392	417	392
query2	6558	2291	2140	2140
query3	6915	203	200	200
query4	23175	21473	21495	21473
query5	19745	6484	6515	6484
query6	284	224	234	224
query7	4338	298	307	298
query8	268	263	255	255
query9	3072	2650	2566	2566
query10	472	319	296	296
query11	15574	15103	14999	14999
query12	121	80	77	77
query13	1026	436	438	436
query14	17838	13481	13634	13481
query15	389	220	224	220
query16	6475	287	254	254
query17	1923	927	914	914
query18	894	328	315	315
query19	209	158	145	145
query20	107	99	108	99
query21	197	100	89	89
query22	5224	5052	5026	5026
query23	34045	33326	33246	33246
query24	6955	6327	6302	6302
query25	546	439	437	437
query26	990	165	162	162
query27	2289	294	290	290
query28	6131	2265	2234	2234
query29	2968	2778	2838	2778
query30	244	176	167	167
query31	971	768	763	763
query32	70	65	60	60
query33	441	251	264	251
query34	846	478	473	473
query35	1156	927	896	896
query36	1361	1464	1251	1251
query37	92	58	58	58
query38	3093	2915	2911	2911
query39	1364	1325	1326	1325
query40	208	98	96	96
query41	37	37	38	37
query42	89	90	87	87
query43	623	568	586	568
query44	1157	707	716	707
query45	241	235	230	230
query46	1239	941	947	941
query47	1827	1713	1917	1713
query48	496	418	410	410
query49	626	363	376	363
query50	871	618	619	618
query51	4752	4667	4682	4667
query52	94	86	71	71
query53	237	185	191	185
query54	2668	2457	2473	2457
query55	93	85	83	83
query56	229	215	207	207
query57	1251	1126	1203	1126
query58	237	217	211	211
query59	3502	3306	3309	3306
query60	220	205	204	204
query61	97	95	104	95
query62	791	448	423	423
query63	196	178	174	174
query64	3217	1572	1515	1515
query65	3620	3580	3530	3530
query66	785	404	413	404
query67	16023	15867	15256	15256
query68	10472	632	626	626
query69	489	261	275	261
query70	1780	1377	1304	1304
query71	414	307	312	307
query72	6956	4801	4676	4676
query73	742	318	329	318
query74	6214	5797	5774	5774
query75	5502	3691	3729	3691
query76	6268	1181	1222	1181
query77	1069	260	252	252
query78	12531	11776	11664	11664
query79	7687	636	646	636
query80	1551	385	390	385
query81	496	237	244	237
query82	1585	93	97	93
query83	169	132	147	132
query84	252	73	71	71
query85	878	330	318	318
query86	346	307	276	276
query87	3189	3018	2983	2983
query88	4440	2267	2276	2267
query89	468	307	290	290
query90	1961	209	203	203
query91	157	135	125	125
query92	57	52	51	51
query93	6936	561	555	555
query94	703	211	211	211
query95	1912	1975	1979	1975
query96	661	319	321	319
query97	6431	6332	6424	6332
query98	219	202	193	193
query99	3039	948	879	879
Total cold run time: 320295 ms
Total hot run time: 212256 ms

[doris-robot]

ClickBench: Total hot run time: 31.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fb110614d8a4de9e1f2a940e3a94436f4d7cbdde, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.25	0.05	0.04
query4	1.78	0.06	0.07
query5	0.54	0.53	0.52
query6	1.24	0.61	0.62
query7	0.01	0.01	0.01
query8	0.03	0.03	0.03
query9	0.52	0.49	0.47
query10	0.53	0.54	0.54
query11	0.12	0.09	0.09
query12	0.11	0.10	0.09
query13	0.64	0.62	0.61
query14	0.77	0.79	0.78
query15	0.79	0.76	0.75
query16	0.36	0.38	0.36
query17	0.99	1.02	1.02
query18	0.24	0.23	0.25
query19	1.92	1.83	1.86
query20	0.01	0.02	0.01
query21	15.48	0.57	0.55
query22	2.02	2.08	1.87
query23	17.08	0.99	0.94
query24	5.27	1.00	0.98
query25	0.39	0.08	0.06
query26	0.60	0.17	0.16
query27	0.03	0.03	0.04
query28	7.90	0.79	0.72
query29	12.69	2.38	2.40
query30	0.59	0.54	0.54
query31	2.80	0.38	0.37
query32	3.38	0.51	0.50
query33	3.07	3.08	3.05
query34	15.27	4.79	4.75
query35	4.86	4.84	4.84
query36	1.05	1.02	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.15	0.13	0.14
query41	0.08	0.01	0.01
query42	0.03	0.01	0.02
query43	0.03	0.01	0.02
Total cold run time: 103.83 s
Total hot run time: 31.08 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit fb110614d8a4de9e1f2a940e3a94436f4d7cbdde with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s