[branch-2.0](routine-load) update partition offset cache timely to avoid negative lag

[sollhui]

Proposed changes

#30455

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

[sollhui]

run buildall

[doris-robot]

TPC-H: Total hot run time: 50094 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f82a3ca9977ea8b54e17422a645a330042ebd78a, data reload: false

------ Round 1 ----------------------------------
q1	17606	4381	4331	4331
q2	2045	149	141	141
q3	10472	1903	1881	1881
q4	10347	1241	1321	1241
q5	8630	3954	3963	3954
q6	224	125	123	123
q7	2021	1569	1575	1569
q8	9247	2699	2690	2690
q9	11140	10722	10774	10722
q10	8629	3517	3498	3498
q11	419	245	241	241
q12	478	301	309	301
q13	18362	3971	3993	3971
q14	355	318	325	318
q15	510	455	466	455
q16	701	603	607	603
q17	1124	954	938	938
q18	7176	6882	6810	6810
q19	1660	1559	1522	1522
q20	513	307	303	303
q21	4458	4094	4112	4094
q22	502	388	395	388
Total cold run time: 116619 ms
Total hot run time: 50094 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4306	4251	4250	4250
q2	318	222	227	222
q3	4180	4161	4167	4161
q4	2734	2746	2758	2746
q5	7274	7182	7182	7182
q6	229	120	118	118
q7	3252	2844	2874	2844
q8	4360	4441	4471	4441
q9	17518	17403	17249	17249
q10	4219	4220	4268	4220
q11	753	710	686	686
q12	1024	856	865	856
q13	6575	3692	3746	3692
q14	458	422	435	422
q15	503	457	446	446
q16	764	706	715	706
q17	3832	3896	3856	3856
q18	8799	8789	8697	8697
q19	1708	1694	1601	1601
q20	2414	2127	2076	2076
q21	8461	8361	8436	8361
q22	1055	966	931	931
Total cold run time: 84736 ms
Total hot run time: 79763 ms

[doris-robot]

TPC-DS: Total hot run time: 244402 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f82a3ca9977ea8b54e17422a645a330042ebd78a, data reload: false

query1	925	390	389	389
query2	6522	2355	1936	1936
query3	6911	193	196	193
query4	21171	18116	18030	18030
query5	19713	6465	6460	6460
query6	274	214	223	214
query7	4154	285	295	285
query8	281	229	258	229
query9	3092	2711	2630	2630
query10	422	285	292	285
query11	11288	10672	10679	10672
query12	124	74	69	69
query13	5572	636	615	615
query14	18083	13684	13621	13621
query15	355	226	235	226
query16	6419	265	260	260
query17	1764	1444	854	854
query18	2331	395	408	395
query19	196	137	146	137
query20	70	82	75	75
query21	188	95	90	90
query22	5389	5155	5091	5091
query23	32507	31790	31958	31790
query24	6879	6523	6495	6495
query25	539	430	416	416
query26	505	165	152	152
query27	1790	293	294	293
query28	6080	2206	2174	2174
query29	2901	2676	2903	2676
query30	242	159	161	159
query31	902	738	758	738
query32	61	63	56	56
query33	387	240	237	237
query34	837	458	482	458
query35	1138	951	948	948
query36	1476	1442	1735	1442
query37	84	59	59	59
query38	3123	2902	2907	2902
query39	1353	1309	1315	1309
query40	203	94	92	92
query41	34	32	34	32
query42	85	82	84	82
query43	631	616	537	537
query44	1108	718	720	718
query45	241	226	229	226
query46	1224	964	964	964
query47	1963	1851	1826	1826
query48	956	689	655	655
query49	605	365	357	357
query50	848	633	592	592
query51	5621	5461	5461	5461
query52	88	80	74	74
query53	440	311	329	311
query54	2664	2442	2437	2437
query55	80	85	85	85
query56	218	199	199	199
query57	1160	1144	1150	1144
query58	213	206	211	206
query59	3463	3318	3249	3249
query60	203	184	193	184
query61	84	84	80	80
query62	822	478	460	460
query63	470	328	326	326
query64	2421	1479	1417	1417
query65	3631	3527	3569	3527
query66	763	370	356	356
query67	15398	15019	15281	15019
query68	9680	677	635	635
query69	573	328	340	328
query70	1906	1655	1608	1608
query71	418	295	326	295
query72	6572	3416	3379	3379
query73	717	319	303	303
query74	6289	5842	5784	5784
query75	5333	3715	3628	3628
query76	6072	1146	1156	1146
query77	1035	248	242	242
query78	32189	53693	55422	53693
query79	15631	625	615	615
query80	4794	392	376	376
query81	563	226	224	224
query82	1856	92	92	92
query83	330	133	133	133
query84	258	81	68	68
query85	1831	286	274	274
query86	472	364	434	364
query87	3263	3027	2991	2991
query88	7138	2294	2266	2266
query89	469	266	291	266
query90	2528	201	204	201
query91	150	119	114	114
query92	62	51	53	51
query93	6306	587	577	577
query94	1704	208	203	203
query95	1101	1058	1049	1049
query96	631	326	318	318
query97	6512	6442	6412	6412
query98	202	173	171	171
query99	3687	930	920	920
Total cold run time: 351957 ms
Total hot run time: 244402 ms

[doris-robot]

ClickBench: Total hot run time: 31.69 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f82a3ca9977ea8b54e17422a645a330042ebd78a, data reload: false

query1	0.03	0.02	0.02
query2	0.06	0.02	0.02
query3	0.24	0.05	0.05
query4	1.82	0.07	0.06
query5	0.54	0.53	0.52
query6	1.24	0.63	0.62
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.52	0.49	0.48
query10	0.54	0.54	0.54
query11	0.11	0.09	0.08
query12	0.11	0.08	0.08
query13	0.62	0.62	0.63
query14	0.79	0.78	0.79
query15	0.78	0.76	0.76
query16	0.39	0.36	0.36
query17	1.01	1.00	1.03
query18	0.22	0.27	0.25
query19	1.95	1.89	1.87
query20	0.01	0.01	0.00
query21	15.47	0.57	0.55
query22	1.94	2.18	2.07
query23	17.35	0.95	0.91
query24	4.25	2.98	1.34
query25	1.40	0.11	0.10
query26	0.31	0.13	0.12
query27	0.11	0.11	0.11
query28	7.64	0.71	0.69
query29	12.69	2.32	2.32
query30	0.57	0.53	0.51
query31	2.80	0.39	0.39
query32	3.40	0.51	0.50
query33	3.07	3.08	3.12
query34	15.26	4.83	4.80
query35	4.89	4.86	4.83
query36	1.05	1.00	1.02
query37	0.05	0.04	0.05
query38	0.03	0.02	0.02
query39	0.01	0.01	0.01
query40	0.16	0.14	0.13
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.59 s
Total hot run time: 31.69 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f82a3ca9977ea8b54e17422a645a330042ebd78a with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s