[fix](routine-load) update partition offset cache timely to avoid negative lag #30455

sollhui · 2024-01-27T03:09:06Z

Proposed changes

Lag is negative when we use routine load, it is need check all partitions, for some partitions offset may be out of time.

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

sollhui · 2024-01-27T03:11:17Z

run build

sollhui · 2024-01-27T03:14:17Z

run buildall

doris-robot · 2024-01-27T03:36:07Z

TPC-H: Total hot run time: 37886 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55876042ac2b628373177dae8beec91891a21c27, data reload: false

------ Round 1 ----------------------------------
q1	17916	5234	5060	5060
q2	2330	143	138	138
q3	11452	952	953	952
q4	5575	781	753	753
q5	8428	2917	2866	2866
q6	188	122	126	122
q7	1167	754	734	734
q8	9425	2040	2022	2022
q9	7173	6337	6309	6309
q10	8129	2473	2409	2409
q11	424	204	207	204
q12	743	282	269	269
q13	18016	3316	3325	3316
q14	282	253	240	240
q15	533	492	490	490
q16	448	406	408	406
q17	934	538	494	494
q18	7020	6053	6239	6053
q19	1557	1396	1347	1347
q20	627	342	310	310
q21	6792	3089	3082	3082
q22	788	325	310	310
Total cold run time: 109947 ms
Total hot run time: 37886 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5043	5051	4958	4958
q2	339	242	238	238
q3	2997	2658	2719	2658
q4	1796	1601	1589	1589
q5	5354	5286	5276	5276
q6	213	112	117	112
q7	2119	1820	1733	1733
q8	3094	3188	3187	3187
q9	8516	8581	14032	8581
q10	5616	3461	3460	3460
q11	547	457	473	457
q12	751	592	602	592
q13	16954	3116	3063	3063
q14	274	253	258	253
q15	539	492	492	492
q16	520	460	485	460
q17	2036	1627	1707	1627
q18	9011	10609	8590	8590
q19	24216	1587	1504	1504
q20	4595	1927	1884	1884
q21	15815	4841	4804	4804
q22	2210	474	498	474
Total cold run time: 112555 ms
Total hot run time: 55992 ms

doris-robot · 2024-01-27T03:47:11Z

TPC-DS: Total hot run time: 172510 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 55876042ac2b628373177dae8beec91891a21c27, data reload: false

query1	947	332	329	329
query2	6552	1921	1919	1919
query3	6716	196	200	196
query4	31992	21914	21997	21914
query5	4442	422	371	371
query6	253	152	150	150
query7	4605	259	263	259
query8	253	172	180	172
query9	8555	2290	2267	2267
query10	427	218	211	211
query11	17772	15477	15250	15250
query12	123	68	69	68
query13	1678	364	365	364
query14	9154	6369	6478	6369
query15	203	181	192	181
query16	5695	251	242	242
query17	962	474	455	455
query18	1806	248	247	247
query19	172	128	130	128
query20	72	65	71	65
query21	194	136	129	129
query22	4927	4688	4682	4682
query23	31255	30521	30420	30420
query24	11566	2832	2774	2774
query25	597	308	307	307
query26	1784	135	136	135
query27	3204	281	276	276
query28	7452	1848	1834	1834
query29	1841	617	590	590
query30	276	132	138	132
query31	898	698	722	698
query32	74	48	47	47
query33	692	209	205	205
query34	1179	449	449	449
query35	858	758	755	755
query36	1325	1235	1209	1209
query37	90	59	54	54
query38	3242	3173	3201	3173
query39	1309	1244	1251	1244
query40	336	90	81	81
query41	38	35	34	34
query42	90	78	87	78
query43	507	530	485	485
query44	1095	679	679	679
query45	187	179	184	179
query46	1042	660	648	648
query47	1605	1494	1444	1444
query48	382	303	301	301
query49	1176	285	278	278
query50	669	306	303	303
query51	5353	5155	5224	5155
query52	91	83	77	77
query53	327	252	261	252
query54	259	177	182	177
query55	75	73	70	70
query56	182	160	164	160
query57	973	902	912	902
query58	180	153	150	150
query59	2719	2410	2596	2410
query60	193	174	178	174
query61	81	87	80	80
query62	623	345	350	345
query63	282	264	261	261
query64	5203	1762	1734	1734
query65	3284	3231	3211	3211
query66	1291	318	311	311
query67	14426	14422	14159	14159
query68	3740	485	496	485
query69	431	298	293	293
query70	1563	1501	1467	1467
query71	291	217	229	217
query72	3726	2832	2842	2832
query73	689	319	307	307
query74	6613	6227	6311	6227
query75	2793	2325	2308	2308
query76	2614	1027	925	925
query77	366	230	232	230
query78	9072	8827	8449	8449
query79	2110	497	487	487
query80	1702	329	319	319
query81	506	200	200	200
query82	690	83	76	76
query83	224	113	114	113
query84	273	73	70	70
query85	2177	335	324	324
query86	520	406	397	397
query87	3416	3222	3248	3222
query88	3853	2169	2151	2151
query89	439	368	346	346
query90	1957	182	183	182
query91	140	117	117	117
query92	56	49	46	46
query93	2093	425	419	419
query94	1373	161	161	161
query95	488	454	449	449
query96	612	316	310	310
query97	4261	4129	4168	4129
query98	211	195	187	187
query99	1024	703	657	657
Total cold run time: 277834 ms
Total hot run time: 172510 ms

doris-robot · 2024-01-27T03:52:38Z

ClickBench: Total hot run time: 30.45 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 55876042ac2b628373177dae8beec91891a21c27, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.02	0.02
query3	0.23	0.06	0.06
query4	1.65	0.10	0.10
query5	0.53	0.52	0.51
query6	1.23	0.64	0.62
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.54	0.50	0.48
query10	0.56	0.55	0.55
query11	0.13	0.08	0.08
query12	0.11	0.09	0.10
query13	0.60	0.61	0.61
query14	0.77	0.81	0.81
query15	0.78	0.77	0.80
query16	0.39	0.39	0.38
query17	1.02	1.00	1.05
query18	0.24	0.25	0.24
query19	1.79	1.74	1.77
query20	0.01	0.01	0.01
query21	15.42	0.57	0.55
query22	2.77	2.75	1.81
query23	17.18	0.75	0.79
query24	2.41	1.39	0.34
query25	0.25	0.25	0.12
query26	0.54	0.14	0.14
query27	0.06	0.05	0.04
query28	11.83	0.84	0.85
query29	12.59	3.26	3.27
query30	0.73	0.56	0.59
query31	2.78	0.35	0.36
query32	3.34	0.48	0.48
query33	3.22	3.22	3.22
query34	15.85	4.33	4.27
query35	4.32	4.26	4.27
query36	1.08	1.07	1.07
query37	0.07	0.05	0.05
query38	0.05	0.03	0.02
query39	0.02	0.02	0.02
query40	0.17	0.13	0.13
query41	0.07	0.01	0.02
query42	0.03	0.01	0.02
query43	0.02	0.02	0.03
Total cold run time: 105.53 s
Total hot run time: 30.45 s

doris-robot · 2024-01-27T03:55:45Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 55876042ac2b628373177dae8beec91891a21c27 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select:       13.2 seconds inserted 10000000 Rows, about 757K ops/s

liaoxin01

LGTM

github-actions · 2024-01-29T07:48:38Z

PR approved by anyone and no changes requested.

sollhui · 2024-01-30T06:11:33Z

run buildall

doris-robot · 2024-01-30T06:44:44Z

TPC-H: Total hot run time: 36671 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ac95336128a8f38af648c70e2e5cbe131118ff9b, data reload: false

------ Round 1 ----------------------------------
q1	17675	4497	4380	4380
q2	2040	141	131	131
q3	10615	933	886	886
q4	4645	784	706	706
q5	7687	2811	2743	2743
q6	183	122	122	122
q7	1148	719	704	704
q8	9253	1971	1992	1971
q9	7170	6274	6286	6274
q10	8091	2411	2415	2411
q11	405	223	201	201
q12	768	278	264	264
q13	18000	3319	3296	3296
q14	277	248	237	237
q15	535	498	495	495
q16	460	420	397	397
q17	928	493	499	493
q18	6567	5895	5904	5895
q19	1564	1435	1336	1336
q20	564	355	317	317
q21	6775	3102	3132	3102
q22	804	316	310	310
Total cold run time: 106154 ms
Total hot run time: 36671 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4430	4415	4351	4351
q2	334	233	223	223
q3	2979	2866	2814	2814
q4	1860	1659	1674	1659
q5	5226	5187	5179	5179
q6	189	115	114	114
q7	2102	1804	1768	1768
q8	3066	3204	3234	3204
q9	8300	8302	8337	8302
q10	5719	3491	3549	3491
q11	540	472	446	446
q12	733	577	570	570
q13	11657	3110	3075	3075
q14	278	251	268	251
q15	529	491	490	490
q16	512	459	481	459
q17	1840	1664	1725	1664
q18	7918	7722	7579	7579
q19	6931	1492	1543	1492
q20	2116	1929	1908	1908
q21	4793	4617	4733	4617
q22	553	470	465	465
Total cold run time: 72605 ms
Total hot run time: 54121 ms

doris-robot · 2024-01-30T06:55:27Z

TPC-DS: Total hot run time: 171870 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ac95336128a8f38af648c70e2e5cbe131118ff9b, data reload: false

query1	968	332	329	329
query2	6566	1874	1836	1836
query3	6703	197	193	193
query4	27532	22279	22048	22048
query5	4451	398	419	398
query6	252	147	150	147
query7	4599	263	257	257
query8	251	172	193	172
query9	8944	2302	2292	2292
query10	405	211	193	193
query11	17206	15295	15479	15295
query12	126	75	67	67
query13	1689	371	376	371
query14	9172	6463	6563	6463
query15	248	176	187	176
query16	6363	249	250	249
query17	1895	469	464	464
query18	1875	249	247	247
query19	178	130	127	127
query20	73	70	66	66
query21	194	133	133	133
query22	4981	4766	4406	4406
query23	31099	30332	30320	30320
query24	7073	2764	2797	2764
query25	500	298	320	298
query26	807	140	138	138
query27	2503	278	283	278
query28	4040	1842	1852	1842
query29	924	616	604	604
query30	207	136	136	136
query31	915	688	717	688
query32	87	54	49	49
query33	426	215	203	203
query34	811	451	459	451
query35	839	755	748	748
query36	1306	1153	1190	1153
query37	89	57	61	57
query38	3291	3205	3176	3176
query39	1308	1255	1255	1255
query40	192	88	80	80
query41	39	34	35	34
query42	93	78	82	78
query43	491	466	486	466
query44	1084	688	688	688
query45	189	177	179	177
query46	1028	638	631	631
query47	1566	1432	1506	1432
query48	382	303	304	303
query49	626	279	277	277
query50	672	312	309	309
query51	5216	5240	5079	5079
query52	91	78	79	78
query53	326	253	254	253
query54	230	182	189	182
query55	82	72	78	72
query56	188	157	152	152
query57	985	917	910	910
query58	185	150	155	150
query59	2558	2410	2335	2335
query60	206	172	174	172
query61	85	88	81	81
query62	500	348	349	348
query63	286	261	265	261
query64	3623	1757	1713	1713
query65	3299	3226	3221	3221
query66	1115	322	317	317
query67	14416	14099	14013	14013
query68	3878	490	494	490
query69	447	309	299	299
query70	1464	1556	1543	1543
query71	291	215	229	215
query72	3653	2834	2811	2811
query73	684	324	306	306
query74	6598	6258	6233	6233
query75	2802	2279	2271	2271
query76	2921	953	875	875
query77	380	231	235	231
query78	9081	8632	8397	8397
query79	2562	488	501	488
query80	1417	317	312	312
query81	491	198	202	198
query82	953	83	78	78
query83	222	115	112	112
query84	231	76	75	75
query85	1104	330	329	329
query86	463	399	379	379
query87	3395	3297	3217	3217
query88	3766	2171	2162	2162
query89	429	360	347	347
query90	2122	185	184	184
query91	146	116	118	116
query92	59	46	46	46
query93	1872	423	407	407
query94	1290	159	153	153
query95	502	452	453	452
query96	611	324	309	309
query97	4273	4121	4126	4121
query98	193	194	180	180
query99	1064	633	643	633
Total cold run time: 260013 ms
Total hot run time: 171870 ms

doris-robot · 2024-01-30T07:00:52Z

ClickBench: Total hot run time: 30.8 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ac95336128a8f38af648c70e2e5cbe131118ff9b, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.02	0.02
query3	0.23	0.06	0.06
query4	1.68	0.10	0.10
query5	0.52	0.51	0.52
query6	1.19	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.02
query9	0.57	0.50	0.50
query10	0.56	0.54	0.55
query11	0.12	0.09	0.09
query12	0.11	0.10	0.09
query13	0.60	0.61	0.61
query14	0.79	0.80	0.81
query15	0.78	0.78	0.77
query16	0.38	0.39	0.37
query17	1.02	1.02	0.99
query18	0.22	0.26	0.25
query19	1.84	1.81	1.79
query20	0.02	0.01	0.01
query21	15.40	0.58	0.58
query22	2.57	2.20	1.55
query23	16.98	1.01	0.76
query24	2.41	0.89	0.98
query25	0.31	0.15	0.06
query26	0.58	0.15	0.14
query27	0.05	0.06	0.05
query28	11.65	0.84	0.83
query29	12.61	3.30	3.29
query30	0.65	0.49	0.54
query31	2.82	0.35	0.35
query32	3.36	0.49	0.49
query33	3.26	3.26	3.22
query34	15.76	4.26	4.29
query35	4.29	4.32	4.34
query36	1.11	1.05	1.05
query37	0.07	0.04	0.04
query38	0.04	0.03	0.03
query39	0.02	0.02	0.02
query40	0.16	0.12	0.12
query41	0.07	0.02	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.99 s
Total hot run time: 30.8 s

doris-robot · 2024-01-30T07:03:55Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ac95336128a8f38af648c70e2e5cbe131118ff9b with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.6 seconds inserted 10000000 Rows, about 735K ops/s

sollhui · 2024-01-30T09:26:33Z

run buildall

doris-robot · 2024-01-30T10:35:28Z

TPC-H: Total hot run time: 37057 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ac95336128a8f38af648c70e2e5cbe131118ff9b, data reload: false

------ Round 1 ----------------------------------
q1	17681	4488	4385	4385
q2	2044	143	135	135
q3	10594	956	926	926
q4	4913	783	701	701
q5	7683	2750	2835	2750
q6	188	120	119	119
q7	1137	717	716	716
q8	9246	1987	2016	1987
q9	7241	6340	6340	6340
q10	8178	2564	2510	2510
q11	424	211	219	211
q12	732	275	281	275
q13	18015	3327	3297	3297
q14	287	250	257	250
q15	526	487	487	487
q16	473	430	405	405
q17	943	467	524	467
q18	6838	6026	6079	6026
q19	1569	1370	1379	1370
q20	604	332	337	332
q21	6974	3069	3068	3068
q22	810	300	304	300
Total cold run time: 107100 ms
Total hot run time: 37057 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4475	4454	4397	4397
q2	319	222	233	222
q3	3069	2882	2781	2781
q4	1892	1662	1676	1662
q5	5191	5239	5242	5239
q6	195	113	114	113
q7	2175	1759	1747	1747
q8	3106	3254	3272	3254
q9	8402	8345	8258	8258
q10	5919	3592	3579	3579
q11	541	453	455	453
q12	763	580	595	580
q13	12267	3104	3068	3068
q14	274	261	270	261
q15	540	497	488	488
q16	493	443	475	443
q17	1829	1734	1658	1658
q18	7959	7776	7592	7592
q19	10943	1580	1573	1573
q20	2161	1912	1908	1908
q21	4695	4542	4658	4542
q22	554	442	464	442
Total cold run time: 77762 ms
Total hot run time: 54260 ms

doris-robot · 2024-01-30T10:46:19Z

TPC-DS: Total hot run time: 173166 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ac95336128a8f38af648c70e2e5cbe131118ff9b, data reload: false

query1	928	338	328	328
query2	6571	2146	1966	1966
query3	6705	208	208	208
query4	26887	22132	22124	22124
query5	4455	431	427	427
query6	250	163	157	157
query7	4608	274	263	263
query8	247	172	174	172
query9	8980	2245	2237	2237
query10	412	203	199	199
query11	17130	15294	15329	15294
query12	116	65	63	63
query13	1674	371	370	370
query14	9149	6511	6414	6414
query15	206	172	180	172
query16	6167	260	254	254
query17	1807	482	481	481
query18	1789	251	244	244
query19	180	136	131	131
query20	70	78	66	66
query21	193	123	122	122
query22	4967	4755	4782	4755
query23	31132	30440	30330	30330
query24	10297	2767	2758	2758
query25	539	308	321	308
query26	886	143	138	138
query27	2785	282	289	282
query28	5919	1830	1804	1804
query29	903	655	628	628
query30	280	136	138	136
query31	924	692	736	692
query32	92	57	50	50
query33	554	207	213	207
query34	821	453	460	453
query35	828	739	768	739
query36	1274	1247	1126	1126
query37	90	56	59	56
query38	3281	3126	3147	3126
query39	1316	1240	1246	1240
query40	190	89	82	82
query41	38	37	35	35
query42	82	83	82	82
query43	501	491	502	491
query44	1060	678	675	675
query45	208	180	182	180
query46	1030	632	627	627
query47	1539	1483	1450	1450
query48	378	306	301	301
query49	1130	286	273	273
query50	696	313	313	313
query51	5322	5189	5194	5189
query52	87	78	77	77
query53	319	267	271	267
query54	225	185	189	185
query55	77	75	73	73
query56	182	161	157	157
query57	1001	916	889	889
query58	184	159	154	154
query59	2514	2351	2309	2309
query60	197	179	181	179
query61	85	91	82	82
query62	586	344	338	338
query63	273	254	264	254
query64	4084	1767	1739	1739
query65	3283	3222	3210	3210
query66	1143	316	312	312
query67	14581	14149	14322	14149
query68	3850	510	491	491
query69	441	318	311	311
query70	1520	1521	1521	1521
query71	278	208	232	208
query72	3706	2848	2868	2848
query73	692	321	316	316
query74	6623	6232	6255	6232
query75	2752	2314	2322	2314
query76	2920	997	951	951
query77	380	231	226	226
query78	9015	8816	8598	8598
query79	2726	503	493	493
query80	1324	315	316	315
query81	532	200	199	199
query82	1024	82	78	78
query83	208	118	117	117
query84	237	81	78	78
query85	1074	341	327	327
query86	459	388	382	382
query87	3419	3293	3267	3267
query88	3861	2264	2261	2261
query89	411	360	358	358
query90	2022	194	187	187
query91	151	120	124	120
query92	50	44	41	41
query93	1918	443	421	421
query94	1228	158	160	158
query95	498	461	456	456
query96	634	330	332	330
query97	4239	4121	4131	4121
query98	206	189	180	180
query99	1142	690	669	669
Total cold run time: 265947 ms
Total hot run time: 173166 ms

doris-robot · 2024-01-30T10:51:44Z

ClickBench: Total hot run time: 30.52 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ac95336128a8f38af648c70e2e5cbe131118ff9b, data reload: false

query1	0.04	0.03	0.03
query2	0.05	0.02	0.02
query3	0.23	0.06	0.06
query4	1.67	0.10	0.10
query5	0.54	0.51	0.52
query6	1.18	0.62	0.63
query7	0.01	0.01	0.02
query8	0.04	0.02	0.02
query9	0.55	0.52	0.49
query10	0.56	0.54	0.54
query11	0.11	0.08	0.08
query12	0.10	0.09	0.09
query13	0.61	0.60	0.61
query14	0.79	0.79	0.80
query15	0.80	0.79	0.78
query16	0.38	0.40	0.39
query17	1.01	1.02	1.02
query18	0.22	0.24	0.26
query19	1.90	1.79	1.75
query20	0.01	0.02	0.01
query21	15.43	0.57	0.57
query22	2.38	2.52	1.69
query23	17.30	0.90	0.80
query24	2.94	1.03	0.68
query25	0.44	0.19	0.12
query26	0.41	0.14	0.13
query27	0.06	0.05	0.04
query28	11.18	0.85	0.82
query29	12.57	3.20	3.11
query30	0.65	0.50	0.59
query31	2.78	0.36	0.35
query32	3.35	0.48	0.47
query33	3.20	3.20	3.23
query34	15.81	4.31	4.31
query35	4.32	4.27	4.28
query36	1.09	1.05	1.07
query37	0.06	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.01	0.02
query40	0.17	0.13	0.13
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 105.11 s
Total hot run time: 30.52 s

doris-robot · 2024-01-30T10:54:48Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ac95336128a8f38af648c70e2e5cbe131118ff9b with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.5 seconds inserted 10000000 Rows, about 740K ops/s

dataroaring

LGTM

github-actions · 2024-02-01T06:41:19Z

PR approved by at least one committer and no changes requested.

…ative lag #30455

…ative lag apache#30455

…oid negative lag #30455 (#30937)

…oid negative lag apache#30455 (apache#30937)

sollhui force-pushed the routine_load_lag branch from 6a3dc2b to e3dc434 Compare January 27, 2024 03:11

sollhui changed the title ~~[fix](routine-load) update cache timely to avoid negative lag~~ [fix](routine-load) update partition offset cache timely to avoid negative lag Jan 27, 2024

sollhui force-pushed the routine_load_lag branch from e3dc434 to 5587604 Compare January 27, 2024 03:14

liaoxin01 previously approved these changes Jan 29, 2024

View reviewed changes

github-actions bot added the reviewed label Jan 29, 2024

xiaokang added the dev/2.0.5 label Jan 29, 2024

sollhui closed this Jan 30, 2024

sollhui force-pushed the routine_load_lag branch from 5587604 to 06cbb96 Compare January 30, 2024 06:10

update partition offset cache timely to avoid negative lag

ac95336

sollhui reopened this Jan 30, 2024

dataroaring approved these changes Feb 1, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 1, 2024

sollhui dismissed liaoxin01’s stale review via ac95336 February 1, 2024 10:07

yiguolei merged commit 142cbde into apache:master Feb 1, 2024
32 of 36 checks passed

yiguolei pushed a commit that referenced this pull request Feb 1, 2024

[fix](routine-load) update partition offset cache timely to avoid neg…

7935dc9

…ative lag #30455

sollhui added a commit to sollhui/doris that referenced this pull request Feb 7, 2024

[fix](routine-load) update partition offset cache timely to avoid neg…

162a0cf

…ative lag apache#30455

sollhui mentioned this pull request Feb 7, 2024

[branch-2.0](routine-load) update partition offset cache timely to avoid negative lag #30937

Merged

xiaokang pushed a commit that referenced this pull request Feb 7, 2024

[branch-2.0](routine-load) update partition offset cache timely to av…

8bade7a

…oid negative lag #30455 (#30937)

xiaokang added dev/2.0.5-merged and removed dev/2.0.5 labels Feb 7, 2024

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[branch-2.0](routine-load) update partition offset cache timely to av…

13e8ccc

…oid negative lag apache#30455 (apache#30937)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](routine-load) update partition offset cache timely to avoid negative lag #30455

[fix](routine-load) update partition offset cache timely to avoid negative lag #30455

sollhui commented Jan 27, 2024

sollhui commented Jan 27, 2024

sollhui commented Jan 27, 2024

doris-robot commented Jan 27, 2024

doris-robot commented Jan 27, 2024

doris-robot commented Jan 27, 2024

doris-robot commented Jan 27, 2024

liaoxin01 left a comment

github-actions bot commented Jan 29, 2024

sollhui commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

sollhui commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

dataroaring left a comment

github-actions bot commented Feb 1, 2024

[fix](routine-load) update partition offset cache timely to avoid negative lag #30455

[fix](routine-load) update partition offset cache timely to avoid negative lag #30455

Conversation

sollhui commented Jan 27, 2024

Proposed changes

Further comments

sollhui commented Jan 27, 2024

sollhui commented Jan 27, 2024

doris-robot commented Jan 27, 2024

doris-robot commented Jan 27, 2024

doris-robot commented Jan 27, 2024

doris-robot commented Jan 27, 2024

liaoxin01 left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 29, 2024

sollhui commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

sollhui commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

doris-robot commented Jan 30, 2024

dataroaring left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 1, 2024