Support high priority column stats auto collection.

[Jibing-Li]

• fix visible column (fix visible column #33023)
• Collect high priority columns. ([feature](statistics)Collect high priority columns. #31235)
• High priority queue and map. ([feature](statistics)High priority queue and map. #31509)
• Support column level health value. (Support column level health value. #31794)
• Support follower sync query columns to master. (Support follower sync query columns to master. #31859)
• Support show auto analyze pending jobs. (Support show auto analyze pending jobs. #31926)
• Check column health value earlier, show job priority. (Check column health value earlier, show job priority. #32064)
• support window (support window #32094)
• Refactor. (Refactor. #32273)
• refactor2 (refactor2 #32278)
• Unit test (Add Unit Test #32398)
• Support auto analyze mv (Support auto analyze mv #32433)
• Fix bug (Fix show auto analyze progress all 0. #32454)
• Support identical column name in different index. (Support identical column name in different index. Cherry pick #32957)
• Fix visible column
• Use future to block auto analyze before job finish. (Use future to block auto analyze before job finish. #33083)
• Fix ut. (Fix ut. #33147)
• Fix ut (Fix ut #33161)
• fix p0 (fix p0 #33210)
• Improve failover logic. (Remove analyze retry logic. #33382)
• Improve waiting empty table logic. (Improve waiting empty table logic. #33472)
• Fix pipeline (Fix pipeline #33671)

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[Jibing-Li]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[doris-robot]

TPC-H: Total hot run time: 39833 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d5fbcfd0b5011c47cec8cac15a06ee160be9d5fa, data reload: false

------ Round 1 ----------------------------------
q1	17604	4341	4268	4268
q2	1999	195	191	191
q3	10453	1205	1246	1205
q4	10183	868	743	743
q5	7546	2752	2721	2721
q6	223	127	133	127
q7	1040	633	625	625
q8	9314	2170	2119	2119
q9	8214	6804	6782	6782
q10	8792	3674	3705	3674
q11	469	240	235	235
q12	497	221	234	221
q13	18752	3129	3153	3129
q14	297	248	235	235
q15	515	456	470	456
q16	551	419	391	391
q17	964	670	730	670
q18	7813	7189	7059	7059
q19	4176	1519	1549	1519
q20	650	329	324	324
q21	3695	2830	2915	2830
q22	377	309	319	309
Total cold run time: 114124 ms
Total hot run time: 39833 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4521	4408	4370	4370
q2	379	272	260	260
q3	3207	2948	3003	2948
q4	1983	1688	1691	1688
q5	5678	5378	5564	5378
q6	211	122	123	122
q7	2335	2012	2000	2000
q8	3304	3396	3467	3396
q9	8951	8882	9023	8882
q10	4066	3913	3817	3817
q11	586	483	490	483
q12	755	596	605	596
q13	17126	3110	3100	3100
q14	327	287	287	287
q15	524	469	482	469
q16	478	451	434	434
q17	1824	1510	1465	1465
q18	8092	7572	7388	7388
q19	2209	1519	1591	1519
q20	2029	1773	1742	1742
q21	11781	4746	4683	4683
q22	536	476	476	476
Total cold run time: 80902 ms
Total hot run time: 55503 ms

[doris-robot]

TPC-DS: Total hot run time: 183241 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d5fbcfd0b5011c47cec8cac15a06ee160be9d5fa, data reload: false

query1	900	1124	1113	1113
query2	6485	2655	2433	2433
query3	6638	204	211	204
query4	23105	21298	21288	21288
query5	4112	385	393	385
query6	262	174	172	172
query7	4585	293	282	282
query8	227	179	169	169
query9	9062	2325	2288	2288
query10	432	237	247	237
query11	14720	14386	14242	14242
query12	142	96	88	88
query13	1658	367	358	358
query14	8744	6816	6763	6763
query15	208	177	179	177
query16	6767	276	252	252
query17	1001	550	537	537
query18	1787	273	263	263
query19	189	148	148	148
query20	90	87	86	86
query21	199	132	132	132
query22	4973	4889	4898	4889
query23	33780	33089	32886	32886
query24	7523	2965	2942	2942
query25	459	365	383	365
query26	693	154	150	150
query27	2485	310	320	310
query28	4398	2026	2003	2003
query29	821	623	608	608
query30	258	164	177	164
query31	922	729	721	721
query32	58	51	51	51
query33	506	236	242	236
query34	917	464	477	464
query35	784	696	670	670
query36	1005	907	879	879
query37	101	77	68	68
query38	3392	3197	3234	3197
query39	1578	1539	1534	1534
query40	193	133	125	125
query41	47	45	49	45
query42	103	95	96	95
query43	557	557	541	541
query44	1155	712	716	712
query45	286	271	264	264
query46	1047	729	699	699
query47	1923	1854	1838	1838
query48	360	298	281	281
query49	703	375	360	360
query50	755	374	397	374
query51	6778	6587	6660	6587
query52	102	90	95	90
query53	343	276	282	276
query54	256	228	243	228
query55	76	73	70	70
query56	237	224	215	215
query57	1218	1154	1142	1142
query58	221	200	202	200
query59	3531	3164	3129	3129
query60	261	245	241	241
query61	97	92	97	92
query62	553	437	447	437
query63	303	278	278	278
query64	4213	4086	4010	4010
query65	3105	3015	3020	3015
query66	909	324	329	324
query67	15326	14834	14878	14834
query68	5413	530	517	517
query69	490	304	309	304
query70	1236	1122	1102	1102
query71	391	267	259	259
query72	6333	2677	2430	2430
query73	711	320	319	319
query74	6813	6353	6440	6353
query75	3405	2688	2608	2608
query76	2865	1133	1120	1120
query77	412	249	246	246
query78	10852	10279	10258	10258
query79	3508	515	504	504
query80	1716	440	413	413
query81	517	236	235	235
query82	895	94	98	94
query83	298	170	204	170
query84	269	86	89	86
query85	1279	267	260	260
query86	474	322	305	305
query87	3466	3297	3242	3242
query88	5640	2258	2279	2258
query89	468	378	372	372
query90	2071	172	171	171
query91	122	96	92	92
query92	58	47	48	47
query93	5595	514	486	486
query94	1171	181	178	178
query95	374	286	295	286
query96	618	257	258	257
query97	3079	2929	2951	2929
query98	239	225	214	214
query99	1162	870	880	870
Total cold run time: 272564 ms
Total hot run time: 183241 ms

[doris-robot]

ClickBench: Total hot run time: 30.48 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d5fbcfd0b5011c47cec8cac15a06ee160be9d5fa, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.03
query3	0.23	0.06	0.05
query4	1.68	0.09	0.09
query5	0.50	0.48	0.49
query6	1.48	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.51	0.50
query10	0.55	0.56	0.54
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.60	0.59	0.58
query14	0.76	0.77	0.76
query15	0.83	0.80	0.80
query16	0.40	0.36	0.37
query17	0.94	0.97	1.01
query18	0.21	0.26	0.23
query19	1.76	1.64	1.74
query20	0.01	0.01	0.01
query21	15.73	0.66	0.64
query22	4.21	7.79	2.02
query23	18.28	1.25	1.24
query24	1.69	0.26	0.22
query25	0.15	0.08	0.08
query26	0.25	0.15	0.16
query27	0.08	0.07	0.08
query28	13.43	0.99	0.99
query29	13.83	3.38	3.35
query30	0.23	0.05	0.06
query31	2.88	0.37	0.36
query32	3.30	0.46	0.46
query33	2.83	2.89	2.86
query34	17.14	4.38	4.44
query35	4.43	4.49	4.67
query36	0.65	0.45	0.45
query37	0.17	0.15	0.16
query38	0.17	0.14	0.16
query39	0.05	0.03	0.04
query40	0.16	0.15	0.16
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.83 s
Total hot run time: 30.48 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d5fbcfd0b5011c47cec8cac15a06ee160be9d5fa with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.3 seconds inserted 10000000 Rows, about 751K ops/s