Duration of analyze partition table increased more than 50x

[apollodafoni]

Bug Report

What did you do?

1. create partition table

CREATE TABLE t1 (
id int(11) NOT NULL AUTO_INCREMENT,
k int(11) NOT NULL DEFAULT '0',
c char(120) NOT NULL DEFAULT '',
pad char(60) NOT NULL DEFAULT '',
PRIMARY KEY (id) /*T![clustered_index] CLUSTERED */
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin AUTO_INCREMENT=100279
PARTITION BY HASH (id) PARTITIONS 1000

2. insert 100000 rows data
3. analyze table t1

What did you expect to see?

Analyze finishes in a shorter time.
According to previous test experience, it is within 100s.
The most recent successful test was 37s.

What did you see instead?

"analyze time: "=1929.67098683
Sometimes it can be as long as 4,000 seconds .
A lot of time is spent on: merging global stats for test.t1 columns
The performance degradation is very serious

What version of PD are you using (pd-server -V)?

After testing, it was confirmed that this PR(#8377) caused it

[apollodafoni]

/severity critical

[apollodafoni]

/assign @rleungx

[rleungx]

The issue happens due to:

1. There is only one TiKV with 3 replicas setting, which results in keeping adding replicas but fails. It costs around one core
2. PD only has one core used up because of 1.
3. The previous cache has a TTL mechanism. After 3min, it will stop adding replicas

The above reason causes TSO requests to become very slow and affects the analyze table speed.