Skip to content

Commit

Permalink
update ticdc alert rule (#13812) (#14192)
Browse files Browse the repository at this point in the history
  • Loading branch information
ti-chi-bot authored Jul 11, 2023
1 parent e3a5f5e commit 9d653c4
Showing 1 changed file with 4 additions and 18 deletions.
22 changes: 4 additions & 18 deletions ticdc/ticdc-alert-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ For critical alerts, you need to pay close attention to abnormal monitoring metr

- Alert rule:

(time() - ticdc_processor_checkpoint_ts / 1000) > 600
(time() - ticdc_owner_checkpoint_ts / 1000) > 600

- Description:

Expand All @@ -30,7 +30,7 @@ For critical alerts, you need to pay close attention to abnormal monitoring metr

- Alert rule:

(time() - ticdc_processor_resolved_ts / 1000) > 300
(time() - ticdc_owner_resolved_ts / 1000) > 300

- Description:

Expand Down Expand Up @@ -72,25 +72,11 @@ Warning alerts are a reminder for an issue or error.

Collect TiCDC logs to locate the root cause.

### `ticdc_mounter_unmarshal_and_mount_time_more_than_1s`
### `cdc_sink_flush_duration_time_more_than_10s`

- Alert rule:

`histogram_quantile(0.9, rate(ticdc_mounter_unmarshal_and_mount_bucket[1m])) * 1000 > 1000`

- Description:

It takes a replication task more than 1 second to unmarshal the data changes.

- Solution:

Collect TiCDC logs to locate the root cause.

### `cdc_sink_execute_duration_time_more_than_10s`

- Alert rule:

`histogram_quantile(0.9, rate(ticdc_sink_txn_exec_duration_bucket[1m])) > 10`
`histogram_quantile(0.9, rate(ticdc_sink_txn_worker_flush_duration[1m])) > 10`

- Description:

Expand Down

0 comments on commit 9d653c4

Please sign in to comment.