Skip to content

Commit

Permalink
sync-diff-inspector: add descriptions and examples of table routers (#…
Browse files Browse the repository at this point in the history
  • Loading branch information
lyzx2001 authored Dec 14, 2023
1 parent d21c180 commit 302dfef
Showing 1 changed file with 132 additions and 3 deletions.
135 changes: 132 additions & 3 deletions sync-diff-inspector/route-diff.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ aliases: ['/docs/dev/sync-diff-inspector/route-diff/','/docs/dev/reference/tools

When using replication tools such as [TiDB Data Migration](/dm/dm-overview.md), you can set `route-rules` to replicate data to a specified table in the downstream. sync-diff-inspector enables you to verify tables with different schema names or table names by setting `rules`.

The following is a simple configuration example. To learn the complete configuration, refer to [Sync-diff-inspector User Guide](/sync-diff-inspector/sync-diff-inspector-overview.md).
The following is a simple configuration example. To learn the complete configuration, refer to [sync-diff-inspector User Guide](/sync-diff-inspector/sync-diff-inspector-overview.md).

```toml
######################### Datasource config #########################
Expand Down Expand Up @@ -58,6 +58,135 @@ target-schema = "test_2" # The name of the schema in the target database
target-table = "t_2" # The name of the target table
```

## Note
## The initialization of table routers and some examples

If `test_2`.`t_2` exists in the upstream database, the downstream database also compares this table.
### The initialization of table routers

- If a `target-schema/target-table` table named `schema.table` exists in the rules, the behavior of sync-diff-inspector is as follows:

- If there is a rule that matches `schema.table` to `schema.table`, sync-diff-inspector does nothing.
- If there is no rule that matches `schema.table` to `schema.table`, sync-diff-inspector will add a new rule `schema.table -> _no__exists__db_._no__exists__table_` to the table router. After that, sync-diff-inspector will treat the table `schema.table` as the table `_no__exists__db_._no__exists__table_`.

- If `target-schema` exists only in the rules as follows:

```toml
[routes.rule1]
schema-pattern = "schema_2" # the schema to match. Support wildcard characters * and ?
target-schema = "schema" # the target schema
```

- If there is no schema `schema` in the upstream, sync-diff-inspector does nothing.
- If there is a schema `schema` in the upstream, and a rule matches the schema, sync-diff-inspector does nothing.
- If there is a schema `schema` in the upstream, but no rule matches the schema, sync-diff-inspector will add a new rule `schema -> _no__exists__db_` to the table router. After that, sync-diff-inspector will treat the table `schema` as the table `_no__exists__db_`.

- If `target-schema.target-table` does not exist in the rules, sync-diff-inspector will add a rule to match `target-schema.target-table` to `target-schema.target-table` to make it case-insensitive, because the table router is case-insensitive.

### Examples

Suppose there are seven tables in the upstream cluster:

- `inspector_mysql_0.tb_emp1`
- `Inspector_mysql_0.tb_emp1`
- `inspector_mysql_0.Tb_emp1`
- `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_1.tb_emp1`
- `inspector_mysql_1.Tb_emp1`
- `Inspector_mysql_1.Tb_emp1`

In the configuration example, the upstream cluster has a rule `Source.rule1`, and the target table is `inspector_mysql_1.tb_emp1`.

#### Example 1

If the configuration is as follows:

```toml
[Source.rule1]
schema-pattern = "inspector_mysql_0"
table-pattern = "tb_emp1"
target-schema = "inspector_mysql_1"
target-table = "tb_emp1"
```

The routing results will be as follows:

- `inspector_mysql_0.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_0.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `inspector_mysql_0.Tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `inspector_mysql_1.tb_emp1` is routed to `_no__exists__db_._no__exists__table_`
- `Inspector_mysql_1.tb_emp1` is routed to `_no__exists__db_._no__exists__table_`
- `inspector_mysql_1.Tb_emp1` is routed to `_no__exists__db_._no__exists__table_`
- `Inspector_mysql_1.Tb_emp1` is routed to `_no__exists__db_._no__exists__table_`

#### Example 2

If the configuration is as follows:

```toml
[Source.rule1]
schema-pattern = "inspector_mysql_0"
target-schema = "inspector_mysql_1"
```

The routing results will be as follows:

- `inspector_mysql_0.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_0.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `inspector_mysql_0.Tb_emp1` is routed to `inspector_mysql_1.Tb_emp1`
- `inspector_mysql_1.tb_emp1` is routed to `_no__exists__db_._no__exists__table_`
- `Inspector_mysql_1.tb_emp1` is routed to `_no__exists__db_._no__exists__table_`
- `inspector_mysql_1.Tb_emp1` is routed to `_no__exists__db_._no__exists__table_`
- `Inspector_mysql_1.Tb_emp1` is routed to `_no__exists__db_._no__exists__table_`

#### Example 3

If the configuration is as follows:

```toml
[Source.rule1]
schema-pattern = "other_schema"
target-schema = "other_schema"
```

The routing results will be as follows:

- `inspector_mysql_0.tb_emp1` is routed to `inspector_mysql_0.tb_emp1`
- `Inspector_mysql_0.tb_emp1` is routed to `Inspector_mysql_0.tb_emp1`
- `inspector_mysql_0.Tb_emp1` is routed to `inspector_mysql_0.Tb_emp1`
- `inspector_mysql_1.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_1.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `inspector_mysql_1.Tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_1.Tb_emp1` is routed to `inspector_mysql_1.tb_emp1`

#### Example 4

If the configuration is as follows:

```toml
[Source.rule1]
schema-pattern = "inspector_mysql_?"
table-pattern = "tb_emp1"
target-schema = "inspector_mysql_1"
target-table = "tb_emp1"
```

The routing results will be as follows:

- `inspector_mysql_0.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_0.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `inspector_mysql_0.Tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `inspector_mysql_1.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_1.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `inspector_mysql_1.Tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_1.Tb_emp1` is routed to `inspector_mysql_1.tb_emp1`

#### Example 5

If you do not set any rules, the routing results will be as follows:

- `inspector_mysql_0.tb_emp1` is routed to `inspector_mysql_0.tb_emp1`
- `Inspector_mysql_0.tb_emp1` is routed to `Inspector_mysql_0.tb_emp1`
- `inspector_mysql_0.Tb_emp1` is routed to `inspector_mysql_0.Tb_emp1`
- `inspector_mysql_1.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_1.tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `inspector_mysql_1.Tb_emp1` is routed to `inspector_mysql_1.tb_emp1`
- `Inspector_mysql_1.Tb_emp1` is routed to `inspector_mysql_1.tb_emp1`

0 comments on commit 302dfef

Please sign in to comment.