Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lightning: add a note for physical import mode (#14108) #14127

Merged
merged 3 commits into from
Jul 5, 2023
Merged

lightning: add a note for physical import mode (#14108) #14127

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8188eab
lightning: add a note for physical import mode
Oreoxmt Jul 4, 2023
f7628d1
Update migrate-large-mysql-to-tidb.md
hfxsd Jul 4, 2023
7b9e962
Add a link
lilin90 Jul 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions migrate-large-mysql-to-tidb.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,7 @@ summary: Learn how to migrate MySQL of large datasets to TiDB.

When the data volume to be migrated is small, you can easily [use DM to migrate data](/migrate-small-mysql-to-tidb.md), both for full migration and incremental replication. However, because DM imports data at a slow speed (30~50 GiB/h), when the data volume is large, the migration might take a long time. "Large datasets" in this document usually mean data around one TiB or more.

This document describes how to migrate large datasets from MySQL to TiDB. The whole migration has two processes:

1. *Full migration*. Use Dumpling and TiDB Lightning to perform the full migration. TiDB Lightning's **local backend** mode can import data at a speed of up to 500 GiB/h.
2. *Incremental replication*. After the full migration is completed, you can replicate the incremental data using DM.
This document describes how to perform the full migration using Dumpling and TiDB Lightning. TiDB Lightning [Physical Import Mode](/tidb-lightning/tidb-lightning-physical-import-mode.md) can import data at a speed of up to 500 GiB/h. Note that this speed is affected by various factors such as hardware configuration, table schema, and the number of indexes. After the full migration is completed, you can replicate the incremental data using DM.

## Prerequisites

Expand Down
Loading