-
Notifications
You must be signed in to change notification settings - Fork 339
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PXB-3034 Reduce the time the instance remains under lock #1530
base: reducedlock-trunk
Are you sure you want to change the base?
Commits on Dec 15, 2023
-
PXB-3034 - Make --lock-ddl option an ENUM
https://jira.percona.com/browse/PXB-3034 Changed lock-ddl option to be an enum. Possible Values are: ON - Same as True OFF - Same as False REDUCED - Enable REDUCED lock mode. The first attempt to copy IBD files are done without locking and changed tables (Affected by DDL) are recopied (if needed) under DDL.
Configuration menu - View commit details
-
Copy full SHA for dec6cab - Browse repository at this point
Copy the full SHA dec6cabView commit details -
PXB-3034 - Add DDL tracking to xtrabackup
https://jira.percona.com/browse/PXB-3034 Add DDL tracking to xtrabackup. This new object is responsible for tracking DDL's while the backup is running. Later those changes will be handled in the end of backup and during prepare.
Configuration menu - View commit details
-
Copy full SHA for 464c7b6 - Browse repository at this point
Copy the full SHA 464c7b6View commit details -
https://jira.percona.com/browse/PXB-3034 Adjusted DDL tracking to produce correct files at the end of backup and handle those files during prepare.
Configuration menu - View commit details
-
Copy full SHA for 15d6004 - Browse repository at this point
Copy the full SHA 15d6004View commit details -
PXB-3034 - Second phase copy Multi thread
https://jira.percona.com/browse/PXB-3034 Added parallel copy capability to the second phase copy of .ibd files.
Configuration menu - View commit details
-
Copy full SHA for c3e0dd6 - Browse repository at this point
Copy the full SHA c3e0dd6View commit details -
https://jira.percona.com/browse/PXB-3034 Added test cases under suite/lockless Fixed test cases using --lock-ddl=false/true
Configuration menu - View commit details
-
Copy full SHA for 3d34e05 - Browse repository at this point
Copy the full SHA 3d34e05View commit details
Commits on Dec 20, 2023
-
PXB-3034 - adjust fil_open_for_xtrabackup
Adjusted fil_open_for_xtrabackup to tolerate file been gone and re-attempt to open the file 10 times.
Configuration menu - View commit details
-
Copy full SHA for 756795d - Browse repository at this point
Copy the full SHA 756795dView commit details
Commits on Apr 22, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 56a5adb - Browse repository at this point
Copy the full SHA 56a5adbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0f5401d - Browse repository at this point
Copy the full SHA 0f5401dView commit details -
We want to note that table will be copied only after it has been open…
…ed and loaded to cache; moving ddl_tracker->add_table to the correct spot
Configuration menu - View commit details
-
Copy full SHA for bedc607 - Browse repository at this point
Copy the full SHA bedc607View commit details
Commits on Apr 23, 2024
-
PXB-3113 : Improve debug sync framework to allow PXB to pause and res…
…ume threads https://perconadev.atlassian.net/browse/PXB-3113 The current debug-sync option in PXB completely suspends PXB process and user can resume by sending SIGCONT signal This is useful for scenarios where PXB is paused and do certain operations on server and then resume PXB to complete. But many bugs we found during testing, involves multiple threads in PXB. The goal of this work is to be able to pause and resume the thread. Since many tests use the existing debug-sync option, I dont want to disturb these tests. We can convert them to the new mechanism later. How to use? ----------- The new mechanism is used with option --debug-sync-thread="sync_point_name" In the code place a debug_sync_thread(“debug_point_1”) to stop thread at this place. You can pass the debug_sync point via commandline --debug-sync-thread=”debug_sync_point1” PXB will create a file of the debug_sync point name in the backup directory. It is suffixed with a threadnumber. Please ensure that no two debug_sync points use same name (it doesn’t make sense to have two sync points with same name) ``` 2024-03-28T15:58:23.310386-00:00 0 [Note] [MY-011825] [Xtrabackup] DEBUG_SYNC_THREAD: sleeping 1sec. Resume this thread by deleting file /home/satya/WORK/pxb/bld/backup//xb_before_file_copy_4860396430306702017 ``` In the test, after activating syncpoint, you can use wait_for_debug_sync_thread_point <syncpoint_name> Do some stuff now. This thread is sleeping. Once you are done, and if you want the thread to resume, you can do so by deleting the file 'rm backup_dir/sync_point_name_*` Please use resume_debug_sync_thread_point <syncpoint_name> <backup_dir>. It dletes the syncpoint file and additionally checks that syncpoint is indeed resumed. More common/complicated scenario: ---------------------------------- The scenario is to signal another thread to stop after reaching the first sync point. To achieve this. Do steps 1 to 3 (above) Echo the debug_sync point name into a file named “xb_debug_sync_thread”. Example: 4. echo "xtrabackup_copy_logfile_pause" > backup/xb_debug_sync_thread 5. send SIGUSR1 signal to PXB process. kill -SIGUSR1 496102 6. Wait for syncpoint to be reached. wait_for_debug_sync_thread <syncpoint_name> PXB acknowledges it 2024-03-28T16:05:07.849926-00:00 0 [Note] [MY-011825] [Xtrabackup] SIGUSR1 received. Reading debug_sync point from xb_debug_sync_thread file in backup directory 2024-03-28T16:05:07.850004-00:00 0 [Note] [MY-011825] [Xtrabackup] DEBUG_SYNC_THREAD: Deleting file/home/satya/WORK/pxb/bld/backup//xb_debug_sync_thread and then prints this once the sync point is reached. 2024-03-28T16:05:08.508830-00:00 1 [Note] [MY-011825] [Xtrabackup] DEBUG_SYNC_THREAD: sleeping 1sec. Resume this thread by deleting file /home/satya/WORK/pxb/bld/backup//xb_xtrabackup_copy_logfile_pause_10389933572825668634 At this point, we have two threads sleeping at two sync points. Either of them can be resumed by deleting the filenames mentioned in the error log. (Or use resume_debug_sync_thread())
Configuration menu - View commit details
-
Copy full SHA for 00ecb25 - Browse repository at this point
Copy the full SHA 00ecb25View commit details -
PXB-3252 : Xtrabackup failed to read page after 10 retries. File ./my…
…sql.ibd seems to be corrupted. https://perconadev.atlassian.net/browse/PXB-3252 Problem: -------- With lock-ddl=REDUCED, ALTER ENCRYPTION='Y'/'N' happens. On general tablespaces, this is done inplace. ie the space_id of tablespace will not change and the pages are encrypted or decrypted. For file per table tablespaces, a new tablespace is created with encryption key and data is copied from old tablespace to new tablespace. In xtrabackup, the files are discovered and then they are copied. Between these two operations, the encrypted tablespace can change. For example, PXB saw that ts1.ibd is encrypted with key1, loaded into cache. Then server did ENCRYPTION='N' and then back to ENCRYPTION='Y', now the tablspace is encrypted with a different key. Now PXB copy threads tries to copy this tablespce and cannot decrypt a page. Page 0 is always unencrypted. So the problem typically detected at Page 1. It can happen on any page. Since PXB cannot decrypt the page, it reports corruption and aborts the backup. Fix: ---- On decryption errors, we track such tablespaces with separate corrupted list. We also them to the recopy tables list. Under lock, these tablespaces are copied again. A .new extension is used. Then we process the corrupted list under lock. Create .corrupt files for the tablespaces from the corrupted list. For example, if the tablespace encrypted is ts1.ibd, the file will be ts1.ibd.corrupted. On prepare, we delete the corresponding ts1.ibd if the ts1.ibd.corrupted is present. This has to be done before the *.ibd scan becuase tablespace loading aborts on processing such half-written tablespaces. If the .corrupted is present in incremental directory, delete the ts1.ibd.meta and ts.ibd.delta files from the incremental backup directory.
Configuration menu - View commit details
-
Copy full SHA for 7a81fd0 - Browse repository at this point
Copy the full SHA 7a81fd0View commit details -
PXB-3246 : Assertion failure: log0recv.cc:2141:!page || fil_page_type…
…_is_index(page_type) Problem: -------- Unable to apply redo log record entry because page is in wrong state. It was observed that tablespace is created by incremental backup How did this happen? -------------------- lets say tablespace is t1.ibd and happily in fullbackup before incremental, this gets renamed to t2.ibd incremental backup creates t2.ibd.delta and t2.ibd.meta files in incremental backup directory later there is drop t2.ibd, we have space_id.del file in incremental backup directory also some redo generated on this table before it is dropped. During prepare of incremental backup, when we process a space_id.del file, we check the tablespace if tablespace is found. Lets say, it 2.del. To process 2.del, we first check, the tabespace that is with space_id 2. Since the tablespace name is t1.ibd in the full backup directory, we delete it. Additionally, we delete the .ibd and .meta files, so we try to delete t1.ibd.meta and t1.ibd.delta files. They never existed, so we ignore the errors to delete them. But in the inc backup directory, we still have t2.ibd.delta and t2.ibd.meta files. So inc backup prepare creates a tablespace with space_id 2 and apply the delta file changes. This tablespace is wrong because, we are creating a dropped tablespace and we dont have all the changes. incremental backup creates this tablespace with all-zero 7 pages. Later when we do MLOG_INSERT into the index page, we find out the page is NOT in correct state. Fix: ---- We have to delete the right incremental files based on space_id. So we build metamap by scanning *.meta files and with the key as space_id (found in meta file). Later, when we process the space_id.del file, after removing the tablespace with space_id, we will now ask aka meta map cache to give the .delta and .meta file belonging to deleted space_id. By deleting the un-necessary .meta file and .delta, the tablespace is considred as dropped by redo and corresponding redo entries are not applied.
Configuration menu - View commit details
-
Copy full SHA for 4e50fe9 - Browse repository at this point
Copy the full SHA 4e50fe9View commit details -
PXB-3253 : [ERROR] [MY-012592] [InnoDB] Operating system error number…
… 2 in a file operation https://perconadev.atlassian.net/browse/PXB-3253 Problem: -------- Files disappear during backup with --lockd-ddl=reduced Analysis: --------- PXB open server files using os_file_create_simple_no_error_handling() via Fil_shard::open_file(), Fil_shard::get_file_size(), Datafile::open_read_only. This API doesn't tolerate file open errors. This particular bug occurs when the file disappeared after get_file_size() in Fil_shard::open_file(). (See the testcase for more details). Fix: ---- If lock ddl is reduced and if we have not yet acquired/entered the copy under lock phase ie is_server_locked() is false, we can tolerate the file open errors. So we use the function/API os_file_create() instead of other variants. Within this, based on lock_ddl reduced mode, we tolerate file opening errors.
Configuration menu - View commit details
-
Copy full SHA for 3925f5b - Browse repository at this point
Copy the full SHA 3925f5bView commit details -
PXB-3223 : PXB must not allow --lock-ddl=REDUCED when pagetracking is…
… enabled Problem: ------- We cannot allow pagetracking with lock-ddl=REDUCED. This is because page-tracking gives us a set of page_ids (space_id, page_nos). PXB should copy these pages and while we copy these pages, tablespace disappear, get renamed, encrypted etc. We will enable it if there is need or usecase for this. For now, we will disable it. Fix: ---- Disable the combination of --page-tracking and --lock-ddl=REDUCED
Configuration menu - View commit details
-
Copy full SHA for 121b023 - Browse repository at this point
Copy the full SHA 121b023View commit details -
PXB-3120 : Assertion failure: Dir_Walker::is_directory
Problem: -------- InnoDB assumes directories or files do not disappear. It is true for the engine because, it is in the startup mode and no opeartions are allowed at this point of time. Analysis: --------- With lock-ddl=RECUCED, tables can be dropped concurrently when pxb does *.ibd scan or subdirectories can disappear too. Fix: ---- Handle walk_posix() for missing files/directories. The scan should continue and skip these deleted files or directories.
Configuration menu - View commit details
-
Copy full SHA for e71924a - Browse repository at this point
Copy the full SHA e71924aView commit details
Commits on Apr 25, 2024
-
PXB-3278 : Wrong parsing of MLOG_FILE_ redo log records with lock-ddl…
…=REDUCED Problem: -------- ddl_tracker_t::backup_file_op assumes the required redo bytes are always present. see the assertion len < 6. But it may happen we sometimes receive redo less than that. In such cases, we return nullptr and let the caller read more read and retry Fix: ---- fil_tablespace_redo_create()/rename()/delete() variants handle this problem by returning nullptr and reading more redo and retry. Moved ddl tracker calls to track after the validation is done.
Configuration menu - View commit details
-
Copy full SHA for 80ea4bb - Browse repository at this point
Copy the full SHA 80ea4bbView commit details -
Merge pull request #1552 from satya-bodapati/dev-reducedlock
PXB-3278 : Wrong parsing of MLOG_FILE_ redo log records with lock-ddl…
Configuration menu - View commit details
-
Copy full SHA for 1d1607f - Browse repository at this point
Copy the full SHA 1d1607fView commit details -
PXB-3281 : With lock-ddl=REDUCED, STL containers used by reduced code…
… are not thread safe Problem: ------- xtrabackup uses multiple threads to scan the *.ibd files. With lock-ddl=reduced, we use several STL maps to track of missing, dropped or renamed tables. Multiple threads are used only when number of IBDs are more than 8K Unsafe calls: ddl_tracker->add_missing_table(phy_filename); ddl_tracker->add_renamed_table(space_id, path); These calls from multiple threads operate on std::map/unordered_map and can cause race conditions. Fix: ---- 1. stream line mutex usage for entire ddl_tracker class. Currently used only for corrupted STL map. 2. Use space id instead of table id in messages 3. Rename add_table() since the name is confusing. Actual map elements should be renamed. it will be done later
Configuration menu - View commit details
-
Copy full SHA for d60a02c - Browse repository at this point
Copy the full SHA d60a02cView commit details -
Merge pull request #1553 from satya-bodapati/dev-reducedlock
PXB-3281 : With lock-ddl=REDUCED, STL containers used by reduced code…
Configuration menu - View commit details
-
Copy full SHA for b5f4017 - Browse repository at this point
Copy the full SHA b5f4017View commit details
Commits on Apr 26, 2024
-
PXB-3241 : Assertion failure: os0file.cc:3416:!exists while taking ba…
…ckups with lock-ddl=REDUCED Problem: -------- Backups taken with lock-ddl=reduced, prepare failed to complete. Analysis: --------- When handling .ren files, the destination file name already exists and this causes assertion failure. See the below backup log ``` 102: 2024-02-28T12:15:49.061631-00:00 1 [Note] [MY-011825] [Xtrabackup] DDL tracking : LSN: 73749548 create table ID: 788 Name: test/#sql-1fc79d_13#p#p3.ibd 423: 2024-02-28T12:15:50.121767-00:00 1 [Note] [MY-011825] [Xtrabackup] DDL tracking : LSN: 74312497 rename table ID: 425 From: test/tt_28_p#p#p3.ibd To: test/#sql2-1fc79d-13#p#p3.ibd 870: 2024-02-28T12:15:50.609015-00:00 2 [Note] [MY-011825] [Xtrabackup] Copying ./test/#sql-1fc79d_13#p#p3.ibd to /home/mohit.joshi/dbbackup_28_02_2024/full/test/#sql-1fc79d_13#p#p3.ibd 1337 2024-02-28T12:15:51.183699-00:00 1 [Note] [MY-011825] [Xtrabackup] DDL tracking : LSN: 74967007 rename table ID: 788 From: test/#sql-1fc79d_13#p#p3.ibd To: test/tt_28_p#p#p3.ibd 1491: 2024-02-28T12:15:51.398615-00:00 2 [Note] [MY-011825] [Xtrabackup] Copying ./test/tt_28_p#p#p3.ibd to /home/mohit.joshi/dbbackup_28_02_2024/full/test/tt_28_p#p#p3.ibd 2115: 2024-02-28T12:15:52.209645-00:00 1 [Note] [MY-011825] [Xtrabackup] DDL tracking : LSN: 75267178 delete table ID: 425 Name: test/#sql2-1fc79d-13#p#p3.ibd ``` Whats going on here? Lets say we have partition p3 with space id 425. This is being altered. So partition algorithm does this: 1. create new copy of space_id: 788 (#sql1). 2. rename the existing table 425 to some temp taname (#sql2) 3. we copied the new copy space_id 788 (#sql1) to backup. 4. we also copied the space_id 425 with original name (p3). 5. Later we saw a rename file for the copied tablespace 788. (788.ren created with destination name as tt_28_p#p#p3.ibd The rename file for space_id 425 is skipped, because we knowthat it is dropped. So only a .del file. Final state of backup is: === 788 in backup with name #sql1 425 in backup with name p3 788.ren file-> 788 From: test/#sql-1fc79d_13#p#p3.ibd To: test/tt_28_p#p#p3.ibd 425.drop file === Now prepare starts to process .ren files it tries to rename 788 from #sql1 to p3. but p3 already exists.. Fix: ---- we skip rename and other operations if we know that tablespace is going to be dropped. In the above example, we skipped 425.ren file. So while preparing, we should handle the .del files first. Then we are applying all the consolidated operations in a way. Then .ren can be processed
Configuration menu - View commit details
-
Copy full SHA for 17121fe - Browse repository at this point
Copy the full SHA 17121feView commit details -
Merge pull request #1554 from satya-bodapati/dev-reducedlock
PXB-3241 : Assertion failure: os0file.cc:3416:!exists while taking ba…
Configuration menu - View commit details
-
Copy full SHA for 36f8cbf - Browse repository at this point
Copy the full SHA 36f8cbfView commit details -
PXB-3245 : Assertion failure: fil0fil.cc:2545:err == DB_SUCCESS found…
… during incremental backup with lock-ddl=REDUCED Problem: -------- File deleted between PXB discovery and opening the file. This time at Fil_shard::create_node. It insists the file to be found. Fix: ---- 1. Tolerate the file missing error 2. Use different error code to track in missing files 3. free the tablespace object on error (otherwise, if fil_space_t remians in cache, pxb will try to copy the file)
Configuration menu - View commit details
-
Copy full SHA for d5a4245 - Browse repository at this point
Copy the full SHA d5a4245View commit details -
Merge pull request #1555 from satya-bodapati/dev-reducedlock
PXB-3245 : Assertion failure: fil0fil.cc:2545:err == DB_SUCCESS
Configuration menu - View commit details
-
Copy full SHA for 1a6b48a - Browse repository at this point
Copy the full SHA 1a6b48aView commit details
Commits on May 2, 2024
-
PXB-3280 : undo log truncation causes assertion failure with reduced …
…lock Problem: ------- With lock-ddl=reduced and Concurrent undo truncations, xtrabackup fails with an assertion Analysis: --------- After truncation, the tablespace id of an undo tablespace id might change and xtrabackup returns error instead of crash. But higher layers of undo discovery do not tolerate missing files or errors. Fix: --- Tolerate file deletions during undo discovery
Configuration menu - View commit details
-
Copy full SHA for 594da1d - Browse repository at this point
Copy the full SHA 594da1dView commit details -
Merge pull request #1556 from satya-bodapati/dev-reducedlock
PXB-3280 : undo log truncation causes assertion failure with reduced …
Configuration menu - View commit details
-
Copy full SHA for c0302e8 - Browse repository at this point
Copy the full SHA c0302e8View commit details
Commits on May 15, 2024
-
PXB-3248 Multiple files found for the same tablespace ID
On prepare phase when handling ddl files some of the data files were not loaded to cache because of the first page validation therefore were left without applying ddls on them. To tolerate this issue we should open and load data files to cache without validation, to do so we are using fil_tablespace_open_for_recovery() function instead of fil_open_for_xtrabackup().
Configuration menu - View commit details
-
Copy full SHA for e95cb93 - Browse repository at this point
Copy the full SHA e95cb93View commit details -
Merge pull request #1557 from aybek/dev-reducedlock-trunk
PXB-3248 Multiple files found for the same tablespace ID
Configuration menu - View commit details
-
Copy full SHA for 4b9f897 - Browse repository at this point
Copy the full SHA 4b9f897View commit details -
PXB-3248 Multiple files found for the same tablespace ID
1. Fix bugs in the previous commit for this issu 2. Remove macro checks for debug_sync_thread()
Configuration menu - View commit details
-
Copy full SHA for caa4040 - Browse repository at this point
Copy the full SHA caa4040View commit details -
Merge pull request #1558 from aybek/dev-reducedlock-trunk
Removing UNIV_DEBUG macro check for debug_sync_thread()
Configuration menu - View commit details
-
Copy full SHA for 23136ed - Browse repository at this point
Copy the full SHA 23136edView commit details
Commits on May 29, 2024
-
PXB-3248 - Multiple files found for the same tablespace ID
1. Latest fix for PXB-3248 which is passing stress tests 2. Remove macro checks for debug_sync_thread() temporarily, for QA testing
Configuration menu - View commit details
-
Copy full SHA for 42cf484 - Browse repository at this point
Copy the full SHA 42cf484View commit details -
Merge pull request #1566 from aybek/dev-reducedlock-trunk2
PXB-3248 - Multiple files found for the same tablespace ID
Configuration menu - View commit details
-
Copy full SHA for 0108546 - Browse repository at this point
Copy the full SHA 0108546View commit details
Commits on Jun 7, 2024
-
PXB-3034: Bring back UNIV_DEBUG on debug-sync-thread.
This will be debug only option. Fix release build issues
Configuration menu - View commit details
-
Copy full SHA for b8373d5 - Browse repository at this point
Copy the full SHA b8373d5View commit details
Commits on Jun 9, 2024
-
Merge pull request #1570 from satya-bodapati/dev-reducedlock
PXB-3034: Bring back UNIV_DEBUG on debug-sync-thread.
Configuration menu - View commit details
-
Copy full SHA for d86332a - Browse repository at this point
Copy the full SHA d86332aView commit details
Commits on Jul 15, 2024
-
PXB-3320 : prepare_handle_del_files() fails to delete the .meta and .…
…delta files for deleted tablespaces in incremental backup directory Problem: -------- 1. take full backup with --lock-ddl=reduced 2. create table t1(a INT), lets say space_id 10 3. start incremental backup and pause before backup_start() function (we take Bakcup lock here) 4. incremental backup copied t1.ibd.meta and t1.ibd.delta by this time 5. DROP TABLE t1 6. resume the incremental backup. 10.del file is created 7. prepare the full backup with --apply-log-only 8. prepare incremental backup incremental backup prepare first processes the .del files. before this all tabelspaces are loaded via .ibd scan since there is no t1.ibd in backup directory( it is only present as meta and delta file) in incremental backup directory, space_id with 10 is not in cache. Hence prepare_handle_del_files() will not delete the files related to space_id 10. We end up with orphan .ibd or .ibu files. Server ignore orphan .ibd But if the tablespace is undo tablespace, orphan .ibu are not ignored by server. Server discovers them via *.ibu scan. This can lead to assertion failures.
Configuration menu - View commit details
-
Copy full SHA for a8eb932 - Browse repository at this point
Copy the full SHA a8eb932View commit details -
Merge pull request #1584 from satya-bodapati/dev-reducedlock
PXB-3320 : prepare_handle_del_files() fails to delete the .meta and .…
Configuration menu - View commit details
-
Copy full SHA for fc66022 - Browse repository at this point
Copy the full SHA fc66022View commit details
Commits on Jul 16, 2024
-
PXB-3318 : prepare_handle_ren_files(): failed to handle .ren files
Problem: ------- 1. take full backup 2. create table t1 before incremental backup 3. Take incremental backup under gdb and pause at backup_start 4. now rename t1 to t2 5. let it finish 6. prepare full 7. prepare incremental It happens because tables created between full and incremental are copied as .delta/*.meta files and not as IBD files. prepare_handle_ren_files() relies on *.ibd scan but this cannot work as the *.delta files are not yet loaded to fil_cache. Fix: ---- Use the meta_map generated from *.meta scan. Use the space_id from space_id.ren to identify the correct .meta and delta files. Rename the matched .meta and .delta files to the destination name stored in the .ren file
Configuration menu - View commit details
-
Copy full SHA for c7cd826 - Browse repository at this point
Copy the full SHA c7cd826View commit details -
Merge pull request #1585 from satya-bodapati/dev-reducedlock
PXB-3318 : prepare_handle_ren_files(): failed to handle .ren files
Configuration menu - View commit details
-
Copy full SHA for 1ad8fd9 - Browse repository at this point
Copy the full SHA 1ad8fd9View commit details
Commits on Jul 17, 2024
-
PXB-3295 : Undo tablespaces are not tracked properly with lock-ddl=RE…
…DUCED Problem: -------- 1. Undo tablespaces are not tracked properly. Since undo tablespaces are not opened via fil_open_for_xtrabackup(), they are not tracked as 'copied'. This leads to wrong decisions in handle_ddl_operations. 2. When new undo tablespaces are created, Server doesn't write a MLOG_FILE_CREATE record and so these are missed by the tracking system. Fix: ---- 1. track undo tablespaces that xtrabackup copies without lock (before lock state) 2. After lock is taken, undo tablespces are discovered again (after lock state) With this before and after states, we now determine undo files to be deleted, undo files to be copied. Truncated undo tablespace use different tablespace id, so old undo file is marked as deleted and new version of undo tablespace is copied For example undo_001.ibu of space_id 10 is truncated, the filename remains same but it space_id becomes 11. xtrabackup creates 11.ibu.del for undo tablespace to be deleted. then undo_001.ibu.new with space_id 11 is copied under lock
Configuration menu - View commit details
-
Copy full SHA for 620cd29 - Browse repository at this point
Copy the full SHA 620cd29View commit details
Commits on Jul 18, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 49c8a34 - Browse repository at this point
Copy the full SHA 49c8a34View commit details -
Merge pull request #1586 from satya-bodapati/dev-reducedlock
PXB-3295 : Undo tablespaces are not tracked properly with lock-ddl=RE…
Configuration menu - View commit details
-
Copy full SHA for 2fe6a4d - Browse repository at this point
Copy the full SHA 2fe6a4dView commit details -
Configuration menu - View commit details
-
Copy full SHA for f55da87 - Browse repository at this point
Copy the full SHA f55da87View commit details -
Merge pull request #1587 from satya-bodapati/dev-reducedlock
Follow up fix for PXB-3318: handle rename source and destination as s…
Configuration menu - View commit details
-
Copy full SHA for 596352f - Browse repository at this point
Copy the full SHA 596352fView commit details
Commits on Jul 19, 2024
-
PXB-3221 : Assertion failure: page0cur.cc:1177:ib::fatal triggered du…
…ring prepare/or next server startup Problem: -------- If there are tables created in system tablespace and if ALTER ADD INDEX/DROP INDEX is executed before the backup lock is taken, system tablespace could end up in corrupted state. This is because this operation is not redologged and we are supposed to recopy the system tablespace files. But we dont track system tablesapce, neither reopen and recopy them. Hence this issue. Fix: ---- 1. Track system tablespace in the list of tables tracked/backedup 2. Removing tracking for tables in system tablespace except of recopy. Other operations can be played via redolog 3. Reopen system tablespace and recopy them as ibdata1.new/ ibdata2.new
Configuration menu - View commit details
-
Copy full SHA for 3099f59 - Browse repository at this point
Copy the full SHA 3099f59View commit details -
Merge pull request #1588 from satya-bodapati/dev-reducedlock
PXB-3221 : Assertion failure: page0cur.cc:1177:ib::fatal triggered du…
Configuration menu - View commit details
-
Copy full SHA for d525604 - Browse repository at this point
Copy the full SHA d525604View commit details
Commits on Jul 22, 2024
-
PXB-3331 : Assertion failure: fil0fil.cc:6422:success
Problem: ------- During prepare, for backups taken with lock-ddl=ON, we did *.ibd scan before recovery. This is allowed only for lock-ddl=REDUCED. Fix: ---- During prepare, do *.ibd scan and processing of .new, .del, .ren, .corrupt files only if lock-ddl=REDUCED
Configuration menu - View commit details
-
Copy full SHA for 686c80c - Browse repository at this point
Copy the full SHA 686c80cView commit details -
Merge pull request #1590 from satya-bodapati/dev-reducedlock
PXB-3331 : Assertion failure: fil0fil.cc:6422:success
Configuration menu - View commit details
-
Copy full SHA for 3b41be9 - Browse repository at this point
Copy the full SHA 3b41be9View commit details