[Subtask] Refactoring the secondary storage implementation in HiveCatalog #318
Comments
xunliu
pushed a commit
that referenced
this issue
Sep 24, 2023
…guarantee SSOT (#403) ### What changes were proposed in this pull request? This is the final work of #250 , with this PR there're several major refactorings: 1. Removing all the entity store operations in HiveCatalogOperation, which makes each CatalogOperation only focus on its own logic. 2. Processing all the additional metadata information in CatalogOperationDispatcher, also guarantees the SSOT. 3. Refactor the BaseXXX (BaseTable, BaseSchema and BaseColumn), to separate the metadata logics from entity information. 4. With all the above changes, changing the UTs accordingly. ### Why are the changes needed? With this PR, we have several advantages: 1. No need to handle entity store operations in each catalog, unify all of them in core module. 2. Remove the complex transaction semantics, using SSOT best effort mechanism. Fix: #318 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Adding new UTs to cover the code
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the subtask
This task tracks the work of refactoring secondary storage implementation in HiveCatalog.
Currently, we have a bunch of audit info related implementations in HiveCatalog, as we mentioned in #317 , we're going to push audit info related code into catalog, so this part of code may not be necessary.
Parent issue
#250
The text was updated successfully, but these errors were encountered: