[Improvement] Improve the metalake/catalog drop mechanism

[jerryshao]

What would you like to be improved?

Currently, when we issue a request to drop a metalake or a catalog, we just simply delete the related metadata from rocksdb, I think this leaves several points that need to improve:

1. We should make sure all sub-levels of entities cannot be accessed anymore.
2. We should have a mechanism to clean/disable the left metadata with a defined behavior.
3. The operation should be lightweight, this drop operation should not block for a long time.

For example, if we drop a metalake, we should make sure all catalogs under this metalake cannot be accessed anymore, and we should clean all the left catalog entities.

How should we improve?

No response

Sub tasks

• [Subtask] Adjust id name mapping to support true deletion and modification of entiies #305
• [Subtask] Refine deletion and update logics in the implementation of EntityStore #306
• [Subtask] Setting buffer time for deletion operations to prevent accidental deletion, etc. #319

[jerryshao]

@yuqi1129 can you please work on this issue, to propose a solution to handle this problem.

[yuqi1129]

Got

[yuqi1129]

Related design doc https://docs.google.com/document/d/1wj4932H1kGYpUlI865p_vRncnotpxkCRXFoeoRNEVIQ/edit
https://docs.google.com/document/d/1YqZ7RfBHsX1a2Ns9PaT34Ln60IO_TVRgCzZ7nRcmGpY/edit#heading=h.m66jgtc6gtj1

[yuqi1129]

@jerryshao
I have added a new doc about refactoring the whole entity encoding plan. After viewing and thinking thoroughly, currently entity encoding plan can hardly handle deletion and renaming. So I came up with the new solution shown in the above doc.
As it's a big change and may affect the whole project, please take time to review it and do not hesitate to give your suggestions. Please forgive me as some points may be neglected due to limited time.