Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve dir store for performance #52

Open
muhamadazmy opened this issue May 17, 2024 · 0 comments
Open

improve dir store for performance #52

muhamadazmy opened this issue May 17, 2024 · 0 comments
Assignees
Milestone

Comments

@muhamadazmy
Copy link
Member

Right now dir store stores all the blobs flat under the store dir path. (all blobs in one directory) which makes the filesystem slow (too much files)

A better layout is to create 2 levels as follows

/ 
  - [XX] / XX...
  -           XX...

Simply use the fist byte of the hex as a directory name, so we have subdirs with 00, 01, 02 ... FF as a sub directory then each dir of these will then has only the blobs that are prefixed with that dir name

IMPORTANT: make sure that on get the store can still use this format and the old format for compatibility

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Verification
Development

No branches or pull requests

3 participants