Skip to content

Commit

Permalink
init s3 storage (#29)
Browse files Browse the repository at this point in the history
* WIP: rewrite fl scehma

* update routes

* WIP: refactor for separate stores

* Separation of cache and store

Also a lot of clean up and refactor

* WIP: pack function

* complete the pack/unpack cycle

* Store needs to return their url and range

* fix bug in router and improve tests for pack/unpack

* Apply clippy

* Assert routes creation

* improve tests

* dynamic creation of stores based on url

* start fs mount based on stores from the fl

* first working version

Next: performance improvments

* implement create

also fix a file size bug

* use different impl pack

* use blake2

* use blake2_simd implementation

* fix bug in parent ino

* implement unpack with no recurssion

add cmdline to do unpacking

* add support to use multiple stores from cmdlin

* clean up dependencies

* updating docs

* fix typos

* implement preserver-ownership flag

* make sure on get stores are tried in random order

Also update docs

* fix ci

* Parallel upload

* init s3 storage

* better error handling and runnable code

* move to store directory and update cargo files

* wip: refactor s3store to implement store and register it

* wip: use `s3` crate and parse config from url

* update lock file

* add test_get_config

* add region_name to the url as a param and better error propagation

* handle requests without tls and add test for setting and getting functions

* resolve comments:
- use the generic Error::Other error
- better support for both http and https
- more readable error handling
- hex encode the key in set/get

* - clearer errors
- add openssl crate
- handle the `=` sign for params in the first parser

* use regex to capture range and url

* add regex crate

* update docs, and make region optional

* change to `unwrap_or_default`

---------

Co-authored-by: Muhamad Azamy <[email protected]>
  • Loading branch information
Omarabdul3ziz and muhamadazmy authored Oct 26, 2023
1 parent 73a25d3 commit 51faf1f
Show file tree
Hide file tree
Showing 6 changed files with 533 additions and 28 deletions.
Loading

0 comments on commit 51faf1f

Please sign in to comment.