Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

init s3 storage #29

Merged
merged 44 commits into from
Oct 26, 2023
Merged

init s3 storage #29

merged 44 commits into from
Oct 26, 2023

Commits on Sep 15, 2023

  1. WIP: rewrite fl scehma

    @muhamadazmy
    muhamadazmy committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    c8d902b View commit details
    Browse the repository at this point in the history

  2. update routes

    @muhamadazmy
    muhamadazmy committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    3f39b68 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2023

  1. WIP: refactor for separate stores

    @muhamadazmy
    muhamadazmy committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    bb44194 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. Separation of cache and store 

    Also a lot of clean up and refactor
    @muhamadazmy
    muhamadazmy committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    1ce3cad View commit details
    Browse the repository at this point in the history

  2. WIP: pack function

    @muhamadazmy
    muhamadazmy committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    4281572 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. complete the pack/unpack cycle

    @muhamadazmy
    muhamadazmy committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    46d5880 View commit details
    Browse the repository at this point in the history

  2. Store needs to return their url and range

    @muhamadazmy
    muhamadazmy committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    0d959b9 View commit details
    Browse the repository at this point in the history

  3. fix bug in router and improve tests for pack/unpack

    @muhamadazmy
    muhamadazmy committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    917f2d8 View commit details
    Browse the repository at this point in the history

  4. Apply clippy

    @muhamadazmy
    muhamadazmy committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    00b2e3b View commit details
    Browse the repository at this point in the history

  5. Assert routes creation

    @muhamadazmy
    muhamadazmy committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    86ed118 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. improve tests

    @muhamadazmy
    muhamadazmy committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    7b973e2 View commit details
    Browse the repository at this point in the history

  2. dynamic creation of stores based on url

    @muhamadazmy
    muhamadazmy committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    8617e5e View commit details
    Browse the repository at this point in the history

  3. start fs mount based on stores from the fl

    @muhamadazmy
    muhamadazmy committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    833e781 View commit details
    Browse the repository at this point in the history

  4. first working version 

    Next: performance improvments
    @muhamadazmy
    muhamadazmy committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    f08ac50 View commit details
    Browse the repository at this point in the history

  5. implement create 

    also fix a file size bug
    @muhamadazmy
    muhamadazmy committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    5bdc98d View commit details
    Browse the repository at this point in the history

  6. use different impl pack

    @muhamadazmy
    muhamadazmy committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    4d909b0 View commit details
    Browse the repository at this point in the history

  7. use blake2

    @muhamadazmy
    muhamadazmy committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    fb6462a View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2023

  1. use blake2_simd implementation

    @muhamadazmy
    muhamadazmy committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    75db699 View commit details
    Browse the repository at this point in the history

  2. fix bug in parent ino

    @muhamadazmy
    muhamadazmy committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    cb5becc View commit details
    Browse the repository at this point in the history

  3. implement unpack with no recurssion 

    add cmdline to do unpacking
    @muhamadazmy
    muhamadazmy committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    f0390fe View commit details
    Browse the repository at this point in the history

  4. add support to use multiple stores from cmdlin

    @muhamadazmy
    muhamadazmy committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    0a706db View commit details
    Browse the repository at this point in the history

  5. clean up dependencies

    @muhamadazmy
    muhamadazmy committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    8df4b97 View commit details
    Browse the repository at this point in the history

  6. updating docs

    @muhamadazmy
    muhamadazmy committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    5cdd2b0 View commit details
    Browse the repository at this point in the history

  7. fix typos

    @muhamadazmy
    muhamadazmy committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    94d38fb View commit details
    Browse the repository at this point in the history

  8. implement preserver-ownership flag

    @muhamadazmy
    muhamadazmy committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    f27801f View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2023

  1. make sure on get stores are tried in random order 

    Also update docs
    @muhamadazmy
    muhamadazmy committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    cb795e6 View commit details
    Browse the repository at this point in the history

  2. fix ci

    @muhamadazmy
    muhamadazmy committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    14b12e4 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2023

  1. Parallel upload

    @muhamadazmy
    muhamadazmy committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    a8ed37b View commit details
    Browse the repository at this point in the history

  2. init s3 storage

    @Omarabdul3ziz
    Omarabdul3ziz committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    47d3ac1 View commit details
    Browse the repository at this point in the history

  3. better error handling and runnable code

    @Omarabdul3ziz
    Omarabdul3ziz committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    5bb0268 View commit details
    Browse the repository at this point in the history

  4. move to store directory and update cargo files

    @Omarabdul3ziz
    Omarabdul3ziz committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    b74ebd8 View commit details
    Browse the repository at this point in the history

  5. wip: refactor s3store to implement store and register it

    @Omarabdul3ziz
    Omarabdul3ziz committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    acdadc9 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. wip: use s3 crate and parse config from url

    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    991d4f9 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. Merge branch 'master' of https://github.com/threefoldtech/rfs into ma… 

    …ster_s3
    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    0e9865b View commit details
    Browse the repository at this point in the history

  2. update lock file

    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    80851b9 View commit details
    Browse the repository at this point in the history

  3. add test_get_config

    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    2f45892 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. add region_name to the url as a param and better error propagation

    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    cabe4a0 View commit details
    Browse the repository at this point in the history

  2. handle requests without tls and add test for setting and getting func… 

    …tions
    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    fd0af8a View commit details
    Browse the repository at this point in the history

  3. resolve comments: 

    - use the generic Error::Other error
- better support for both http and https
- more readable error handling
- hex encode the key in set/get
    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    4f63125 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2023

  1. - clearer errors 

    - add openssl crate
- handle the `=` sign for params in the first parser
    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    c6b78d5 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2023

  1. use regex to capture range and url

    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 8, 2023
    Configuration menu
    Copy the full SHA
    fbd1533 View commit details
    Browse the repository at this point in the history

  2. add regex crate

    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 8, 2023
    Configuration menu
    Copy the full SHA
    119f56d View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. update docs, and make region optional

    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    80064a5 View commit details
    Browse the repository at this point in the history

  2. change to unwrap_or_default

    @Omarabdul3ziz
    Omarabdul3ziz committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    047c4cd View commit details
    Browse the repository at this point in the history