Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Files are removed from s3 during sync #9

Open
mefju opened this issue Jan 5, 2018 · 0 comments
Open

Files are removed from s3 during sync #9

mefju opened this issue Jan 5, 2018 · 0 comments

Comments

@mefju
Copy link

mefju commented Jan 5, 2018

I have directory with 5 files:

ls -al tests
total 28
drwxrwxr-x 2 user user 4096 sty  5 13:43 .
drwxrwxr-x 4 user user 4096 sty  5 14:00 ..
-rw-rw-r-- 1 user user   36 sty  5 13:56 1
-rw-rw-r-- 1 user user   36 sty  5 13:43 2
-rw-rw-r-- 1 user user   36 sty  5 13:43 3
-rw-rw-r-- 1 user user   36 sty  5 13:43 4
-rw-rw-r-- 1 user user   36 sty  5 13:43 5

when I run sync (AWS_SECRET_KEY, AWS_ACCESS_KEY configured), files are copied to bucket::

./s3-cli --skip-existing -v sync tests s3://my_bucket_name/
Copy file://tests/2 -> s3://my_bucket_name/tests/2
Copy file://tests/1 -> s3://my_bucket_name/tests/1
0 B /Copy file://tests/3 -> s3://my_bucket_name/tests/3
Copy file://tests/4 -> s3://my_bucket_name/tests/4
0 B / 16 files to consider - 180 bytes
0 B / 180 B (0.0%)   0 B/secRemove s3://my_bucket_name/tests/
Copy file://tests/5 -> s3://my_bucket_name/tests/5
180 B / 180 B (100.0%)   86 B/sec
./s3-cli -v ls s3://my_bucket_name/tests/
2018-01-05 15:43        36   s3://my_bucket_name/tests/1
2018-01-05 15:43        36   s3://my_bucket_name/tests/2
2018-01-05 15:43        36   s3://my_bucket_name/tests/3
2018-01-05 15:43        36   s3://my_bucket_name/tests/4
2018-01-05 15:43        36   s3://my_bucket_name/tests/5

When I try sync second time (without changing anything in tests directory or bucket), all files are removed from bucked:

./s3-cli --skip-existing -v sync tests s3://my_bucket_name/
Copy file://tests/4 -> s3://my_bucket_name/tests/4
Copy file://tests/3 -> s3://my_bucket_name/tests/3
0 B Copy file://tests/5 -> s3://my_bucket_name/tests/5
0 B / 108 BCopy file://tests/1 -> s3://my_bucket_name/tests/1
0 B / 180 B (0.0%)   0 B/sec10 files to consider - 180 bytes
Remove s3://my_bucket_name/tests/5
Remove s3://my_bucket_name/tests/1
Remove s3://my_bucket_name/tests/2
Remove s3://my_bucket_name/tests/3
Remove s3://my_bucket_name/tests/4
Copy file://tests/2 -> s3://my_bucket_name/tests/2
180 B / 180 B (100.0%)   144 B/sec

if between first and second sync I add something to directory, files in bucket are randomly deleted

./s3-cli -v ls s3://my_bucket_name/tests/
[no output] - files deleted from bucket
Ubuntu 16.04.3 LTS
s3-cli version 0.2.0
go version go1.6.2 linux/amd64
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant