MD5 check isn't executed / is broken

[kaazoo]

Create testfile:

# dd if=/dev/zero bs=1M count=1024 of=testfile
1024+0 Datensätze ein
1024+0 Datensätze aus
1073741824 Bytes (1,1 GB) kopiert, 0,719243 s, 1,5 GB/s

Upload file:

# /root/gof3r_0.5.0_linux_amd64/gof3r put --debug -p testfile  -b mybucket -k _trash/testfile
2016/12/02 20:17:58 9 buffers of 20 MB allocated
2016/12/02 20:17:58 md5:  cd573cfaace07e7949bc0c46028904ff
2016/12/02 20:17:58 md5Path:  .md5/_trash/testfile.md5
duration: 14.297225998s

Download file again:

# /root/gof3r_0.5.0_linux_amd64/gof3r get --debug -p testfile2  -b mybucket -k _trash/testfile
2016/12/02 20:18:28 object size: 1e+03 MB
2016/12/02 20:18:43 Headers:  map[Last-Modified:[Fri, 02 Dec 2016 20:17:45 GMT] Etag:["a2c3aabf482d97af571f386756540311-52"] Content-Type:[binary/octet-stream] Content-Length:[1073741824] Server:[AmazonS3] X-Amz-Id-2:[7QMoa30bQB42gxVMwmblSzduHNeefNGgmaFZjr2Z7bwFOYILxCYdWi2iBhsV6N2q0fd46GRa87Q=] X-Amz-Request-Id:[A3463AE3FAF30D75] Date:[Fri, 02 Dec 2016 20:18:29 GMT] Accept-Ranges:[bytes]]
2016/12/02 20:18:43 md5:  cd573cfaace07e7949bc0c46028904ff
2016/12/02 20:18:43 md5Path:  .md5/_trash/testfile.md5
2016/12/02 20:18:43 13 buffers of 20 MB allocated
duration: 14.84860394s

Modify stored MD5 checksum:

# echo "foobar" | /root/gof3r_0.5.0_linux_amd64/gof3r put --debug  -b mybucket -k .md5/_trash/testfile.md5 --no-md5
2016/12/02 20:19:26 2 buffers of 20 MB allocated
duration: 273.27383ms

Download file again:

# /root/gof3r_0.5.0_linux_amd64/gof3r get --debug -p testfile2  -b mybucket -k _trash/testfile
2016/12/02 20:19:33 object size: 1e+03 MB
2016/12/02 20:19:52 Headers:  map[Content-Type:[binary/octet-stream] Accept-Ranges:[bytes] X-Amz-Request-Id:[DEBEAD24C9670CAB] Date:[Fri, 02 Dec 2016 20:19:34 GMT] Last-Modified:[Fri, 02 Dec 2016 20:17:45 GMT] Etag:["a2c3aabf482d97af571f386756540311-52"] Content-Length:[1073741824] Server:[AmazonS3] X-Amz-Id-2:[B3bbToVSwfk8RAoHim0Sl/PqQal9V/j0cEfdj9GypoM4/Vc991YIsSeGNgY8gKYIY+rpGCQguSc=]]
2016/12/02 20:19:52 md5:  cd573cfaace07e7949bc0c46028904ff
2016/12/02 20:19:52 md5Path:  .md5/_trash/testfile.md5
2016/12/02 20:19:52 13 buffers of 20 MB allocated
duration: 23.065767305s

Check return value:

# echo $?
0

Check stored checksum:

# /root/gof3r_0.5.0_linux_amd64/gof3r get --debug  -b mybucket -k .md5/_trash/testfile.md5 --no-md5
2016/12/02 20:20:28 object size: 6.7e-06 MB
foobar
2016/12/02 20:20:28 Headers:  map[X-Amz-Request-Id:[EC4DCA8943BC90C0] Last-Modified:[Fri, 02 Dec 2016 20:19:27 GMT] Content-Length:[7] Server:[AmazonS3] X-Amz-Id-2:[A1UklwYWk0NL/31f9NfEGfgjo/zsOmqGxUacs6JZrTAws1q6ydT3esI0afx5FshmgQF1inOMHh4=] Date:[Fri, 02 Dec 2016 20:20:29 GMT] Etag:["61b41eaba9f5ce4ef90abf78b33d814a-1"] Accept-Ranges:[bytes] Content-Type:[binary/octet-stream]]
duration: 161.653791ms

Check MD5 checksums of original and downloaded file:

# md5sum testfile*
cd573cfaace07e7949bc0c46028904ff  testfile
cd573cfaace07e7949bc0c46028904ff  testfile2

gof3r does return an error when stored checksum doesn't match the checksum of the downloaded file!

[kaazoo]

With version 0.3.2 the expected behavior is shown:

# gof3r get -p testfile2  -b mybucket -k _trash/testfile
2016/12/02 21:38:58 GET:  &{testfile2 {_trash%2Ftestfile mybucket false 10 20971520 s3.amazonaws.com false} }
2016/12/02 21:38:59 chunk total:  52
2016/12/02 21:38:59 content length :  1073741824
2016/12/02 21:38:59 concurrency:  10
2016/12/02 21:39:13 makes: 14
2016/12/02 21:39:13 md5:  cd573cfaace07e7949bc0c46028904ff
2016/12/02 21:39:13 md5Path:  .md5/_trash/testfile.md5
MD5 mismatch. given:foobar
 calculated:cd573cfaace07e7949bc0c46028904ff