storage-users high memory usage when S3ng multipart uploads are enabled

[wkloucek]

Originally posted by @butonic in #8589 (comment)

so now I see the storageusers pods being OOM killed as described in #9656 (comment)

I currently think we are running into kubernetes/kubernetes#43916 (comment)

edit: @wkloucek pointed out that we have to disable mime multipart uploads because it allocates too much memory

[wkloucek]

I created this separate issue to put more attention to the high memory usage of the S3ng multipart upload.

[wkloucek]

Background on the used library: minio/minio-go#1496

[wkloucek]

Also tracked in https://kiteworks.atlassian.net/browse/OCSRE-100

[micbar]

@wkloucek Is this a blocker? This has not changed between ocis 5 and ocis 7

[wkloucek]

@wkloucek Is this a blocker? This has not changed between ocis 5 and ocis 7

Yes, still it got some visibility during the last month, so priorities might change at some point. I added it to Would be nice: on the release blockers ticket for some additional visibility, but I'm also fine with removing it again.

[micbar]

As far as i remember, the issue is quite tricky:

1. When using multipart uploads the bucket policy requires the S3 client to send checksums for each part
2. In order to calculate the checksums of the part (ocis knows the checksum of the whole file), the client needs a buffer to calculate this checksum
3. That means, we will need significantly more memory when doing multipart uploads. It depends a bit on the sizes of the parts and how many are uploaded in parallel. (see upstream discussion for optimisation), but in general we cannot avoid that.