"Connection closed" message when syncing files larger then +- 100Mb

[Jhutjens92]

I have my nextcloud installation running in a docker container. It's connected to a MySQL DB (another docker container) and exposed to the web using SWAG. All the relevant php.ini/config files have the following settings

• client_max_body_size set 0
• upload_max_filesize 16G
• post_max_size 16G
• max_input_time 3600
• max_execution_time 3600

Uploading via webserver is no problem but whenever i try to sync the same file using the windows sync client i receive a "Connection Closed"

Expected behaviour

Files should just upload to the nextcloud server.

Actual behaviour

Files aren't being uploaded and client throws an error:

Steps to reproduce

1. Run nextcloud in docker container.
2. Use SWAG (docker container) to link (sub)domain to nextcloud
3. Upload large file (larger then 100Mb) (in my case a .mp4)
4. See response in client log.

Client configuration

Client version: 3.4.2

Operating system: Microsoft Windows 10 Pro (10.0.19041 Build 19041)

OS language: Dutch

Installation path of client: C:\Program Files\Nextcloud

Nextcloud version: Nextcloud Hub II (23.0.0)

Storage backend: Local server storage

Logs

1. Client logfile:
   Client_20220213_2039_owncloud.log.0.txt

2. Web server error log:
   N.A.

3. Server logfile: nextcloud log (data/nextcloud.log):
   nextcloud.log

(ignore the failed login)

[metheis]

are you using Cloudflare in your setup? I'm running into the same issue because of Cloudflare's 100 MB upload limit and the client not seeming to chunk.
#4271

[Jhutjens92]

are you using Cloudflare in your setup? I'm running into the same issue because of Cloudflare's 100 MB upload limit and the client not seeming to chunk. #4271

I am using cloudflare for SSL on SWAG with my domain. I didn't even stop to think about it. Did you figure out a way around it/fix? Nextcloud is chunking the file (correct me if I'm wrong) so something is not working as intented. I could turn off cloudflare for my subdomain but that would defeat the purpose of using it in the first place.

[metheis]

Right, agreed, I prefer to keep Cloudflare on. What I found works is if I uninstall the client (I had version 3.4.1) and reinstall version 3.3.6 from October, the issue goes away. I think the issue must be with how the newer client chunks (or lack there of).

[Jhutjens92]

Version 3.3.6 is confirmed to be working. Just tested it out. So something in 3.4.1 broke the chunking.

[metheis]

@allexzander is it possible you could weigh in on this? Thank you for your help

[tusharraval102]

I can confirm that I have the same issue. I recently upgraded to 3.4.2 and my client app would not sync files larger than 100mb. I just downgraded to 3.3.6 and it works without any issues. I am using CloudFlare, MacOS Monterey, and the latest NextCloud server.

[metheis]

@Valdnet or @er-vin it seems like a number of users are running into this issue starting with 3.4.1 (and still present in 3.4.2). Any thoughts on what change lead to this?

[ThatTallGuy21]

I'm on 3.4.1 as well and utilize SSL via CF with Nginx. Running into the same issue as described.

[Vixavius]

I was experiencing the same issue with 3.4.2.

v3.3.6 working well.

[ghost]

This is happening to me, I am using cloudflare and the 100MB limit doesn't affect my upload but the client is bringing up errors when uploading files, i'm trying 3.3.6 now to see if this is just a bug

[ghost]

Yep I can confirm on my side aswell version 3.3.6 works fine then latest client broke chunking and possibly keep alive "closing connection"

[mitohund]

Can confirm that 3.4.3 does not work for me, but 3.3.6 does.

[Kn-ut99]

Same issue here with client v3.4.3 on Linux (Manjaro) + cloudflare.

[relains]

Same here on Android. If in cloudflare I check DNS only - it works, switch back to proxied - dosen't work.

[WasabiCarpet]

Dropping a comment here to say I'm experiencing the same issue as above and am also using cloudflare.

Edit: I'll add that reverting to 3.3.6 also fixed the issue for me and all my large files upload just fine now

[PaperMemo]

I randomly put this thing into nextcloud.cfg (in [General] section) and somehow it works for me. (I don't know if it works for other people or not)

chunkSize=10000000
minChunkSize=1000000
maxChunkSize=50000000
targetChunkUploadDuration=6000

Note: I use version 3.4.3 on Manjaro Linux
Note 2: I read from https://docs.nextcloud.com/desktop/3.0/advancedusage.html because, in the documentation, I didn't see [General] section in 3.4 🤔

[jospoortvliet]

that is interesting, if that works for others perhaps we can update the documentation at least.

[weeix]

I randomly put this thing into nextcloud.cfg (in [General] section) and somehow it works for me. (I don't know if it works for other people or not)

chunkSize=10000000
minChunkSize=1000000
maxChunkSize=50000000
targetChunkUploadDuration=6000

Note: I use version 3.4.3 on Manjaro Linux Note 2: I read from https://docs.nextcloud.com/desktop/3.0/advancedusage.html because, in the documentation, I didn't see [General] section in 3.4 🤔

I confirm that this works for Nextcloud 3.4.4 (Windows).

[Kn-ut99]

I randomly put this thing into nextcloud.cfg (in [General] section) and somehow it works for me. (I don't know if it works for other people or not)

chunkSize=10000000
minChunkSize=1000000
maxChunkSize=50000000
targetChunkUploadDuration=6000

Note: I use version 3.4.3 on Manjaro Linux Note 2: I read from https://docs.nextcloud.com/desktop/3.0/advancedusage.html because, in the documentation, I didn't see [General] section in 3.4 thinking

I too can confirm that this solved the issue (Nextcloud client 3.4.3, Manjaro).
Only setting "chunkSize" does not work, I had to set all 4 settings in the config file. I don't really understand why, but hey, it works.
Thank you ❤️

[Gwindalmir]

I can confirm this fixes it on Nextcloud 4.4.4 on Windows 10.

The only setting I needed was this:
targetChunkUploadDuration=6000

That matches what I noticed empirically, which was that the syncing failed and restarted about every 6 seconds.

Iirc from the log, the default was like 20 or 28 seconds.

[jospoortvliet]

review/feedback on the above welcome!

[jospoortvliet]

The only setting I needed was this:
targetChunkUploadDuration=6000

What surprises me is that according to the docs, this is the default...

[vithusel]

For me personally at least (I'm not the person you responded to), I wanted to use Cloudflare to hide my home IP (and thus geolocation) from people I might share files with. I ended up having to proxy my home Nextcloud through a rented server to accomplish the same due to this issue.

Understood.

My only issue with using Cloudflare tunnel to achieve this is the fact that their T&C's don't allow for anything like NextCloud and it's unfair to NextCloud to have to implement a change for something that's technically not even supported and may in future get blocked by Cloudflare.

Renting your own VPS is probably the best option for such a use case. It's what I have done as well. There's multiple methods to achieve the end goal. I use a mixture of applications (haproxy, wireguard and SSH tunneling) to provide a redundant network.

[codefaux]

it's unfair to NextCloud to have to implement a change for something that's technically not even supported and may in future get blocked by Cloudflare.

I'm not using Cloudflare. This is not a problem exclusive to Cloudflare users. It also makes Nextcloud resilient against issues with large transfers any time said large transfers fail, to my (possibly inaccurate) understanding.

Nextcloud is being asked to fix a problem in their software which causes issues with Cloudflare AND other, apparently much less common, scenarios.

So, there's that.

[sorvani]

I've been experiencing this on Fedora (37, 38, & 39 for certain) for a year or more. I just grumbled and switched to the web interface to upload the problematic files.
Adding maxChunkSize=50000000 to my client's config file resolved the issue. So if the submitted PR makes the client dynamically adjust the chunk size, it should resolve the issue.

[artforlife]

Same issue. Error syncing files 100-500 Mb in size or larger. In fact, I cannot seem to see a dependence on file size at all. At this point, the file-sharing part is completely unusable, which was a major part of using the platform for me in the first place. I ended up setting up the Seafile instead.

P.S. I have tried setting the chunk sizes as recommended in this thread, and it seemed to have fixed the behavior briefly, but then the same issue returned again.

[walec51]

setting maxChunkSize to 50 MB solved the problem for me but now I have to give instructions to every user how to modify their config file

for 2 years now a lot of people had problems with syncing large files, moving from OwnCloud was a mistake I regret every year...

even with a contributed PR this project is not capable of fixing bugs...

the default maxChunkSize used to be 100 MB which makes sens and this is how it remains on https://doc.owncloud.com/desktop/5.2/advanced_usage/configuration_file.html

on NextCloud we now have 5 GB ! which makes no sense - why have chunking at all at this point?!

I propose to change the default to 50 MB - as many PHP envs have an even lower max file upload limit of 96 MB

this would be a one line PR of which this team may by capable of reviewing and merging

[ShyViolets]

Wow, glad I found this, but disappointed this is STILL an issue. I was driving myself insane trying to find the solution for my closed-connection and didn't realize it was related to Cloudflare limits and the client itself.

[modernNeo]

my 2 cents: I think the suggestion that @metheis has posted on here is all that is really needed. I think the only issue is that I had to go through like 2-3 hours of investigation to find it, via https://help.nextcloud.com/t/if-youre-seeing-connection-closed-errors-uploading-large-files-100mb-while-using-cloudflare-we-have-a-fix/137510 which to me, implies that maybe this is a popular enough use case for the desktop to suggest @metheis's fix alongside the error?

[fracture-point]

Adding my 2c that (1) I am also affected on an out of the box install behind Cloudflare, (2) the workaround works, and (3) this was a major reason why I moved off of owncloud - the mobile app did not chunk properly which made syncing behind CF unusable.

[fracture-point]

@codefaux can you link me to the section of the ToS that you believe restricts using Nextcloud behind it? Are you talking about using a CF tunnel or just the cloud proxy? I do not use Cloudflare tunnels; I only use the traditional cloud proxy.

They used to have a restriction on non-HTML traffic, but that section was updated and removed. They discuss limitations on assets hosted on the CDN, but CF does not cache any assets stored in NC, just elements of the frontend. There is also a Reddit that discusses this with the same conclusion.

[sorvani]

I do not use Cloudflare with my Nextcloud server and I also have this problem, I stated that previously. Today, I have a new Windows laptop I jsut setup and installed the current agent and still have this problem. I guess Nextcloud wants me to switch to OneDrive, Google Drive, or even back to OwnCloud?

[devotoare]

I ran into this error with an Unraid Docker container. The issue ended up being the nginx config had client_max_body_size set to 512M in a secondary config file that got created a few updates back from the LinuxServer.io folks. Whereas when I initially created the container there was only one config file.

[Soulplayer]

This is a really annoying issue. I have to explain to all of my users how to edit the config files. I hope this gets fixed!

[artforlife]

This is a really annoying issue. I have to explain to all of my users how to edit the config files. I hope this gets fixed!

Could you explain how to edit the configs and get it working?

[Soulplayer]

This is a really annoying issue. I have to explain to all of my users how to edit the config files. I hope this gets fixed!

Could you explain how to edit the configs and get it working?

#4278 (comment) > follow this and it works.