Massive slow down when NC tray icon is open

[koppd]

⚠️ Before submitting, please verify the following: ⚠️

• This is a bug, not a question or a configuration issue.
• This issue is not already reported on Github (I've searched it).
• Nextcloud Server and Desktop Client are up to date. See Server Maintenance and Release Schedule and Desktop Releases for supported versions.
• I agree to follow Nextcloud's Code of Conduct

Bug description

When the tray icon is open (I don't know how to describe it) then the network performance is slow down by 90%.
Here's an example of a 400 MB file transfer between two virtual machine with 1 GBit/s connection. => only 20 MBit/s

Without the opened tray icon the speed is MUCH faster => 400 MBit/s

Steps to reproduce

Copy some files (40 x 10 MB) and wait a second for the sync to start. (=upload to server; VFS is used)
Open the tray icon a the speed goes down to 20 MBit/s; close the tray icon and the transfer speed accelerate to 400 Mbit/s

Expected behavior

Transfer should not be affected if tray icon is opened or not

Which files are affected by this bug

10485760 - Kopie (5) - Kopie - Kopie - Kopie - Kopie - Kopie - Kopie - Kopie - Kopie.txt

Operating system

Windows

Which version of the operating system you are running.

Windows 10

Package

Appimage

Nextcloud Server version

25.0.0 beta 5

Nextcloud Desktop Client version

3.6.0

Is this bug present after an update or on a fresh install?

Updated to a major version (ex. 3.3.6 to 3.4.0)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

Are you using an external user-backend?

• Default internal user-backend
• LDAP/ Active Directory
• SSO - SAML
• Other

Nextcloud Server logs

No response

Additional info

No response

[PhilippSchlesinger]

I'd like to add my current experience with a newly added sync folder using virtual files:
When syncing 80000+ placeholders, I could see that progress on creating placeholder files is very fast in the beginning and slows down massively in the open tray over time. (sorry for being that vague, this is just an impression as I could not measure it).

Maybe this is just a problem of logging all the activity in the tray window, even while syncing initially? Or a repaint problem?
It could maybe help to lazily load activity in initial or larger syncing runs.

Solving this heavy issue could also pay off in improving the speed problems with initial sync mentioned in #4424

I can confirm that restarting the client helps for some time until performance decreases again.

[wonx]

I can also confirm this behavior with the placeholders (virtual files) in my computer with Windows 10.

[CoYoNq]

+1 same behavior noticed

[PhilippSchlesinger]

This issue should not be very hard to profile: e. g. add 100 000 small files (a maildir folder, for example) to a Nextcloud and measure the download to a Nextcloud Client. Even the upload to Nexcloud via a Client should suffer from the same problem.

[amilopowers]

Hello
I am on Version 3.6.2

on Ubuntu 22.04.01 LTS

And experience the same. Sync never stops and the app freezes.
Uses quite some CPU resources and the fan is constantly on.

[PhilippSchlesinger]

In

https://help.nextcloud.com/t/windows-11-desktop-client-syncs-slower-after-a-while-and-syncs-1200-something-files/146347/7

it is mentioned that performance decreases with time and restarting the client helps, which underpins the suggestions I made in #4918 (comment)

I can confirm that restarting the client helps for some time until performance decreases again.

[PhilippSchlesinger]

See #4424 (comment) for a good analysis of the issue.

[Phil-Barker]

For me it doesn't appear to be opening the tray but if I open the settings page it grinds to a complete halt

The tray seems to work fine when started but over a few days it becomes completely bogged down, just opening the tray takes foverer, it refreshes very slowly, network transfer has ground to a halt

It seems like others that restarting the app every day keeps it somewhat alive but there's definitely some sort of issue with logs or something that block it up over time

[amilopowers]

In my case it was beacause I set upload speed limitation to "automatic". I switched back to "unlimited" and everything works as it should.

[PhilippSchlesinger]

With latest Nextcloud Client 3.7.3 an inital sync on ~150k files took <1 hour where it was a whole night and endless errors in the past.
Maybe you guys could also check again and see if it improved with the latest version.

Even if the sync process is now way faster, I can see a huge difference in speed where the NC tray window is shown or not. In my case there are multiple accounts connected in the Nextcloud Client.
Switching to another accound in the tray window leads to faster progressing file sync in the account that currently does the inital sync.
As previously mentioned in #4918 (comment) this could be maybe due to a repaint problem.

[daramarak]

With latest Nextcloud Client 3.7.3 an inital sync on ~150k files took <1 hour where it was a whole night and endless errors in the past. Maybe you guys could also check again and see if it improved with the latest version.

Even if the sync process is now way faster, I can see a huge difference in speed where the NC tray window is shown or not. In my case there a multiple accounts connected in the Nextcloud Client. Switching to another accound in the tray window leads to faster progressing file sync in the account that currently does the inital sync. As previously mentioned in #4918 (comment) this could be maybe due to a repaint problem.

I will do a new timing test on my setup, I thought I saw a speed up after the last update, will verify it.

[koppd]

It's still an issue with 3.7.3.
I'm testing with a few big files and the sync speed is terrible slow when the tray window is open.
When it's closed, it's fine. (up to 20 times faster)

Please keep in mind, that syncing thousands of small files are a different story and worth a different bug report. (which are surely exist)
I guess it's a repaint issue.

[jonofmac]

I am seeing the same issue. Windows 10. Latest client.

I'd even go a step further and say that the application syync speed is just MUCH slower than all other methods.

Downloading and uploading the same 12 GB file to the same location through 3 different methods (Samba, Web client, and windows application) showed the windows client is 10x+ slower than any other method, even in best case for windows app.

I have a 10GbE network, and expect to run up against my hard drive throughput limit on the server, not some arbitrary 25 MB/sec limit. The app is terribly slow.

[PhilippSchlesinger]

I am seeing the same issue. Windows 10. Latest client.

@jonofmac
Did you also check if it makes a difference in the measurements if the tray window is shown or not?

[jonofmac]

I am seeing the same issue. Windows 10. Latest client.

@jonofmac Did you also check if it makes a difference in the measurements if the tray window is shown or not?

These tests were all done with the tray icon closed. When i open the tray, the speeds plummet to 5-8 MB/sec. Close tray and it jumps back to 25

[pampersrocker]

The issue has some more implications than I can read from here:
See the attached Gif:

1. A big file is placed into a sync directory (BigFile2 with ~4.5GB)
2. The Nextcloud (3.8.1) client is started
3. The client scans the drive for a few seconds and starts the upload with 450MBit (~50% of the 1GBit connection)
4. The tray icon window is opened - the upload comes to a crawl to 5-8 MBit
5. The tray icon is closed and speed goes up to ~60MBit, but does not recover the original "full speed"
6. The client is closed so the upload is cancelled
7. The file is uploaded to the server using a Web client with the full GBit network limit.

Basically opening the client in any way destroys any upload performance. Additionally it never reaches its full network potential.

[aortmannm]

I have the same issue that my nextcloud app not even responding on my windows 11 machine. I used the latest stable 3.8.2 for some days and now I downgraded to 3.7.3 and it seems to work well.

[JohanBoshoffWebAware]

This problem is affecting users on virtual files only it seems. We have the latest available version of Nextcloud 25, Nextcloud 25.0.6 on server and the clients have the latest client installed, version 3.8.2. We signed up a new company and they have about 6 users and the files shared among them totals more than 800 000 small files.

The users cannot work at all as their PC's are too slow. It is as if Nextcloud restarts the sync just as it completed the previous sync and just never stops.

This is a huge issue and we are about to loose our new customer. As this thread is almost a year old, it seems the Nextcloud devs are not looking into this or releasing any fixes at all. What is going on?

[StefanoV1989]

I have the same issue.
The reason is subfolders number.
This software provide a continuous scan of sync folders. So, if your folder is pretty big and complex, everything start to be slow.
The only way to "fix" this, is to make separated sync works for each folder, instead of syncing parent folder with everything inside.

[JohanBoshoffWebAware]

This problem is affecting users on virtual files only it seems. We have the latest available version of Nextcloud 25, Nextcloud 25.0.6 on server and the clients have the latest client installed, version 3.8.2. We signed up a new company and they have about 6 users and the files shared among them totals more than 800 000 small files.

The users cannot work at all as their PC's are too slow. It is as if Nextcloud restarts the sync just as it completed the previous sync and just never stops.

This is a huge issue and we are about to loose our new customer. As this thread is almost a year old, it seems the Nextcloud devs are not looking into this or releasing any fixes at all. What is going on?

I can confirm that using the Nextcloud client 3.7.3 works well and the issue stops. Please Nextcloud, please fix the latest client to have better performance using sync.

[aortmannm]

3.7.3 stopped working for me after some hours. I needed to disable now the virtual files.

[D1StrX]

Windows client version 3.9, slowdown issue with tray open issue is still here.

[felixwrt]

I can confirm the slowdown when the tray is open. My setup:

• Windows 11 running Nextcloud Desktop Client 3.9.0
• Server running 26.0.3
• Both server and client use SSDs and are connected through 1Gb/s Ethernet

Test case:

• Uploading a 2Gb file using the client
• With / without VFS

Observations:

• Initially, I get ~470 Mbps upload speed
• opening the tray reduces the speed to 8 Mbps
• closing it gives me 90 Mbps
• For the rest of the transfer, it stays at 8 Mbps (tray open) or 90 Mbps (tray closed)

Findings:

• Opening the tray makes the transfer massively slower
• Closing it increases the transfer speed, but doesn't get back to the initial speed
• I'm seeing this behavior for uploads and downloads
• Using VFS or not doesn't make a difference, the slowdown is present in both cases

I also recorded a quick video showing the slowdown when downloading a large file (using VFS): https://www.youtube.com/watch?v=THqeszUO9Ws

In the video, the transfer already starts "medium slow". I'm not sure yet when exactly this happens. Restarting the client and starting the download without opening the tray resulted in a higher initial transfer speed.

I guess it's a repaint issue.

That's what I was thinking too...

I'll take a look at the code and see if I can find something.

[felixwrt]

I've done some investigation and found what's causing the issue. While synchronizing, the window shows an animated blue sync icon. The animation of the icon is causing the slowdown. I've disabled the animation locally and now the synchronization runs with full speed - whether the tray is open or not.

That's the change that disables the animation:

diff --git a/src/gui/tray/syncstatussummary.cpp b/src/gui/tray/syncstatussummary.cpp
index 8f74985c8..adf8519cd 100644
--- a/src/gui/tray/syncstatussummary.cpp
+++ b/src/gui/tray/syncstatussummary.cpp
@@ -151,7 +151,7 @@ void SyncStatusSummary::setSyncStateForFolder(const Folder *folder)
         break;
     case SyncResult::SyncRunning:
     case SyncResult::NotYetStarted:
-        setSyncing(true);
+        setSyncing(false);
         setSyncStatusString(tr("Syncing"));
         setSyncStatusDetailString("");
         setSyncIcon(Theme::instance()->syncStatusRunning());

Here's a video showing the slowdown when the tray is open: https://drive.google.com/file/d/18FHSYBcek133wcvFyhY27Y7bwTN63GwM/view?usp=sharing

And here's the same upload operation again with the animation being deactivated: https://drive.google.com/file/d/1iNM3KMazQZAqZP_VrSuQ9cKzOUOyNmu9/view?usp=sharing

I'm not sure yet why the animation is causing such a big slowdown. CPU usage isn't high while this is happening. I suspect that it's a scheduling issue somehow (e.g. that scheduling the animation X times per second is in the same loop as the code driving the file transfer), but that's only a guess currently. I'll see if I can find the root cause.

Anyway, would it be an option to disable the animation for the time being? It's not as pretty, but the animation is causing slowdowns even after the tray has been closed again and I'd argue for performance rather than aesthetics.

I can do a PR deactivating the animation if that's a viable option.

[JohanBoshoffWebAware]

In my opinion the faster Nextcloud can operate the better and I am sure no one will mind the animation going away. Either the Nextcloud team must look into this issue and release a fix or the animation must be disabled completely until there is a fix

[PhilippSchlesinger]

The animation of the icon is causing the slowdown.

Great finding, kudos!

Can you see if this holds true for the animation in the settings/account window (like https://nextcloud.com/wp-content/uploads/2022/04/linux.png)?

[jonofmac]

Unfortunate to still see this issue. I'm on the latest NC client for windows with latest nextcloud 25 release. In my case, I transfer video files. So it's large files (several gigs each), but not a ton of them. I still arbitrarily see this 25 MB/sec limitation via the client when I keep the tray minimized/hidden. Very frustrating that the client is so slow. Has anyone been able to see transfer speeds anywhere near a 1 gbE limit with the app? Again, I have 10GbE for my video editing machines (specifically for fast transfers to my server) and the nextcloud desktop client never syncs faster than 25 MB/sec (~200 mbps). No heavy CPU usage, it's not a network limitation... I can open the nextcloud web client and download files at near full speed of the array (it's a ZFS raid 10-like rust array, so ~560MB/s (~5 gbps) for sequential writes. With nextcloud app I can't even come close to touching 1 gbe link speeds, let alone near the array speed...

[Ador-able]

Unfortunate to still see this issue. I'm on the latest NC client for windows with latest nextcloud 25 release. In my case, I transfer video files. So it's large files (several gigs each), but not a ton of them. I still arbitrarily see this 25 MB/sec limitation via the client when I keep the tray minimized/hidden. Very frustrating that the client is so slow. Has anyone been able to see transfer speeds anywhere near a 1 gbE limit with the app? Again, I have 10GbE for my video editing machines (specifically for fast transfers to my server) and the nextcloud desktop client never syncs faster than 25 MB/sec (~200 mbps). No heavy CPU usage, it's not a network limitation... I can open the nextcloud web client and download files at near full speed of the array (it's a ZFS raid 10-like rust array, so ~560MB/s (~5 gbps) for sequential writes. With nextcloud app I can't even come close to touching 1 gbe link speeds, let alone near the array speed...

The situation is similar. When I synchronize with the client, I can reach 300 mbps in the first half minute. Over time it will drop to around 150 mbps. No high cpu utilization. Using web uploads and downloads can run up bandwidth, only the client is so slow

[niixxlegacy]

Using client 3.11.0 on Win 11 22631.2861, when leaving the tray open while uploading a large file (12G ISO) a "504 MOVE" occurs and needs file to be entirely re-uploaded . Logged neither on nginx or nextcloud logs (server configured correctly btw).
No error with the linux client (or when leaving the tray closed).

[MindStudioOfficial]

Still an Issue for me: When Tray is closed I get around 200MBit/s Upload speed and when it is opened I get around 10 MBit/s or slower.

Also other people mentioned, that the client slows down in general after some time. I also observed this behavior. Shortly after restarting the client I get upload speeds at around 1GBit/s and after some time it slows down to the above 200MBit/s when the tray is closed.

[JohanBoshoffWebAware]

This issue has been around for at least a year. We also experience the same problem. The issue with the tray is the animation of the sync icon. When that is stopped, the tray does not limit download speed.

See this article for the fix: #4918 (comment)

[allexzander]

Feel free to test this build with a fix https://cloud.nextcloud.com/s/2X5wM5Y3rRMs5ZJ, in my tests, this specific issue is fixed with it

[PhilippSchlesinger]

@allexzander I have no possibility to test with that test build for the next days, but maybe some other contributors of this issue do.
Is there a PR that contains the modifications to fix the issue?

[roberix]

Hi.

I have tested with following observations.

1. Upload generally increased from 20 MB/s to ~ 140 MB/s
2. When opening the NC tray the speed drops from 140 MB/s to ~ 110 MB/s

Regards

Rob

[koppd]

Hi Allexzander,
I've tested tested your built and the issue is resolved.
Download and upload are now fast, regardless of whether NC tray icon is open or not. (> 300 MBit/s on local VM to local VM)
The tests were performed with 188 files of 10 MB each (total 1,8 GB).

regards
Dominik

PS: Of course, synchronization speed could be even better, but the specific problem with the open tray icon has been solved.

[PhilippSchlesinger]

I can also confirm the observations by @roberix and @koppd.
This specific issue with the tray window seems resolved.

There are other problems mentioned in this comment #4918 (comment) that are not covered by the solution.

[IzunaMoon]

I can confirm the same issue with the client. Although it took me a while to figure it out, the Nextcloud sync client was buggy since I had just installed Nextcloud. So, I thought something was wrong with my Nextcloud VM.

Indeed, there is an issue with the live version of the client. While the tray is open, the upload speed tanks, but that's not the only thing that happens. In my findings:

I have a server with a 6-core processor, 16GB of RAM, running Ubuntu 24 Noble, latest version from 08/05/2024.

I noticed that when the tray was open and the upload issue was occurring, my Nextcloud server began to spike in RAM usage significantly. It was using the entire 16GB while essentially doing nothing, just waiting for the file. I can also confirm this is not due to Redis since I installed Redis on a separate VM.

I tried the new build you made, @allexzander A. This seems to solve all those issues.

[smokingwheels]

Has anybody tried it, that is, has CPU core parking disabled?
Try uninstalling Spotify if you have it and reboot but not opening the nextcloud client GUI.

[nirurin]

Feel free to test this build with a fix https://cloud.nextcloud.com/s/2X5wM5Y3rRMs5ZJ, in my tests, this specific issue is fixed with it

Is there a timeline for when this build will be folded into the nextcloud client proper? It seems the test build link no longer works, but the issue is still present in current nextcloud desktop client

[PhilippSchlesinger]

@nirurin
The PR #6691 containing a fix went into 3.12.4 but as far as I can see did not make it into 3.13.0.
A corresponding backport PR (#6698) got merged, so it should be part of a yet to be released 3.13.1.

[PhilippSchlesinger]

@nirurin The PR #6691 containing a fix went into 3.12.4 but as far as I can see did not make it into 3.13.0. A corresponding backport PR (#6698) got merged, so it should be part of a yet to be released 3.13.1.

3.13.1 got released recently.

[felixwrt]

I just upgraded to 3.13.1 and can confirm that the fix works. When opening the main dialogue while downloading, there's no slowdown anymore.

[joshtrichards]

Fixed in #6691