Sync conflict

[Queeq]

Steps to reproduce

Change nothing in existing auto-upload settings. Make sure new files are created in the folder configured for auto-upload.
or
Attempt manual file upload (files do not exist on the server)

I suspect this is started after the latest (3.11.0) update.

Expected behaviour

Files are uploaded

Actual behaviour

Files are not uploaded.

Notification: "File upload conflict. Pick which version to keep of <filename>"

In the app, in the "Uploads" screen, files are in the "Failed/pending restart" category with the message: "Sync conflict, please resolve manually". Pressing "Resolve conflict" does nothing. Pressing "Delete" does not delete anything, but attempts to upload are stopped.

Server configuration detail

Operating system: Linux 4.4.110 #62 SMP Fri Jan 5 20:38:04 EET 2018 x86_64

Webserver: nginx/1.10.3 (fpm-fcgi)

Database: mysql 10.0.38

PHP version:

7.0.33-0ubuntu0.16.04.12
Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, Reflection, SPL, session, standard, cgi-fcgi, mysqlnd, PDO, xml, apcu, calendar, ctype, curl, dom, mbstring, fileinfo, ftp, gd, gettext, iconv, igbinary, imagick, imap, intl, json, exif, mcrypt, mysqli, pdo_mysql, pdo_pgsql, pdo_sqlite, pgsql, Phar, posix, readline, redis, shmop, SimpleXML, sockets, sqlite3, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xmlreader, xmlwriter, xsl, zip, Zend OPcache

Nextcloud version: 15.0.14 - 15.0.14.1

Updated from an older Nextcloud/ownCloud or fresh install: Updated

Where did you install Nextcloud from: I don't remember

List of activated apps

Enabled:
 - accessibility: 1.1.0
 - activity: 2.8.2
 - cloud_federation_api: 0.1.0
 - comments: 1.5.0
 - dav: 1.8.2
 - federatedfilesharing: 1.5.0
 - federation: 1.5.0
 - files: 1.10.0
 - files_pdfviewer: 1.4.0
 - files_sharing: 1.7.0
 - files_texteditor: 2.7.0
 - files_trashbin: 1.5.0
 - files_versions: 1.8.0
 - files_videoplayer: 1.4.0
 - firstrunwizard: 2.4.0
 - gallery: 18.2.0
 - issuetemplate: 0.6.0
 - logreader: 2.0.0
 - lookup_server_connector: 1.3.0
 - nextcloud_announcements: 1.4.0
 - notifications: 2.3.0
 - oauth2: 1.3.0
 - password_policy: 1.5.0
 - provisioning_api: 1.5.0
 - serverinfo: 1.5.0
 - sharebymail: 1.5.0
 - support: 1.0.0
 - survey_client: 1.3.0
 - systemtags: 1.5.0
 - theming: 1.6.0
 - twofactor_backupcodes: 1.4.1
 - updatenotification: 1.5.0
 - workflowengine: 1.5.0
Disabled:
 - admin_audit
 - bruteforcesettings
 - encryption
 - files_external
 - user_ldap

Configuration (config/config.php)

{
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "dbtype": "mysql",
    "version": "15.0.14.1",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbtableprefix": "oc_",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "installed": true,
    "forcessl": true,
    "loglevel": 1,
    "theme": "",
    "maintenance": false,
    "memcache.local": "\\OC\\Memcache\\APCu",
    "memcache.locking": "\\OC\\Memcache\\Redis",
    "redis": {
        "host": "***REMOVED SENSITIVE VALUE***",
        "port": 6379
    },
    "trusted_domains": [
        "***REMOVED SENSITIVE VALUE***"
    ],
    "secret": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpmode": "smtp",
    "mail_from_address": "***REMOVED SENSITIVE VALUE***",
    "mail_domain": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpport": "25",
    "trashbin_retention_obligation": "auto",
    "updatechecker": false,
    "htaccess.RewriteBase": "\/",
    "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
    "updater.release.channel": "stable",
    "overwrite.cli.url": "https:\/\/***REMOVED SENSITIVE VALUE***",
    "mysql.utf8mb4": true
}

Are you using external storage, if yes which one: no

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Android version: 9
Device model: SM-G930F
Stock or customized system: LineageOS
Nextcloud app version: 3.11.0

Logs

Web server error log

2020/03/27 22:20:48 [info] 64263#64263: *1162408 client <IP> closed keepalive connection
2020/03/27 22:20:48 [info] 64263#64263: *1162406 client <IP> closed keepalive connection
2020/03/27 22:21:27 [info] 64263#64263: *1162427 client <IP> closed keepalive connection

Nextcloud log

Empty

Web server access log

<IP> - phone [27/Mar/2020:22:43:25 +0200] "HEAD /remote.php/webdav/IMG_20200327_163254.jpg HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.11.0"
<IP> - phone [27/Mar/2020:22:43:25 +0200] "HEAD /apps/files/ HTTP/1.1" 200 0 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.11.0"
<IP> - phone [27/Mar/2020:22:43:25 +0200] "HEAD /remote.php/webdav/IMG_20200327_163714.jpg HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.11.0"
<IP> - phone [27/Mar/2020:22:43:25 +0200] "HEAD /apps/files/ HTTP/1.1" 200 0 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.11.0"

[Queeq]

Following advice in #3635, pinging @tobiasKaminsky and @ArisuOngaku

[zero-24]

It happen to me as well it might be a good idea to add a switch to turn this check off per auto upload rule? Or having a global option to turn that check off.

[Queeq]

Also I tried to roll back to Nextcloud Android client 3.10.1 and it started to work, so the problem is definitely introduced in 3.11.0.

[tobiasKaminsky]

What kind of files are those?
Auto upload is expected to work fine with files that not have the same file name, e.g. IMG-2020-03-30-090910.jpg, but not with e.g. backup1.sql

[zero-24]

Yes for me it is the WhatsApp Backup. WhatsApp creates two files the backup with date and time attaced (that uploads fine) and the latest version msgstore.db.crypt12 as well as some setting files that always have the same name and should be overwritten with the latest version but now create this conflict message.

I agree that in the most cases the message is fine but therefore i suggested to make it possible to switch the check off on an per auto upload rule basis.

[tobiasKaminsky]

Thanks for info.

@ArisuOngaku seems, we really need a default policy for auto upload folders.
Can you do this? ❤️
If not, I will schedule some time for it.

[GitOkkie]

I've got this problem with the log file of an app I use. Until now, the new one just overwrites the old one. Since this week, I get this message every day after using this app. Unfortunately the message keeps popping up, even after switching on "Also upload existing files".

The file name is "aCar.log" and this is my config for that directory:

[dbrand666]

We don't need an option. We just need to fix the conflict resolution. If the file has only changed on one end, it's fine to overwrite the one that hasn't changed. The only time it should report a conflict is if the file changed on both sides since the last sync.

[ashpieboop]

@tobiasKaminsky I'll do it! :)

@dbrand666 what if the local version is not supposed to change and it changes for a user-unexpected reason? Overwriting the remote file means loss of data.
This feature is called "auto upload", not "file/folder sync", so this would be misleading.
If there is no existing issue about this, you may want to open one as this is, imho, a design suggestion.

[ashpieboop]

@tobiasKaminsky My apologies, I widely underestimated the amount of work it'll take to develop this properly; I unfortunately can't do this in the near future. Sorry for the inconvenience.

[GitOkkie]

I don't know what the status of this bug is now. What I do know is that I get 10 to 20 alerts every single day for manual conflict resolution of this logfile. This was never a problem, but it is now.

I still want to upload this directory since it also conains timestamped backup files. I don't really need any smart sync capabilitiy, though I understand the new feature. It's just that it doesn't work for me. As you can see above, I've got it set to "Also upload existing files", yet it still wants me to decide if I want to overwrite the old one. 20 times per day.

Should I file a seperate bug for this problem?

[Queeq]

I agree with @dbrand666, even if it's "Auto upload", it's reasonable to expect that if a file is modified locally, it would be automatically uploaded again. The user should be fully aware what he uploads.

Overall, although slightly offtopic, why was it decided to do it like this? I guess the majority of people would expect mobile client to behave in the same way as desktop one, and this difference is somewhat confusing.

[tuedelue]

I can confirm these issues - I get lots auf sync conflicts for my auto uploaded pictures. Probably google photos is changing timestamps or whatever.
Regardless, I would expect the client to resolve these issues automatically - since the pics on the phone have newer timestamps, it should simply upload them again/overwrite the old file.
The workaround resolving all conflicts manually helps a couple of days, yet the same pictures are producing conflicts again in future.
Edit: I will check out the new upload existing option. Does it help?!

[jo-me]

Same issue here. I rolled back to 3.10.1 and autoupload is working again...

[ashpieboop]

--> #5810 should help with (if not fix) the issue

btw related/duplicate issue #5599

[piknew]

This issue is closed, however... Sync conflict which is discussed in #5810 seems to be for existing remote file. Strange, as I am getting sync conflict when auto upload tries to push file to server when remote file does not exist yet. Conflict with non-existing file?

[piknew]

Not fixed with #5810.

[snowtr]

This is not fixed. I can confirm that it's still an issue in 3.11.1

[Queeq]

Could this issue be reopened? I also still experience problems with 3.11.1.

[zero-24]

@snowtr @Queeq did you checked the settings from the auto upload? Per default the new behavior is enabled and you have to explizit configure the 'overwrite' behavior. That works great for me since the update.

[tobiasKaminsky]

@ikke-t this should be a parallel installation, so there should be no uploads in queue yet?

@Queeq @ikke-t
Can you both test this

• take a screenshot
• let it be uploaded by auto upload
• clear all uploads in uploads view
• modify this screenshot locally
• this triggers a new upload
  -> choose any option on conflict dialog
• verify if new upload is vanished from upload queue

[Queeq]

@tobiasKaminsky Do you want us to configure separate screenshot folder for auto upload or it's fine to copy a newly made screenshot to another folder that's already configured to auto upload (Camera in my case)? Is it important for the screenshot to be created in an auto upload folder right away or it could be copied there?

[ikke-t]

My android creates a new image if I modify any image in gallery, so it won't reproduce edited image. There is some error state sometimes, which happens likely due some network problem, e.g. starting upload on 4G, but in middle switching to WLAN. Once that happens, it's common to get such forever error causing sync messages.

Somehow the android app fails to take action on what ever get's selected in the selection dialog for conflicts.

[ikke-t]

I now renamed the image at the server to get rid of this constant nagging of the nextcloud app. I'll investigate more once it happens again. It's not often when it happens, it's rather rare luckily.

[tobiasKaminsky]

@Queeq I did not try it with copy, so you can test it,but I am not sure if this is then a meaningful test.
Best is to do it exactly in the steps I wrote

[Queeq]

@tobiasKaminsky So I tried it with Screenshots folder (and the recent dev .apk). It fails at the very beginning: once I take the screenshot, Nextcloud tries to upload it and instantly fails claiming upload conflict. If I set to overwrite remote version in Auto Upload preferences for the folder, the result is the same.

[tigernero79]

in version 20.0.0.1 rc1 this bug has come back.

[tobiasKaminsky]

Is this really related to server?

[Queeq]

Most likely not. I'm still on my old 15th version.

[Queeq]

Tested it with 16338.apk. It's definitely better now and newly created files are uploaded. But I had a bunch of unsynced files due to this bug and it doesn't upload them. Manual upload results in conflict again, and it's indefinitely looping trying to upload them and failing unless I cancel it.

P.S. Tried to wipe the app settings and reconfigure, the conflicts are still there. But now it doesn't loop over them. Pressing "Resolve conflict" in Uploads screen does nothing.

[e-alfred]

Does this error cause the sync conflicts?

{"reqId":"X56-ZtKjspcMRQSgaRIspwAAAUU","level":0,"time":"2020-11-01T15:00:07+01:00","remoteAddr":"1.1.1.1","user":"user","app":"no app in context","method":"HEAD","url":"/owncloud/remote.php/webdav/SofortUpload/test/4546754456/CSVLogs/CSVLog_20201031_172528.csv","message":"Deprecated event type for {\"[object] (OCP\\SabrePluginEvent)\":{\"*statusCode\":200,\"*message\":\"\",\"*server\":{\"[object] (OCA\\DAV\\Connector\\Sabre\\Server)\":{\"tree\":\"[object] (OCA\\DAV\\Connector\\Sabre\\ObjectTree)\",\"*baseUri\":\"/owncloud/remote.php/webdav/\",\"httpResponse\":\"[object] (Sabre\\HTTP\\Response)\",\"httpRequest\":\"[object] (Sabre\\HTTP\\Request)\",\"sapi\":\"[object] (Sabre\\HTTP\\Sapi)\",\"*plugins\":{\"...\":\"Over 20 items, aborting normalization\"},\"transactionType\":\"head\",\"protectedProperties\":{\"...\":\"Over 20 items, aborting normalization\"},\"debugExceptions\":false,\"resourceTypeMapping\":[],\"enablePropfindDepthInfinity\":true,\"xml\":\"[object] (Sabre\\DAV\\Xml\\Service)\",\"*listeners\":{\"...\":\"Over 20 items, aborting normalization\"},\"*wildcardListeners\":[],\"*listenerIndex\":[],\"*logger\":null}},\"Symfony\\Contracts\\EventDispatcher\\EventpropagationStopped\":false}}: null","userAgent":"Mozilla/5.0 (Android) Nextcloud-android/3.13.1","version":"19.0.4.2"}

[Queeq]

I'm still experiencing sync conflict issues even with new files in Camera folder (same 16388 dev version). The difference now is that it doesn't attempt to upload automatically anymore (basically auto upload doesn't work with any combination of settings). If I try to upload manually it still claims there's a conflict...

[piknew]

The same, since a lot of versions I am getting sync conflict for files which does not exist on server yet. Clicking conflict record is either executing "checking version on server" (but nothing is changed, item still is "conflicted").

[soxhi8]

Getting a conflict message every 20 minutes or so, for a Signal-backup from yesterday. Tapping on the notification shows an error message about not being able to create the conflict-dialog.
#Edit: Manual resolve by going to uploads in the Nextcloud-app worked. Did have to download the server-version first, maybe that crashed the notification?

[nonyhaha]

I think i have the same issues on my wife's phone. I see there is no true workaround for these errors? :( I really hoped this will be a good replacement for backing up photos locally but...

[CamZie]

Also having the same problem with Nextcloud Android app version 3.16.1.

We are synchronising many files and choosing if we want to keep the "New file" or "Already existing file" to each file one by one is too much work, there is also no option like on the web interface to choose for all the files affected. As a workaround we have to manually go to "Uploads" and do "Clear failed uploads" and delete the notifications.. I hope this will be resolved soon.

[KamilBaczkowski]

Nextcloud 3.17.0 on Android and this still happens. I get conflicts for each and every photo that I have ever taken on my current phone, over 500 errors. Even if I resolve an error manually, it is going to happen on the next sync.
Couldn't the app/server just check the hashes of files to see if they are the same, and if yes, just accept that the file is good?

[ikke-t]

It occationally still happens to me. I've noticed it mostly appears when I come home. So jumping into home wlan. And it never accepts to keep the server version, only way to resolve it is to reload the image up to server. So to choose the phone version of the image. Only that makes the question go away.

[Queeq]

For me the problem was completely solved by moving to the latest (22) Nextcloud server version. Fresh install and moving files over (database was regenerated). I think older versions handle stuff in some specific way incompatible with the Android client, but I never figured out what exactly goes wrong.

[starox]

It occationally still happens to me. I've noticed it mostly appears when I come home. So jumping into home wlan. And it never accepts to keep the server version, only way to resolve it is to reload the image up to server. So to choose the phone version of the image. Only that makes the question go away.

Exactly same behaviour for me, it happens on Android version 3.17.1 and nextcloud 21.0.4.

[doggodanubus]

Android 11 UI 3.1 , App 3.18.1 constant and unrelenting squalling about existing files that bleeps about every second and would probably do so for over an hour. The self hosted server appears to be 23 but the about screen is broken on the first panel so maybe the 2nd panel is correct.

This is not fixed and makes this unreliable and unusable. I left it alone after killing notifications and nothing was synchronized. I need to be able to set 'skip' 'overwrite' 'save a copy'. I'd actually prefer a true sync and get have a checksum comparison done. I don't know if you have that.

[rchovan]

I can confirm it., When "Also upload existing files" is enabled, there is issue with sync conflict. Even when I try to manually solve this issue with overwrite.
Android version 3.20.3
Server version: 24.0.2

[bannisdale]

Most of the comments here seem to concern "auto upload", however the initial posting states the same error for manual uploads. That's what is happening for me.

I updated both the Nextcloud server (currently 24.0.3) and the Android app (currently 3.20.3). I also uninstalled the app and did a fresh install with a fresh login. Same issue. I tried both versions from F-Droid and Play Store.

Upload file (that doesn't exist): immediate "resolve conflict" notification. Clicking the notification: "Error creating conflict dialog". Upload is stopped. Upload tab: "resolve conflict" doesn't do anything, clicking the file starts a new upload that's immediately stopped by the conflict.

Desktop version works, downloading via Android app works, uploading via web/desktop works. Most error reports I have found concern auto upload, but as far as I can see, even all of them are now "closed" (like this one) without any proper solution? That's quite bothersome, especially since the original report is now more than two years old ...

[tbdal]

Most of the comments here seem to concern "auto upload", however the initial posting states the same error for manual uploads. That's what is happening for me.

I updated both the Nextcloud server (currently 24.0.3) and the Android app (currently 3.20.3). I also uninstalled the app and did a fresh install with a fresh login. Same issue. I tried both versions from F-Droid and Play Store.

Upload file (that doesn't exist): immediate "resolve conflict" notification. Clicking the notification: "Error creating conflict dialog". Upload is stopped. Upload tab: "resolve conflict" doesn't do anything, clicking the file starts a new upload that's immediately stopped by the conflict.

Desktop version works, downloading via Android app works, uploading via web/desktop works. Most error reports I have found concern auto upload, but as far as I can see, even all of them are now "closed" (like this one) without any proper solution? That's quite bothersome, especially since the original report is now more than two years old ...

i get the exact same behaviour with nextcloud server 24.0.2 and Android dev 20220715 & Android 3.20 & Android 3.21.0 RC1

[bannisdale]

Finally resolved my problem. For a possible solution, see #10628 (comment).