Error handling folders via Mac OS Finder after E2EE successful enablement

[ghost]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.

Steps to reproduce

1. I created a folder in the top level structure of my NC via iOS (folder Test) and I encrypted it with E2EE
2. I accessed the folder via desktop Finder (Mac OS) - all OK
3. I deleted the folder via desktop Finder - folder Test goes in the bin
4. the folder appears deleted in the Finder, however I get an error in the desktop sync app and in iOS the folder Test is still there in the top level
5. I moved back the folder Test from the bin into the Finder, desktop app syncs successfully
6. in iOS there is no way to delete the folder Test anymore: error 403
7. in iOS the folder is still accessible as well as its content

Expected behaviour

Removing the encrypted folder via the Finder in Mac OS should sync and remove the encrypted folder in iOS without errors.

Actual behaviour

I have a folder in my NC that I cannot move or delete or decrypt anymore

Server configuration

Operating system:
Mac OS X
Web server:

Database:

PHP version:

Nextcloud version: (see Nextcloud admin page)
19.0.4
Updated from an older Nextcloud/ownCloud or fresh install:
updated from 19.0.3
Where did you install Nextcloud from:

Signing status:

Signing status

Login as admin user into your Nextcloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results here.

No errors have been found.

List of activated apps:

App list

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your Nextcloud installation folder

If not, provide at least the version number of the End-to-End-Encryption app.

Enabled:

• accessibility: 1.5.0
• activity: 2.12.1
• admin_audit: 1.9.0
• bruteforcesettings: 2.0.1
• cloud_federation_api: 1.2.0
• comments: 1.9.0
• contactsinteraction: 1.0.0
• dav: 1.15.0
• encryption: 2.7.0
• end_to_end_encryption: 1.5.2
• federatedfilesharing: 1.9.0
• federation: 1.9.0
• files: 1.14.0
• files_antivirus: 3.0.0
• files_pdfviewer: 1.8.0
• files_rightclick: 0.16.0
• files_sharing: 1.11.0
• files_trashbin: 1.9.0
• files_versions: 1.12.0
• files_videoplayer: 1.8.0
• firstrunwizard: 2.8.0
• groupfolders: 7.1.1
• logreader: 2.4.0
• lookup_server_connector: 1.7.0
• nextcloud_announcements: 1.8.0
• notifications: 2.7.0
• oauth2: 1.7.0
• password_policy: 1.9.1
• photos: 1.1.0
• privacy: 1.3.0
• provisioning_api: 1.9.0
• quota_warning: 1.8.0
• ransomware_detection: 0.8.0
• ransomware_protection: 1.7.0
• recommendations: 0.7.0
• serverinfo: 1.9.0
• settings: 1.1.0
• sharebymail: 1.9.0
• support: 1.2.1
• suspicious_login: 3.2.1
• systemtags: 1.9.0
• text: 3.0.1
• theming: 1.10.0
• twofactor_admin: 3.0.0
• twofactor_backupcodes: 1.8.0
• twofactor_totp: 5.0.0
• unsplash: 1.1.7
• updatenotification: 1.9.0
• viewer: 1.3.0
• w2g2: 3.0.3
• workflowengine: 2.1.0
  Disabled:
• calendar
• contacts
• files_external
• files_fulltextsearch
• fulltextsearch
• fulltextsearch_elasticsearch
• mail
• onlyoffice
• spreed
• survey_client
• user_ldap

Nextcloud configuration:

Config report

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your Nextcloud installation folder

or 

Insert your config.php content here. 
Make sure to remove all sensitive content such as passwords. (e.g. database password, passwordsalt, secret, smtp password, …)

{
"system": {
"instanceid": "REMOVED SENSITIVE VALUE",
"passwordsalt": "REMOVED SENSITIVE VALUE",
"secret": "REMOVED SENSITIVE VALUE",
"trusted_domains": [
"REMOVED",
"REMOVED"
],
"datadirectory": "REMOVED SENSITIVE VALUE",
"dbtype": "mysql",
"version": "19.0.4.2",
"overwrite.cli.url": "REMOVED",
"dbname": "REMOVED SENSITIVE VALUE",
"dbhost": "REMOVED SENSITIVE VALUE",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "REMOVED SENSITIVE VALUE",
"dbpassword": "REMOVED SENSITIVE VALUE",
"installed": true,
"twofactor_enforced": "true",
"twofactor_enforced_groups": [],
"twofactor_enforced_excluded_groups": [],
"enable_previews": true,
"mail_smtpmode": "smtp",
"mail_sendmailmode": "smtp",
"mail_smtpauth": 1,
"mail_from_address": "REMOVED SENSITIVE VALUE",
"mail_domain": "REMOVED SENSITIVE VALUE",
"mail_smtphost": "REMOVED SENSITIVE VALUE",
"mail_smtpport": "587",
"mail_smtpname": "REMOVED SENSITIVE VALUE",
"mail_smtppassword": "REMOVED SENSITIVE VALUE",
"mail_smtpauthtype": "LOGIN",
"mail_smtpsecure": "tls",
"memcache.local": "\OC\Memcache\APCu",
"maintenance": false,
"theme": "",
"loglevel": 2,
"updater.release.channel": "stable",
"activity_expire_days": 14,
"auth.bruteforce.protection.enabled": true,
"blacklisted_files": [
".htaccess",
"Thumbs.db",
"thumbs.db"
],
"cron_log": true,
"enabledPreviewProviders": [
"OC\Preview\PNG",
"OC\Preview\JPEG",
"OC\Preview\GIF",
"OC\Preview\BMP",
"OC\Preview\XBitmap",
"OC\Preview\Movie",
"OC\Preview\PDF",
"OC\Preview\MP3",
"OC\Preview\TXT",
"OC\Preview\MarkDown"
],
"filesystem_check_changes": 0,
"filelocking.enabled": "true",
"htaccess.RewriteBase": "/",
"integrity.check.disabled": false,
"knowledgebaseenabled": false,
"logfile": "REMOVED",
"logtimezone": "Europe/Paris",
"log_rotate_size": 104857600,
"overwriteprotocol": "https",
"preview_max_x": 1024,
"preview_max_y": 768,
"preview_max_scale_factor": 1,
"quota_include_external_storage": false,
"skeletondirectory": ""
}
}

Are you using external storage, if yes which one: local/smb/sftp/...
No
Are you using encryption: yes/no
Yes
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
No

Client configuration

Browser:
Brave/Chromium
Operating system:
Mac OS X 10.15.7

Logs

Web server error log

Web server error log

Insert your webserver log here

Nextcloud log (data/nextcloud.log)

Nextcloud log

Insert your Nextcloud log here

{"reqId":"46Jpzjev55zVGEviEuOe","level":3,"time":"2020-11-10T23:12:48+01:00","remoteAddr":"REMOVED","user":"alberto","app":"no app in context","method":"DELETE","url":"/ocs/v2.php/apps/end_to_end_encryption/api/v1/lock/509?format=json&e2e-token=tbqYej72dHn7Fmfp3uMQsFV3fHbqekzcYPn2Wr99cIWffOZSg34GW23AJRLEkzHW","message":{"Exception":"OCA\EndToEndEncryption\Exceptions\MissingMetaDataException","Message":"Intermediate meta-data file missing","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/end_to_end_encryption/lib/Controller/LockingController.php","line":146,"function":"saveIntermediateFile","class":"OCA\EndToEndEncryption\MetaDataStorage","type":"->","args":["alberto",509]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":170,"function":"unlockFolder","class":"OCA\EndToEndEncryption\Controller\LockingController","type":"->","args":[509]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":100,"function":"executeController","class":"OC\AppFramework\Http\Dispatcher","type":"->","args":[{"class":"OCA\EndToEndEncryption\Controller\LockingController"},"unlockFolder"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":137,"function":"dispatch","class":"OC\AppFramework\Http\Dispatcher","type":"->","args":[{"class":"OCA\EndToEndEncryption\Controller\LockingController"},"unlockFolder"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Routing/RouteActionHandler.php","line":47,"function":"main","class":"OC\AppFramework\App","type":"::","args":["OCA\EndToEndEncryption\Controller\LockingController","unlockFolder",{"class":"OC\AppFramework\DependencyInjection\DIContainer"},{"id":"509","_route":"ocs.end_to_end_encryption.Locking.unlockFolder"}]},{"function":"__invoke","class":"OC\AppFramework\Routing\RouteActionHandler","type":"->","args":[{"id":"509","_route":"ocs.end_to_end_encryption.Locking.unlockFolder"}]},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":297,"function":"call_user_func","args":[{"class":"OC\AppFramework\Routing\RouteActionHandler"},{"id":"509","_route":"ocs.end_to_end_encryption.Locking.unlockFolder"}]},{"file":"/var/www/nextcloud/ocs/v1.php","line":88,"function":"match","class":"OC\Route\Router","type":"->","args":["/ocsapp/apps/end_to_end_encryption/api/v1/lock/509"]},{"file":"/var/www/nextcloud/ocs/v2.php","line":24,"args":["/var/www/nextcloud/ocs/v1.php"],"function":"require_once"}],"File":"/var/www/nextcloud/apps/end_to_end_encryption/lib/MetaDataStorage.php","Line":186,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (iOS) Nextcloud-iOS/3.0.12","version":"19.0.4.2"}

Browser log

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...

[tobiasKaminsky]

@marinofaggiana @er-vin

[er-vin]

Sounds like a duplicate, we got a report of the folder removal sometime failing on the desktop repository.

[marinofaggiana]

I can't make a test because the desktop app in my Macs don't works (don't synchronized correctly)

[ghost]

Just to provide an update, following various forums posts for solutions, I tried runnning occ files:scan --all; the command worked but it didn't solve the issue.
There is discussion about manually unlocking files via mysql as temp fix, but I would like to avoid touching the db without you guys first looking into the issue - as this is linked to E2EE somehow, not just locked files

[marinofaggiana]

@albazilla you can look if exists yet a lock in your mysql and remove it without problem, can be dangerous only if another user change the metadata file

[ghost]

Easier solution to my locked encrypted folders:

1. log in the admin account web interface and disable the E2EE
2. log in the user web interface and manually delete the locked folders

Still, the E2EE is not stable as simply creating, moving, deleting files from desktop clients totally breaks the implementation. Secondly, there is also the issue about copying moving multiple files at once using the desktop clients, that breaks the desktop sync... I had to stop it and force it again multiple times to ensure all the files would be sync'd up. But this is now for another bug another time I feel like trusting E2EE again. Serious QA issues here guys.

[joshtrichards]

Many changes have taken place since this report, many surrounding matters like this - both here and at the client level. Since the OP is unreachable and this hasn't seen traffic in there years I'm closing is. Anyone experiencing similar scenario today, please create a dedicated Issue with your respect environment details. Thanks!