[Bug]: Web File Upload buggy/not resilient on bigger files (Firefox)

[domdorn]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

The Web File Uploader is buggy/not resilient enough when uploading larger files (e.g. a 800mb file). Random chunks might fail to upload on the browser side and are not retried.

Also there seems to be a timing issue which deletes upload directories before the file upload is actually complete. The Move command also seems to be missing.

Here the first chunk fails, but the file upload continues. The Chunk is never retried:

Here we can then see a timing issue where the web upload client deletes the upload directory before the file upload is finished:

Another try on uploading the same file, here a random later chunk failed, no retry is done. The Move command is not sent but the upload directory is deleted in the end. No error message was displayed in the browser

These are the log messages in the web developer console. Apparently the code sees that a chunk failed to upload, but does not retry to upload it.

The NS ERROR NET INTERRUPT seems to be a bug in firefox related to http2/http3 ( https://stackoverflow.com/a/76873402/979507 ) . I will disable http2/http3 in the meantime. While this issue might be related to the bug in firefox, it highlights the underlying issue that the web uploader is not retrying to upload a chunk if it failed for some reason.

Steps to reproduce

0. either front nextcloud with a http/2 proxy (e.g. nginx) or enable http/2 in apache
1. Create a folder, e.g. "Test"
2. Drag a larger file (e.g. 800mb) from Finder (I'm on Mac) to the browser (Firefox 123.0.1 (64-Bit))
3. A random chunk might not get uploaded, upload finishes without a error message

Expected behavior

1. File upload of large files works.
2. If a chunk is not uploaded, it gets retried at least once.
3. A success or error message is displayed when the file upload finishes
4. if you decide that retry is not to be implemented, the whole upload should immediately stop if a chunk fails, as there is no point in uploading more chunks.

Installation method

Community Docker image

Nextcloud Server version

28.0.3

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.3

Web server

Apache (supported)

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "overwriteprotocol": "https",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "28.0.3.2",
        "overwrite.cli.url": "https:\/\/***REMOVED SENSITIVE VALUE***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "logtimezone": "UTC",
        "loglevel": 1,
        "default_phone_region": "DE",
        "forcessl": true,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "filelocking.enabled": true,
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "timeout": 0,
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "mail_sendmailmode": "smtp",
        "mail_smtpauth": 1,
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "theme": "",
        "maintenance": false,
        "mail_smtpsecure": "ssl",
        "skeletondirectory": "\/var\/www\/skeleton",
        "language": "de",
        "default_language": "de",
        "default_locale": "de_DE",
        "force_language": "de"
    }
}

List of activated Apps

Enabled:
  - activity: 2.20.0
  - announcementcenter: 6.8.0
  - calendar: 4.6.6
  - cloud_federation_api: 1.11.0
  - comments: 1.18.0
  - contacts: 5.5.3
  - contactsinteraction: 1.9.0
  - dashboard: 7.8.0
  - dav: 1.29.1
  - deck: 1.12.2
  - event_update_notification: 2.3.0
  - federatedfilesharing: 1.18.0
  - files: 2.0.0
  - files_pdfviewer: 2.9.0
  - files_reminders: 1.1.0
  - files_sharing: 1.20.0
  - files_trashbin: 1.18.0
  - files_versions: 1.21.0
  - forms: 4.1.1
  - group_everyone: 0.1.14
  - impersonate: 1.15.0
  - logreader: 2.13.0
  - lookup_server_connector: 1.16.0
  - notifications: 2.16.0
  - oauth2: 1.16.3
  - password_policy: 1.18.0
  - photos: 2.4.0
  - privacy: 1.12.0
  - provisioning_api: 1.18.0
  - related_resources: 1.3.0
  - serverinfo: 1.18.0
  - settings: 1.10.1
  - sharebymail: 1.18.0
  - systemtags: 1.18.0
  - tasks: 0.15.0
  - text: 3.9.1
  - theming: 2.3.0
  - twofactor_backupcodes: 1.17.0
  - user_status: 1.8.1
  - viewer: 2.2.0
  - welcome: 1.0.10
  - workflowengine: 2.10.0
Disabled:
  - admin_audit: 1.18.0
  - bruteforcesettings: 2.8.0
  - circles: 28.0.0-dev (installed 28.0.0-dev)
  - encryption: 2.16.0
  - federation: 1.18.0 (installed 1.18.0)
  - files_external: 1.20.0
  - firstrunwizard: 2.17.0 (installed 2.17.0)
  - nextcloud_announcements: 1.17.0 (installed 1.17.0)
  - polls: 6.1.6 (installed 6.1.6)
  - recommendations: 2.0.0 (installed 2.0.0)
  - support: 1.11.0 (installed 1.11.0)
  - survey_client: 1.16.0 (installed 1.16.0)
  - suspicious_login: 6.0.0
  - twofactor_totp: 10.0.0-beta.2
  - updatenotification: 1.18.0 (installed 1.18.0)
  - user_ldap: 1.19.0
  - weather_status: 1.8.0 (installed 1.8.0)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

nextcloud.log is 20mb big, too big for this issue. is there another way to share?

Additional info

I'm fronting the nextcloud installation with nginx on docker:

version: "2"

services:
  nginx:
    image: nginx
    container_name: nginx
    restart: always
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/data/nginx/conf.d:/etc/nginx/conf.d"
      - "./configs/cloud.redacted.de:/etc/nginx/vhost.d/cloud.redacted.de"
      - "/data/nginx/vhost.d:/etc/nginx/vhost.d"
      - "/data/nginx/html:/usr/share/nginx/html"
      - "/data/nginx/certs:/etc/nginx/certs:ro"
      - "./htpasswd:/etc/nginx/htpasswd"
    networks:
      - nginx-front
  nginx-gen:
    image: jwilder/docker-gen
    container_name: nginx-gen
    mem_limit: 512m
    restart: always
    volumes:
      - "/var/run/docker.sock:/tmp/docker.sock:ro"
      - ./nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl:ro
      - "/data/nginx/conf.d:/etc/nginx/conf.d"
      - "/data/nginx/vhost.d:/etc/nginx/vhost.d"
      - "/data/nginx/html:/usr/share/nginx/html"
      - "/data/nginx/certs:/etc/nginx/certs:ro"
    volumes_from:
      - nginx
    entrypoint: /usr/local/bin/docker-gen -notify-sighup nginx -watch -wait 5s:30s /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf
  letsencrypt-nginx-proxy-companion:
    image: jrcs/letsencrypt-nginx-proxy-companion
    container_name: letsencrypt-nginx-proxy-companion
    restart: always
    mem_limit: 192m
    volumes_from:
      - nginx
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "/data/nginx/certs:/etc/nginx/certs:rw"
      - "/data/nginx/html:/usr/share/nginx/html"
    environment:
      - NGINX_DOCKER_GEN_CONTAINER=nginx-gen

with this custom config for the hostname in question:

## Start of configuration add by letsencrypt container
location ^~ /.well-known/acme-challenge/ {
   auth_basic off;
   auth_request off;
   allow all;
   root /usr/share/nginx/html;
   try_files $uri =404;
   break;
}
## End of configuration add by letsencrypt container
client_max_body_size 4g;

[susnux]

Probably duplicate of nextcloud-libraries/nextcloud-upload#5

[tstreibl]

In my case the probles came from traefik in the 2.11.1 version (so it's not related to this issue because here it seems ngnix is used - sorry for interfering). I downgraded traefik to 2.10.7 and now everything works fine. Before I couldn't upload files greater approx. 50MB. Only the Nextcloud PC Client could do because it failed, and recovered in several iterations until the upload was done... (no retry for failed chuncs from browser upload). Uploaded 8GB this way. Traefik seems to play around with this (not helpful) fix: traefik/traefik#10599. Welcome in configuration hell: https://github.com/traefik/traefik/wiki/respondingTimeouts-for-applications. All this goes beyond by skills. I was not able to configure the latest traefik version to work correctly wih nextcloud. Never had a problem the last 5 years with traefik...

[skjnldsv]

Probably duplicate of nextcloud-libraries/nextcloud-upload#5

Yes, I agree, let's close this one then :)

[bugsyb]

Could this be the same I did run into?
Might be related to maybe readTimeout on Traefik, though it would be more likely in case of migration to v3. I've run into this and setting appropriately (or disabling) readTimeout sorted issue.
See this #37695 (comment) for example entry for Traefik to fix it, if this is indeed timeout issue - check your logs.