Contacts initiates a new request for each contact picture - when scrolling, large number of php processes

[lyallp]

Steps to reproduce

1. Contacts, in NextCloud web ui
2. Scroll rapidly to bottom of contacts list
3. Contacts fails to load colourful circles with contact initials, or is extremely slow, or even server errors as php processes are killed off as process limits are enforced by shared linux environment.

Contacts with pictures seem to load immediately.
I have hundreds of contacts, of which, only a few actually have pictures associated with them.

Expected behavior

Given I suspect it's the colourful circles with the contact initials that is being generated server side is causing the issue.
Maybe only generate 5 (an arbitrary number) concurrently, rather than simply blasting out potentially hundreds of requests, if the end user scrolls quickly.

Actual behavior

See To Reproduce.

Server configuration detail

Operating system: Linux 2.6.32-954.3.5.lve1.4.78.el6.x86_64 #1 SMP Thu Mar 26 08:20:27 EDT 2020 x86_64

Webserver: LiteSpeed (litespeed)

Database: mysql 10.2.39

PHP version:

7.4.22
Modules loaded: Core, date, libxml, openssl, pcre, sqlite3, zlib, bz2, calendar, ctype, curl, hash, filter, ftp, gettext, gmp, SPL, iconv, pcntl, readline, Reflection, session, standard, shmop, SimpleXML, mbstring, tokenizer, xml, litespeed, bcmath, dom, fileinfo, gd, imagick, imap, intl, json, exif, mysqli, mysqlnd, PDO, pdo_mysql, pdo_sqlite, Phar, posix, soap, sockets, timezonedb, xmlreader, xmlrpc, xmlwriter, xsl, zip, Zend OPcache

Nextcloud version: 22.1.1 - 22.1.1.2

Updated from an older Nextcloud/ownCloud or fresh install: Updated from older version

Where did you install Nextcloud from: downloaded nextcloud source distribution

Signing status

Array
(
)

List of activated apps

Enabled:
 - accessibility: 1.7.0
 - activity: 2.15.0
 - admin_audit: 1.11.0
 - apporder: 0.13.0
 - bookmarks: 4.4.1
 - calendar: 2.3.3
 - cloud_federation_api: 1.4.0
 - contacts: 4.0.2
 - dashboard: 7.1.0
 - dav: 1.18.0
 - extract: 1.3.2
 - federatedfilesharing: 1.11.0
 - files: 1.16.0
 - files_markdown: 2.3.4
 - files_pdfviewer: 2.3.0
 - files_rightclick: 1.1.0
 - files_sharing: 1.13.2
 - files_texteditor: 2.14.0
 - files_videoplayer: 1.11.0
 - issuetemplate: 0.7.0
 - keeweb: 0.6.6
 - logreader: 2.7.0
 - lookup_server_connector: 1.9.0
 - notes: 4.1.1
 - notifications: 2.10.1
 - oauth2: 1.9.0
 - password_policy: 1.12.0
 - photos: 1.4.0
 - privacy: 1.6.0
 - provisioning_api: 1.11.0
 - recommendations: 1.1.0
 - serverinfo: 1.12.0
 - settings: 1.3.0
 - sharebymail: 1.11.0
 - support: 1.5.0
 - tasks: 0.14.2
 - text: 3.3.0
 - theming: 1.12.0
 - twofactor_backupcodes: 1.10.1
 - updatenotification: 1.11.0
 - viewer: 1.6.0
 - workflowengine: 2.3.1
Disabled:
 - circles
 - comments
 - contactsinteraction
 - encryption
 - federation
 - files_external
 - files_trashbin
 - files_versions
 - firstrunwizard
 - nextcloud_announcements
 - survey_client
 - systemtags
 - user_ldap
 - user_status
 - weather_status

Configuration (config/config.php)

{
    "installed": true,
    "dbtype": "mysql",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbtableprefix": "oc_",
    "forcessl": false,
    "theme": "",
    "3rdpartyroot": "",
    "3rdpartyurl": "",
    "blacklisted_files": [
        ".htaccess"
    ],
    "default_language": "en",
    "default_locale": "en_AU",
    "default_phone_region": "AU",
    "overwritehost": "",
    "overwriteprotocol": "https",
    "overwritewebroot": "",
    "overwritecondaddr": "",
    "overwrite.cli.url": "https:\/\/cloud.remotely-helpful.com\/",
    "htaccess.RewriteBase": "\/",
    "defaultapp": "dashboard",
    "knowledgebaseenabled": true,
    "knowledgebaseurl": "http:\/\/api.apps.nextcloud.com\/v1",
    "appstoreenabled": true,
    "appstoreurl": "https:\/\/apps.nextcloud.com\/api\/v1",
    "apps_paths": [
        {
            "path": "\/home\/remotely\/www\/cloud\/apps",
            "url": "\/apps",
            "writable": true
        }
    ],
    "mail_smtpdebug": false,
    "mail_smtpmode": "smtp",
    "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpport": "587",
    "mail_smtptimeout": 10,
    "mail_smtpauthtype": "LOGIN",
    "appcodechecker": true,
    "log_type": "nextcloud",
    "logfile": "\/home\/remotely\/cloud\/logs\/nextcloud.log",
    "loglevel": 0,
    "logdateformat": "F d, Y H:i:s",
    "logtimezone": "Australia\/Sydney",
    "log_rotate_size": 104857600,
    "0": true,
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "hashingCost": 10,
    "trashbin_retention_obligation": "180, auto",
    "allow_user_to_change_display_name": true,
    "xframe_restriction": true,
    "custom_csp_policy": "default-src 'self'; script-src 'self' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src *; font-src 'self' data:; media-src *",
    "customclient_desktop": "",
    "customclient_android": "",
    "customclient_ios": "",
    "remember_login_cookie_lifetime": 1296000,
    "session_keepalive": true,
    "updatechecker": true,
    "updater.server.url": "https:\/\/updates.nextcloud.com\/updater_server\/",
    "updater.release.channel": "stable",
    "has_internet_connection": true,
    "writable_appsdir": true,
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "version": "22.1.1.2",
    "maxZipInputSize": "838860800",
    "allowZipDownload": true,
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "maintenance": false,
    "trusted_domains": [
        "cloud.remotely-helpful.info",
        "cloud.remotely-helpful.com",
        "remotely-helpful.info",
        "remotely-helpful.com",
        "remotelyhelpful.info",
        "remotelyhelpful.com",
        "remotelyhelpful.info.au",
        "remotelyhelpful.com.au",
        "remotely-helpful.info.au",
        "remotely-helpful.com.au",
        "vmres08.auserver.com.au"
    ],
    "secret": "***REMOVED SENSITIVE VALUE***",
    "mail_from_address": "***REMOVED SENSITIVE VALUE***",
    "mail_domain": "***REMOVED SENSITIVE VALUE***",
    "appstore.experimental.enabled": true,
    "mail_smtpauth": 1,
    "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
    "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
    "app_install_overwrite": [
        "keeweb",
        "apporder",
        "calendar",
        "contacts",
        "external",
        "camerarawpreviews",
        "tasks",
        "bookmarks",
        "occweb",
        "extract",
        "gpxedit",
        "issuetemplate"
    ],
    "mysql.utf8mb4": false,
    "enabledPreviewProviders": [
        "OC\\Preview\\PNG",
        "OC\\Preview\\JPEG",
        "OC\\Preview\\GIF",
        "OC\\Preview\\HEIC",
        "OC\\Preview\\BMP",
        "OC\\Preview\\XBitmap",
        "OC\\Preview\\MP3",
        "OC\\Preview\\TXT",
        "OC\\Preview\\MarkDown",
        "OC\\Preview\\OpenDocument",
        "OC\\Preview\\Krita"
    ],
    "tempdirectory": "\/home\/remotely\/tmp\/NextCloud"
}

Are you using external storage, if yes which one: local

Are you using encryption:

Are you using an external user-backend, if yes which one: local

Client configuration

Browser: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36

Operating system: Gentoo Linux 64 bit

Logs

Web server error log

n/a

Nextcloud log

n/a

Browser log

n/a

[lyallp]

An addition suggestion, if a contact has no picture, potentially (maybe a setting option), assign the coloured circle to the contact, so the loading issue only occurs on first contact display.
This as well as limiting the rate of requests for contacts with no images.

[BMerz]

In addition I get a 404 error in the webserver log for each contact that has no profile picture.

[astehlik]

Can confirm this issue for Nextcloud 24.0.1 and Contacts version 4.1.1.

When opening the contacts app the requests made to load the Avatar images bring down the instance :(

[astehlik]

I managed to work around this issue by adding a rule to "My Filters" in the uBlock origin settings:

||mycloud.mydomain.tld/remote.php/dav/addressbooks/users/*/*/*.vcf?photo$image,domain=mycloud.mydomain.tld

The downside of course is that you do not have any contact images any more 😉

[knobhead]

Hi,
encountered this issue on NC 25.0.1 with contacts 5.0.1 as well.

Operating system: Linux 3.10.0-1160.31.1.el7.x86_64 #1 SMP Thu Jun 10 13:32:12 UTC 2021 x86_64

Webserver: Apache/2.4.54 (Unix) (fpm-fcgi)

Database: mysql 10.3.22

PHP version: 8.1.12

Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, json, Reflection, SPL, session, standard, cgi-fcgi, restrictdocroot, bcmath, bz2, calendar, ctype, curl, dba, dom, mbstring, fileinfo, ftp, gd, gettext, gmp, iconv, imap, intl, ldap, exif, mysqlnd, PDO, pgsql, Phar, posix, shmop, SimpleXML, soap, sockets, sodium, sqlite3, sysvmsg, sysvsem, sysvshm, tidy, tokenizer, xml, xmlwriter, xsl, mcrypt, mysqli, pdo_mysql, pdo_pgsql, pdo_sqlite, xmlreader, xmlrpc, zip, apcu, decimal, gnupg, igbinary, imagick, mailparse, msgpack, yaml, mongodb, redis, Zend OPcache

@lyallp can I help with any information needed to triage?

@astehlik thanks for your ABP workaround, saves a lot of nerves atm.

[astehlik]

There is also another fix / workaround for this issue, see: nextcloud/server#33048

When the 404 errors are handled correctly and do not cause a redirect to the dashboard the number of requests for loading the contact images should not be a problem any more.

[lyallp]

My argument is that we should never reach the stage of having to deal with 404 in the case of Contact images.
Images should be loaded in a controlled and rate limited manner.

[Uatschitchun]

Is this still being worked on? My contacts list is slow as hell ;(

[Uatschitchun]

Couldn't the cache images for contact initials simply be cached somewhere?