@scrypted/arlo fails to load because of permissions

[bemky]

@bjia56 This is really an issue with arlo plugin. If there's a better place to log issues with the plugin, please let me know.

I'm running scrypted on a new install of Raspberry Pi 4 running Raspberry Pi OS.

Here's error:

ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: 'dependency_links.txt'
Consider using the `--user` option or check the permissions.

Here's full console log:

linux arm #1642 SMP PREEMPT Mon Apr  3 17:24:16 BST 2023
server version: 0.55.0
plugin version: @scrypted/arlo 0.8.27
########################
9/23/2023, 11:37:23 PM
########################
=============================================
Python machine vs architecture mismatch detected. Plugin installation may fail.
This issue occurs if a 32bit system was upgraded to a 64bit kernel.
Reverting to the 32bit kernel (or reflashing as native 64 bit is recommended.
https://github.com/koush/scrypted/issues/678
=============================================
python version: python3.9
python prefix: /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64
Removing old dependencies: /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64
requirements.txt (outdated)
paho-mqtt==1.6.1
aiohttp==3.8.4
requests==2.28.2
cachetools==5.3.0
scrypted-arlo-go==0.5.2
cloudscraper==1.2.71
curl-cffi==0.5.7
async-timeout==4.0.2
beautifulsoup4==4.12.2
--extra-index-url=https://bjia56.github.io/armv7l-wheels/
--extra-index-url=https://bjia56.github.io/scrypted-arlo-go/
--prefer-binary
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple, https://bjia56.github.io/armv7l-wheels/, https://bjia56.github.io/scrypted-arlo-go/
Collecting paho-mqtt==1.6.1 (from -r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 1))
  Using cached https://www.piwheels.org/simple/paho-mqtt/paho_mqtt-1.6.1-py3-none-any.whl (75 kB)
Collecting aiohttp==3.8.4 (from -r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 2))
  Downloading https://github.com/bjia56/armv7l-wheels/releases/download/aiohttp-3.8.4-cpython3.9/aiohttp-3.8.4-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (822 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 822.8/822.8 kB 6.2 MB/s eta 0:00:00
Collecting requests==2.28.2 (from -r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 3))
  Using cached https://www.piwheels.org/simple/requests/requests-2.28.2-py3-none-any.whl (62 kB)
Collecting cachetools==5.3.0 (from -r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 4))
  Using cached https://www.piwheels.org/simple/cachetools/cachetools-5.3.0-py3-none-any.whl (9.3 kB)
Collecting scrypted-arlo-go==0.5.2 (from -r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 5))
  Downloading https://github.com/bjia56/scrypted-arlo-go/releases/download/v0.5.2/scrypted_arlo_go-0.5.2-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (4.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 7.8 MB/s eta 0:00:00
Collecting cloudscraper==1.2.71 (from -r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 6))
  Using cached https://www.piwheels.org/simple/cloudscraper/cloudscraper-1.2.71-py3-none-any.whl (99 kB)
Collecting curl-cffi==0.5.7 (from -r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 7))
  Downloading https://github.com/bjia56/armv7l-wheels/releases/download/curl_cffi-0.5.7-cpython3.9/curl_cffi-0.5.7-cp37-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (5.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.9/5.9 MB 7.5 MB/s eta 0:00:00
Collecting async-timeout==4.0.2 (from -r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 8))
  Using cached https://www.piwheels.org/simple/async-timeout/async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting beautifulsoup4==4.12.2 (from -r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 9))
  Using cached https://www.piwheels.org/simple/beautifulsoup4/beautifulsoup4-4.12.2-py3-none-any.whl (142 kB)
Requirement already satisfied: attrs>=17.3.0 in /usr/lib/python3/dist-packages (from aiohttp==3.8.4->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 2)) (20.3.0)
Collecting charset-normalizer<4.0,>=2.0 (from aiohttp==3.8.4->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 2))
  Using cached https://www.piwheels.org/simple/charset-normalizer/charset_normalizer-3.2.0-py3-none-any.whl (46 kB)
Collecting multidict<7.0,>=4.5 (from aiohttp==3.8.4->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 2))
  Downloading https://github.com/bjia56/armv7l-wheels/releases/download/multidict-6.0.4-cpython3.9/multidict-6.0.4-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.1/118.1 kB 5.0 MB/s eta 0:00:00
Collecting yarl<2.0,>=1.0 (from aiohttp==3.8.4->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 2))
  Downloading https://github.com/bjia56/armv7l-wheels/releases/download/yarl-1.9.2-cpython3.9/yarl-1.9.2-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (167 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 167.4/167.4 kB 5.3 MB/s eta 0:00:00
Collecting frozenlist>=1.1.1 (from aiohttp==3.8.4->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 2))
  Downloading https://github.com/bjia56/armv7l-wheels/releases/download/frozenlist-1.4.0-cpython3.9/frozenlist-1.4.0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (160 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 160.1/160.1 kB 5.8 MB/s eta 0:00:00
Collecting aiosignal>=1.1.2 (from aiohttp==3.8.4->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 2))
  Using cached https://www.piwheels.org/simple/aiosignal/aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests==2.28.2->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 3)) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests==2.28.2->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 3)) (1.26.5)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests==2.28.2->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 3)) (2020.6.20)
Collecting requests-toolbelt>=0.9.1 (from cloudscraper==1.2.71->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 6))
  Using cached https://www.piwheels.org/simple/requests-toolbelt/requests_toolbelt-1.0.0-py2.py3-none-any.whl (51 kB)
Requirement already satisfied: pyparsing>=2.4.7 in /usr/lib/python3/dist-packages (from cloudscraper==1.2.71->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 6)) (2.4.7)
Requirement already satisfied: cffi>=1.12.0 in /usr/local/lib/python3.9/dist-packages (from curl-cffi==0.5.7->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 7)) (1.15.1)
Requirement already satisfied: soupsieve>1.2 in /usr/lib/python3/dist-packages (from beautifulsoup4==4.12.2->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 9)) (2.2.1)
Requirement already satisfied: pycparser in /usr/local/lib/python3.9/dist-packages (from cffi>=1.12.0->curl-cffi==0.5.7->-r /home/admin/.scrypted/volume/plugins/@scrypted/arlo/python3.9-Linux-aarch64/requirements.txt (line 7)) (2.21)
Installing collected packages: scrypted-arlo-go, paho-mqtt, multidict, frozenlist, charset-normalizer, cachetools, beautifulsoup4, async-timeout, yarl, requests, curl-cffi, aiosignal, requests-toolbelt, aiohttp, cloudscraper
  Attempting uninstall: beautifulsoup4
    Found existing installation: beautifulsoup4 4.9.3
    Uninstalling beautifulsoup4-4.9.3:
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: 'dependency_links.txt'
Consider using the `--user` option or check the permissions.

pip install result 1
plugin start/fork failed
Traceback (most recent call last):
  File "/home/admin/.scrypted/node_modules/@scrypted/server/python/plugin_remote.py", line 380, in loadZip
    return await self.loadZipWrapped(packageJson, zipData, options)
  File "/home/admin/.scrypted/node_modules/@scrypted/server/python/plugin_remote.py", line 634, in loadZipWrapped
    raise Exception('non-zero result from pip %s' % result)
Exception: non-zero result from pip 1

[bjia56]

This error is caused by the plugin installation process attempting to upgrade some system package, which fails (as expected) since scrypted is not running as root. This installation is currently controlled by scrypted server, so there's not much that I can do from the plugin side other than make changes to the code to not use the problematic dependencies. I can look into removing the dependencies, but no guarantees that it can be done without significant effort since one of them (requests) is used frequently.

I highly recommend moving to a docker install when running scrypted on Linux, since that avoids any dependency issues with what's on the host. This error should go away when running on docker.

As an unrelated observation, your raspberry pi is running in mixed 32/64bit mode. Please see the linked github issue in the console output and set your kernel correctly, or install full 64 bit raspbian.

[bemky]

I can work toward doing a docker installation. For now, can I install packages globally via sudo?

[bjia56]

You can try sudo pip install requests==2.28.2 beautifulsoup4==4.12.2

[bemky]

That did the trick! Thank you @bjia56