Exceptions when enabling portainer

[ktsakalozos]

Summary

When enabling the portainer addon in a our tests (https://github.com/canonical/microk8s-community-addons/actions/runs/5820285792/job/15881038615) the following error appears:

2023-08-14T16:22:29.8461520Z Infer repository community for addon portainer
2023-08-14T16:22:40.3430883Z Traceback (most recent call last):
2023-08-14T16:22:40.3431757Z   File "/var/snap/microk8s/common/addons/community/addons/portainer/enable", line 114, in <module>
2023-08-14T16:22:40.3432938Z     main()
2023-08-14T16:22:40.3433756Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
2023-08-14T16:22:40.3434379Z     return self.main(*args, **kwargs)
2023-08-14T16:22:40.3435016Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 717, in main
2023-08-14T16:22:40.3435503Z     rv = self.invoke(ctx)
2023-08-14T16:22:40.3436245Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
2023-08-14T16:22:40.3436824Z     return ctx.invoke(self.callback, **ctx.params)
2023-08-14T16:22:40.3438162Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
2023-08-14T16:22:40.3438765Z     return callback(*args, **kwargs)
2023-08-14T16:22:40.3439256Z   File "/var/snap/microk8s/common/addons/community/addons/portainer/enable", line 80, in main
2023-08-14T16:22:40.3440072Z     ensure_addon(metric_server_addon)
2023-08-14T16:22:40.3441007Z   File "/var/snap/microk8s/common/addons/community/addons/portainer/enable", line 35, in ensure_addon
2023-08-14T16:22:40.3441537Z     output = subprocess.check_output(
2023-08-14T16:22:40.3442066Z   File "/snap/microk8s/5764/usr/lib/python3.8/subprocess.py", line 415, in check_output
2023-08-14T16:22:40.3442630Z     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
2023-08-14T16:22:40.3443204Z   File "/snap/microk8s/5764/usr/lib/python3.8/subprocess.py", line 516, in run
2023-08-14T16:22:40.3444124Z     raise CalledProcessError(retcode, process.args,
2023-08-14T16:22:40.3445039Z subprocess.CalledProcessError: Command '['/snap/microk8s/5764/microk8s-status.wrapper', '-a', 'core/metrics-server']' returned non-zero exit status 1.
2023-08-14T16:22:44.1902749Z Infer repository community for addon portainer
2023-08-14T16:22:44.9644588Z Traceback (most recent call last):
2023-08-14T16:22:44.9645122Z   File "/var/snap/microk8s/common/addons/community/addons/portainer/enable", line 114, in <module>
2023-08-14T16:22:44.9645467Z     main()
2023-08-14T16:22:44.9646166Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
2023-08-14T16:22:44.9647252Z     return self.main(*args, **kwargs)
2023-08-14T16:22:44.9647806Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 717, in main
2023-08-14T16:22:44.9648198Z     rv = self.invoke(ctx)
2023-08-14T16:22:44.9648728Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
2023-08-14T16:22:44.9649208Z     return ctx.invoke(self.callback, **ctx.params)
2023-08-14T16:22:44.9649759Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
2023-08-14T16:22:44.9650171Z     return callback(*args, **kwargs)
2023-08-14T16:22:44.9650602Z   File "/var/snap/microk8s/common/addons/community/addons/portainer/enable", line 68, in main
2023-08-14T16:22:44.9650980Z     ensure_addon(dns_addon)
2023-08-14T16:22:44.9651416Z   File "/var/snap/microk8s/common/addons/community/addons/portainer/enable", line 35, in ensure_addon
2023-08-14T16:22:44.9651855Z     output = subprocess.check_output(
2023-08-14T16:22:44.9652291Z   File "/snap/microk8s/5764/usr/lib/python3.8/subprocess.py", line 415, in check_output
2023-08-14T16:22:44.9652743Z     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
2023-08-14T16:22:44.9653206Z   File "/snap/microk8s/5764/usr/lib/python3.8/subprocess.py", line 516, in run
2023-08-14T16:22:44.9653649Z     raise CalledProcessError(retcode, process.args,
2023-08-14T16:22:44.9654393Z subprocess.CalledProcessError: Command '['/snap/microk8s/5764/microk8s-status.wrapper', '-a', 'core/dns']' returned non-zero exit status 1.
2023-08-14T16:22:48.7068857Z Infer repository community for addon portainer
2023-08-14T16:22:49.4905494Z Traceback (most recent call last):
2023-08-14T16:22:49.4907078Z   File "/var/snap/microk8s/common/addons/community/addons/portainer/enable", line 114, in <module>
2023-08-14T16:22:49.4907784Z     main()
2023-08-14T16:22:49.4909007Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
2023-08-14T16:22:49.4909774Z     return self.main(*args, **kwargs)
2023-08-14T16:22:49.4910681Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 717, in main
2023-08-14T16:22:49.4911758Z     rv = self.invoke(ctx)
2023-08-14T16:22:49.4913884Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
2023-08-14T16:22:49.4914563Z     return ctx.invoke(self.callback, **ctx.params)
2023-08-14T16:22:49.4915380Z   File "/snap/microk8s/5764/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
2023-08-14T16:22:49.4915976Z     return callback(*args, **kwargs)
2023-08-14T16:22:49.4917101Z   File "/var/snap/microk8s/common/addons/community/addons/portainer/enable", line 68, in main
2023-08-14T16:22:49.4917654Z     ensure_addon(dns_addon)
2023-08-14T16:22:49.4918283Z   File "/var/snap/microk8s/common/addons/community/addons/portainer/enable", line 35, in ensure_addon
2023-08-14T16:22:49.4918927Z     output = subprocess.check_output(
2023-08-14T16:22:49.4919556Z   File "/snap/microk8s/5764/usr/lib/python3.8/subprocess.py", line 415, in check_output
2023-08-14T16:22:49.4920221Z     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
2023-08-14T16:22:49.4920890Z   File "/snap/microk8s/5764/usr/lib/python3.8/subprocess.py", line 516, in run
2023-08-14T16:22:49.4922086Z     raise CalledProcessError(retcode, process.args,
2023-08-14T16:22:49.4923201Z subprocess.CalledProcessError: Command '['/snap/microk8s/5764/microk8s-status.wrapper', '-a', 'core/dns']' returned non-zero exit status 1.

This is probably because the prerequisite addons (eg dns) cause the snap services to be restarted. We probably need a silent retry when enabling the addons.

[ktsakalozos]

@testA113 have you seen this issue?