Skip to content
This repository has been archived by the owner on Sep 12, 2024. It is now read-only.

dsc_host keeps crashing #764

Open
johanburati opened this issue Jan 14, 2021 · 10 comments
Open

dsc_host keeps crashing #764

johanburati opened this issue Jan 14, 2021 · 10 comments

Comments

@johanburati
Copy link

dsc_host keeps crashing.

kernel: dsc_host[9960]: segfault at 28 ip 0000000000433fb4 sp 00007ffc964cb850 error 4 in dsc_host[400000+94000]
kernel: Code: 89 4c 24 30 64 48 8b 04 25 28 00 00 00 48 89 84 24 a8 00 00 00 31 c0 4d 85 e4 b0 04 0f 84 94 01 00 00 49 c7 04 24 00 00 00 00 <48> 8b 6f 28 49 89 fe 49 8b 50 18 48 8d 05 3e 19 04 00 49 89 cf 4c
@Klaas-
Copy link

Klaas- commented Jan 14, 2021

We also opened a support case with Microsoft

@johanburati
Copy link
Author

johanburati commented Jan 15, 2021

It's kind of the same thing as for RHEL 8 and other extensions: Azure/WALinuxAgent#1719

WORKAROUND

This should prevent dsc_host from crashing and filling up the fs with core files until the devs push a new release that fix the issue for good.

python is missing on those images, you can create and make a link and make it point to python3 with the following command:

sudo alternatives --set python /usr/bin/python3

Or if you are seeing issue with python3, you can install python2 and make python point to python2 :

sudo dnf install python2 -y
sudo alternatives --set python /usr/bin/python2

Tried it on a couple of VMs and did not see any crash after using either option.

@Klaas- could you try on your VMs and confirm ?

@Klaas-
Copy link

Klaas- commented Jan 15, 2021

Hi,
I would not consider this a solution but more a workaround :) Waagent/oms do not need this kind of workaround anymore. I am guessing you should either default to python3 directly or build something to recognize the python version:
https://github.com/microsoft/OMS-Agent-for-Linux/blob/73d35e11a5ec4a2541731d737816d3ad2ff6018f/tools/LogCollector/source/omslinux_agentlog.sh#L49-L68
https://github.com/microsoft/OMS-Agent-for-Linux/blob/b6f2658136f5f84974be7401d9f18945ef0bf541/installer/bundle/bundle_skel.sh#L652-L661

Red Hat does not recommend to set an unversioned python link ( see man unversioned-python ) on a recent rhel.

@johanburati
Copy link
Author

johanburati commented Jan 16, 2021

@Klass- right, I changed "solution" for "workaround", at least it will prevent the program from crashing and generating core files until the developers release a new version that fix the issue. I hope this will help other ppl having this issue.

@robfordww
Copy link

Do you know if I somehow can disable this feature on Ubuntu 20.04 running on Azure. It fills up the entire dmesg log system. I could try the python thing mentioned above, but why not remove the entire thing?

@johanburati
Copy link
Author

@robfordww You need to disconnect the VMs within your OMS workplace and all the OMS components will get uninstalled. Please cut a support ticket if you need help with that.

@sbonds
Copy link

sbonds commented Feb 26, 2021

This directly relates to MS case 121012721003494. I'll use the workaround for now, but will also be eagerly awaiting a RHEL8-compatible solution. (Honestly, though, why did RedHat need to break "#!/usr/bin/env python"? That's the method that's supposed to always work!)

@sbonds
Copy link

sbonds commented Mar 31, 2021

Note that the advice to use python3 as the alternative seems to work best on RHEL8 and is the closest to the RHEL8 general guidelines to extremely preferentially use python3. However, this advice contradicts the advice given in this Microsoft document:

https://docs.microsoft.com/en-us/azure/azure-monitor/agents/agent-linux#python-requirement

That advice is given in the context of "if you must use an older agent" and I'll suggest an improvement to make that clearer.

sbonds added a commit to sbonds/azure-docs that referenced this issue Mar 31, 2021
As part of an internal conversation around microsoft/PowerShell-DSC-for-Linux#764 it was suggested that we use python2 on RHEL8 since it was "recommended by Microsoft" in this document. This seems to be a misunderstanding of the context of this fix and goes against the strong preference for python3 in RHEL8. Help clarify that with this trivial improvement.
@Klaas-
Copy link

Klaas- commented Apr 6, 2021

@sbonds I think this is a more general recommendation by python developers -- see "man unversioned-python" on an EL8 system or https://www.python.org/dev/peps/pep-0394/

@Klaas-
Copy link

Klaas- commented May 17, 2021

This seems to be fixed by omsconfig-1.1.1-930.x86_64, you should no longer recommend the python workarounds in docs :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants