Upgrading a CentOS Stream 9 guest VM triggers the qemu shim bug

[michel-slm]

BEFORE SUBMITTING YOUR ISSUE, PLEASE LOOK AT THE PINNED ISSUES AND USE THE SEARCH FUNCTION TO MAKE SURE IT IS NOT ALREADY REPORTED. ALWAYS COMMENT ON AN EXISTING ISSUE INSTEAD OF MAKING A NEW ONE.

Describe the issue
I have a CentOS Stream 9 VM that was last updated in May. If I do dnf upgrade and reboot, the system is unbootable - it crashes before reaching GRUB

The behavior matches lima-vm/lima#1645 and https://gitlab.com/qemu-project/qemu/-/issues/1990

The bizarre thing is excluding kernel packages from the update does not fix the behavior - and the shim was not updated either.

Configuration

• UTM Version: 4.5.2 (need to retry on 4.5.3)
• macOS Version: 14.5
• Mac Chip (Intel, M1, ...): M3 Pro

Crash log
N/A

Debug log
For all issues related to running a VM, including crashes, you should attach a debug log. (This is unavailable for macOS-on-macOS VMs. Attach an excerpt of your system log instead.)
To get the Debug log: open UTM, and open the settings for the VM you wish to launch. Near the top of the QEMU page is Debug Log. Turn it on and save the VM. After you experience the issue, open the VM settings again and select Export Log... and attach it here.
Will attach once I repro

Upload VM
If your issue is related to a specific VM, please upload the config.plist from your VM's .utm directory. To get this, right-click the VM in UTM and choose "Show in Finder". Then right-click the selected file in Finder and choose "Show Package Contents". The config.plist file is now visible. Right-click it and choose "Compress". Attach the resulting config.plist.zip file here.
You can upload the entire .utm if needed, but note this includes your VM's drive image and may contain personal data. Since Github has an attachment size limit, you may want to upload to another service such as Google Drive. Link it here.

[9001]

I'm also hitting this with a fedora VM, same behavior on 39 and 40, or rather I think it's the same issue --

in my case, even booting grubx64.efi directly doesn't work, as the grubx64.efi that gets built by fedora's mojo when installing updates fails with the Synchronous Exception error.

My workaround is to use a copy of grubx64.efi from the cd, placing it at /boot/efi/efi/fedora/works.efi and adding that as a boot entry in the ovmf boot manager

[maksimu]

Not sure if this is related, I just installed Rocky Linux 9.4 from DVD ISO, ran sudo yum -y updated and rebooted the vm. The VM couldn't start up after that. Keep getting error Synchronous Exception at <some address>

Fixed switching to Apple Virtualization from QEMU.

[vietvudanh]

Not sure if this is related, I just installed Rocky Linux 9.4 from DVD ISO, ran sudo yum -y updated and rebooted the vm. The VM couldn't start up after that. Keep getting error Synchronous Exception at <some address>

Fixed switching to Apple Virtualization from QEMU.

I had same problem with AlmaLinux9.4.

[plarkinjr]

Not sure if this is related, I just installed Rocky Linux 9.4 from DVD ISO, ran sudo yum -y updated and rebooted the vm. The VM couldn't start up after that. Keep getting error Synchronous Exception at <some address>

Fixed switching to Apple Virtualization from QEMU.

I observed exact same behavior a few weeks ago on Oracle Linux 9.3 (which had been updating & running fine for many months) after 'sudo dnf update'. Same with Fedora 39.

Using a Fedora 40 ISO, I was able to dnf update just without issue, but using a fresh Oracle Linux 9.4 ISO, dnf update still causes the failure to boot with "Synchronous Exception" described above.

I don't recall which virtualization I used, and have not tried switching it, yet.

So, based on this thread, what we know now is that: Alma9, Rocky9, CentOS9, OL9, and Fedora 39 see this behavior. My experiments show that Fedora 40 (and Ubuntu 22.04, 24.03, Debian 11 & 12) are unaffected.

[clairchiara]

I'm having the same issue after CentOS 9 update.
I'm not sure how to switch to Apple Virtualization for an existing VM as there's no option for it.

[plarkinjr]

I had another go at this today, and a fresh install of Oracle Linux 9 under "Apple Virtualization" was able to survive 'dnf update'. Just to compare, I did another fresh install with Qemu virtualization and 'dnf update' still caused the Synchronous Exception on reboot.

[maksimu]

Yeah some here, one of my co-workers used QEMU and things broke.
I think it is related to something that update installs, maybe it is setting up a new page size?

[clairchiara]

I'm having the same issue after CentOS 9 update. I'm not sure how to switch to Apple Virtualization for an existing VM as there's no option for it.

Update: I "migrated" to Apple Virtualization (that is to say, I converted the disk image to raw and imported it into a new Apple Virtualization VM configuration) and it boots fine with it.

[jgpawletko]

Hello @clairchiara ,

I am new to UTM and ran into this issue.

Are there any resources that you could point me to that show how to create a new VM in UTM using the raw disk image as you describe?
If not, no problem.

Thank you for your time!
Best-
Joe

[clairchiara]

@jgpawletko
This guide explains how to convert images: https://www.digitalocean.com/community/tutorials/how-to-convert-virtual-machine-image-formats
After creating a new VM with Apple Virtualization, you can go in the settings and replace the new image with the converted one