bhyve VMs unbootable after system upgrades

[kaseg]

Both the upgrade from 11.2-U1 -> U2 and U2 -> U2.1 have rendered my bhyve virtual machines unusable.

I didn't give this much thought after the first upgrade and just recreated them from scratch, but the same thing has now happened after the upgrade to U2.1.

Thoughts?

 

[dlavigne]

Define "unusable", as in, they won't start? They give specific error messages?

 

[kaseg]

They won't start. They just drop into the UEFI shell, or the ISO installer if its still in the device list.

 

[thedudeabides]

I encountered the same issue. I installed a Debian VM, and after the U2 -> U2.1 upgrade, it drops to the UEFI shell.

Inside the pool where the VM disks are stored, I see it got duplicated (wireguard-*), not sure if this is normal after an upgrade or this is how FreeNAS backs up.

 

[kaseg]

Mine aren't duplicated, but they still don't work.

Also running Debian VM's (stretch/stable) - not sure if it matters, but its a data point.

 

[thedudeabides]

I noticed this happens with Debian 9+UEFI when the VM is set to boot on start and you reboot FreeNAS. I tried with Ubuntu 18.04 and I had no such issue. Seems this has to do specifically with Debian, I haven't tried any other distro.

 

[kaseg]

That describes my situation exactly (Debian 9, UEFI, Start on Boot). It would be nice to see this fixed, but for my uses Ubuntu is fine also, so I'll look into that as a workaround.

Perhaps the update has nothing to do with this, and its actually just the restarting of the machine (which only happens during updates for me).

 

[KrisBee]

Sounds like your debian VM was never configured to boot with UEFI correctly, hence you're dropping to the UEFI shell. ( see : https://www.ixsystems.com/community/threads/howto-how-to-boot-linux-vms-using-uefi.54039/)

For a post-install fix, first exit the UEFI shell and navigate the "Boot Manager" to boot your VM. (see here and follow steps 1 to 7 only:
https://www.jongibbins.com/solving-uefi-boot-problems-on-bhyve-freenas-vm/ ) Then as root, execute this command: grub-install --efi-directory=/boot/efi --boot-directory=/boot --removable

If starting with a new VM, use the expert install and answer "YES" when asked to "Force grub-efi installation to the removable media path".

 

[thedudeabides]

Ha, that's it :) Thanks @KrisBee , guess we should have searched better. I also found a bug report https://redmine.ixsystems.com/issues/27288. I think we can consider this as solved!

 

[kaseg]

I'm sure that's it for me as well, will fix my install next week.

Would be nice if there was a warning in the UI when creating a UEFI VM (the default option) that some installations may need a non-standard option during install. Looks like this has been an ongoing confusion for nearly 2 years.