TrueNAS Scale virtualization and Debian UEFI boot loader issue

[e7balt]

I've been testing TrueNAS Scale (currently running nightly TrueNAS-SCALE-21.05-MASTER-20210504-072918) and have run into some issues with virtualization- specifically with Debian Linux (UEFI boot).

It seems that TrueNAS Scale virtualization has a similar issue as TrueNAS Core when it comes to Debian and UEFI boot (see post at: https://www.truenas.com/community/threads/bhyve-vm-uefi-boot-fails-after-each-upgrade.59825/).

Specifically, after initial install of a Debian VM (with UEFI boot) it is not able to find the boot loader (i.e. debian/grubx64.efi). This can be fixed temporarily by going into the virtual UEFI/BIOS menus of the VM and manually adding a new boot option and pointing it to grubx64.efi and then saving the changes. However, it seems rebooting and/or upgrading TrueNAS Scale will cause this saved boot setting to be lost. As a result the VM can't boot up successfully without manual intervention after a reboot of TrueNAS Scale and I've had to manually go back into the UEFI/BIOS of the VM and re-add the boot option to point it to debian/grubx64.efi.

In the TrueNAS Core post and associated bug entry https://redmine.ixsystems.com/issues/27288 a workaround is mentioned about copying debian/grubx64.efi to EFI/BOOT/bootx64.efi. I have not tried the workaround on TrueNAS scale yet to see it it works but will do so the next time I have to take my server down for maintenance or an update.

Is there anyway Scale could be updated so that it persists the UEFI/BIOS changes across TrueNAS Scale upgrades and reboots?

Doing some digging around the file system I see .fd files for each VM under /var/lib/libvirt/qemu/nvram/ (for example, /var/lib/libvirt/qemu/nvram/1_vmname_VARS.fd). This file is also referenced in the VM XML config for the vm (under /etc/libvirt/qemu/) in the <os><nvram> section. I suspect that the .fd files might be what holds the UEFI/BIOS configuration settings but I'm not 100% sure.

Thanks!

 

[sretalla]

Have you sen this? :

[HOWTO] How-to Boot Linux VMs using UEFI

There have been a number of threads were people have failed to boot a Linux VM after the installation phase using the webUI of FreeNAS 11 nightly, finding themselves stuck in the EFI shell and not knowing how to proceed nor how to avoid this happening each time their VM is started. It has been...

www.truenas.com

 

[e7balt]

Thanks- I had not seen that specific post which has a bit more detail than the other forum post I listed initially.

One particularly useful link that further helps explain the situation is the following: https://wiki.debian.org/UEFI#Force_grub-efi_installation_to_the_removable_media_path. But forcing the grub efi install to the removable media path upon initial install is still a workaround for not being able to properly persist the UEFI boot options in the virtual UEFI.

One other thing I just thought of- I suspect that any changes to the UEFI settings in the virtual UEFI (for example, Secure Boot setting?) would also get lost after TrueNAS scale reboots/upgrades.

 

[Kulmala]

copying debian/grubx64.efi to EFI/BOOT/bootx64.efi.

This worked for TrueNAS-SCALE-22.02-RC.2 running debian 11 wm. Thank you.

 

[adornala]

The above page with the fix gave me 404, so here is the solution if someone is still searching for it.

Code:

su -
cd /boot/efi/EFI
mkdir BOOT
cp debian/grubx64.efi BOOT/bootx64.efi
# Now you can reboot to check if evrything is good. Save your work and reboot
reboot

 

[kashiwagi]

I just ran into this, and tried just copying like adornala mentioned above.

Didn't work for me ...

So anyway, doing this seems to have done the trick

Code:

apt-get install --reinstall grub-efi
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=grub

Used this as a reference

[HOWTO] How-to Boot Linux VMs using UEFI

There have been a number of threads were people have failed to boot a Linux VM after the installation phase using the webUI of FreeNAS 11 nightly, finding themselves stuck in the EFI shell and not knowing how to proceed nor how to avoid this happening each time their VM is started. It has been...

www.truenas.com

 

[bitbit]

I just ran into this, and tried just copying like adornala mentioned above.

Didn't work for me ...

So anyway, doing this seems to have done the trick

Code:

apt-get install --reinstall grub-efi
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=grub

Used this as a reference

[HOWTO] How-to Boot Linux VMs using UEFI

There have been a number of threads were people have failed to boot a Linux VM after the installation phase using the webUI of FreeNAS 11 nightly, finding themselves stuck in the EFI shell and not knowing how to proceed nor how to avoid this happening each time their VM is started. It has been...

www.truenas.com

Thanks for this! Solved the issue for me as well. Cheers!

 

[-jim]

I just ran into this, and tried just copying like adornala mentioned above.

Didn't work for me ...

So anyway, doing this seems to have done the trick

Code:

apt-get install --reinstall grub-efi
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=grub

Used this as a reference

[HOWTO] How-to Boot Linux VMs using UEFI

There have been a number of threads were people have failed to boot a Linux VM after the installation phase using the webUI of FreeNAS 11 nightly, finding themselves stuck in the EFI shell and not knowing how to proceed nor how to avoid this happening each time their VM is started. It has been...

www.truenas.com

I end up at best in the Install CD or can exist to the Grub shell. How would I be alble to do the commandlines without the device being booted?

What am I missing?

 

[Marek S.]

I fixed it like this:

Go to virtualization > Your VM > Display

SHELL> type: exit
> Boot Maintenance Manager
-> Boot from file
--> Select grubx64.efi

Boot into Debian

> login as root
-> cd /boot/efi/EFI
--> mkdir /boot/efi/EFI/BOOT
---> cp /boot/efi/EFI/debian/grubx64.efi /boot/efi/EFI/BOOT/bootx64.efi

reboot