Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

[HOWTO] How-to Boot Linux VMs using UEFI

Western Digital Drives - The Preferred Drives of FreeNAS and TrueNAS CORE

BreAKDoWN

Newbie
Joined
Jun 27, 2014
Messages
2
rather than do this fix manually, the following grub command executed as root will create the necessary dir & files under "/boot/efi/EFI" (assuming you have the ESP partition mounted at /boot/efi)

Code:
grub-install --efi-directory=/boot/efi --boot-directory=/boot --removable
Would that be better or would creating a symbolic link with ln -s be a better solution? I'm not too sure how symbolic links work.
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,219

BreAKDoWN

Newbie
Joined
Jun 27, 2014
Messages
2
I mean I don't understand symbolic links like the back of my hand. I just don't know if the syntax has to be ran again if the host file being linked is changed. I just don't know which solution is the more permanent. i.e. which ever solution is the best to set it and forget it.
 

DrumNBisco

Junior Member
Joined
Nov 5, 2013
Messages
15
Thank you!
 

windyboi

Member
Joined
Jan 7, 2016
Messages
76
Hi, is there a permanent fix available for CentOS / RHEL?

Temporary fix worked for me:
"Boot Maintenance Manager" -> "Boot From File" -> Choose "No Volume Label" -> "EFI" -> "redhat" -> "grub.efi"

But the suggest permanent fix does not work, "command not found":
grub-install --efi-directory=/boot/efi --boot-directory=/boot --removable

Thanks
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,219
Hi, is there a permanent fix available for CentOS / RHEL?

Temporary fix worked for me:
"Boot Maintenance Manager" -> "Boot From File" -> Choose "No Volume Label" -> "EFI" -> "redhat" -> "grub.efi"

But the suggest permanent fix does not work, "command not found":
grub-install --efi-directory=/boot/efi --boot-directory=/boot --removable

Thanks
grub install command is applicable to debian/ubuntu and derivatives.

I don't use CentOS, but a post-install fix for bhyve UEFI booting shouldn't be necessary as the default BOOTX64.EFI file is created by CentOS7 and should be found by the bhyve UEFI firmware.

However, a quick test with centos7 minimal 1801 iso installing to ext4 (no LMV or XFS)shows this is not working. The correct dir/files are in place on the ESP partition as per screenshots attached. But the BOOTX64.EFI is no longer recognised by bhyve UEFI firmware. Selecting the frame buffer efi file fbx64.efi does enable CentOs7 to boot. Could be a bug has crept into or has been exposed in bhyve firmware interaction with bhyve vnc console.

Manually copying fbx64.efi to BOOTX64.EFI seems to be a fix, but I wouldn't
expect this hack it to survive a kernel or grub update.



centos1.jpeg centos2.jpeg centos3.jpeg
 
Last edited:

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,219
@JOduMonT

It would be OK it worked, but reboot your linux VM and you're back to square one as the byhve UEIF firmware does not emulate storing EFI nvram so the settings are lost. So theses changes are not persistent. That's precisely why you need to "mess with gub" as you put it. If bhyve had a more straight forward non-uefi loader people could this problem.
 

JOduMonT

Neophyte
Joined
Jan 27, 2015
Messages
9
in fact rebooting the linux works
it's when you stop it and start it again

@JOduMonT

It would be OK it worked, but reboot your linux VM and you're back to square one as the byhve UEIF firmware does not emulate storing EFI nvram so the settings are lost. So theses changes are not persistent. That's precisely why you need to "mess with gub" as you put it. If bhyve had a more straight forward non-uefi loader people could this problem.
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,219
in fact rebooting the linux works
it's when you stop it and start it again
So it's not persistent and your suggestion is only useful if you never intend to stop and start your Vm. I suppose it never crossed your mind that this "idea" had already tried by others in the past, only for them to discover the obvious limitation of Bhyve's UEFI firmware.
 

JOduMonT

Neophyte
Joined
Jan 27, 2015
Messages
9
So it's not persistent and your suggestion is only useful if you never intend to stop and start your Vm. I suppose it never crossed your mind that this "idea" had already tried by others in the past, only for them to discover the obvious limitation of Bhyve's UEFI firmware.
that's why I delete it
because this tread is enough confuse without mine ;)

thanks for you solution and you reply
 
Last edited:

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,219
I agree, the thread has become untidy, I've been meaning to edit the OP for a long time.

I've no idea what active development is going on for Bhyve, not see anything about boot loaders. The byhve UEFI CSM variant might have been useful for Linux VMs, but AFAIK it doesn't work and there's no upstream fix. So to avoid UEFI would mean reverting to using either vm-bhyve of iohyve at the command line.

Adding a new boot option via UEFI manager does work in qemu/kvm with omvf as nvram variables have been emulated, and in virtualbox too. so far, this functionality is missing from the bhyve hypervisor.
 

dtom10

Member
Joined
Oct 16, 2014
Messages
81
Is there a working fix for this? Does anyone care to share a-z steps that actually work? Went to "Boot Maintenance Manager" only to not see any files, booted from live ubuntu only to not get it working. How do I get a Centos 7 VM working on FreeNAS 11.1-U4 ??? Is that hard to paste some commands???

This is getting ridiculous.
 

dtom10

Member
Joined
Oct 16, 2014
Messages
81
Ok, I have found the cause for not seeing the filesystem in the UEFI Boot Manager file selection. Unless you use VirtIO for your disk, the UEFI code can't read the partition table or that's what I think it's happening.

You don't need to recreate your VM but I have reinstalled mine so I can't tell if the fix will work for an already installed OS. The steps are to boot from the ISO image, do not let the installer use defaults for the disk layout, customize it. Delete the existing partitions from the bottom minus button, clear them all. Select from the drop-down standard partition and click the link at the top to bring back the layout. Make sure in the right hand pane that you select ext4 for your filesystems except the EFI one which is a basic vfat.

Install the OS, take out the CD-ROM device, boot. It will fail, at the shell> prompt type exit and navigate to the boot manager. You should see the local disk and navigate to EFI/BOOT/centos/grubx64.efi and press enter. Now you should see the Centos 7 boot loader. Boot the system, go to /boot/efi/EFI and copy grubx64.efi from the centos directory to BOOT. Now your VM will be able to boot everytime. If you update the system repeat the copy if the grubx64.efi differs between the /boot/efi/EFI/BOOT and /boot/efi/EFI/centos directories.

I apologize for my previous post but this wasted half of my night instead of playing Surviving Mars :D
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,950
I apologize for my previous post but this wasted half of my night instead of playing Surviving Mars :D
No worries, we all have moments of motivational frustration.
 

bitsquirrel

Neophyte
Joined
Dec 30, 2017
Messages
6
Hello I have a probably-silly question.

I installed an Ubuntu server image on FreeNAS-11.1u4 (currently) by booting off a CDROM mapped to a standard .ISO file. After installation I did a large amount of customization and then it worked fine... for a while... then it started having periodic lags of several seconds... and now (presumably after a reboot attempting to fix it), I cannot boot the system. It drops to the UEFI shell and says "Press ESC in 1s to skip startup.nsh or any other key to continue" before presenting a prompt.

I have no idea if I installed in BIOS mode or UEFI mode. I've followed advice in this thread and attempted to "exit" and then run the boot manager to boot from /UEFI/ubuntu/grubx64.efi but it simply drops me at a black screen with an X cursor and nothing else. Sometimes when I boot it dumps me directly here, and I have to do some things to get a UEFI prompt again (can't remember what). Attempts to boot from shimx64.efi and mmx64.efi (whatever those are) also fail.

I apologize for not being up-to-date on UEFI but I'm really lost. I have hand-coded x86 stage 1 boot loaders in machine language in the distant past, and I've been a very minor contributor to GRUBv1, and I've dinked around with uboot in embedded systems, but I'm confused about how to troubleshoot this because I haven't learned about UEFI apart from the GUIs that some mobos give me.

I tried to install another VM by booting a CD ISO in the same way, and this time I noticed it did say something about some other stuff being in a BIOS mode and it asked me if I wanted to install in UEFI mode anyway, and I said yes, but I ended up the same place. So now I'm coming here - help kindly appreciated.
 

dtom10

Member
Joined
Oct 16, 2014
Messages
81
First things first, this is not a proper UEFI/HW system, look at it as more of an emulator of a PC. It uses software to emulate UEFI code and there are bugs and limitations since the code is not a full hardware implementation. It tries to mimic hardware but it does so in a limited way. Now, the problem is that the UEFI code does not support all filesystems out there. It needs to find a specific file on a specific filesystem type. Do not use LVM and/or any advanced filesystems as you will end up hitting some code limitation. In my case, I've installed Centos 7 using a basic partition layout and the ext4 filesystem. As the disk type you need to select VirtIO to have the UEFI code read your disk and if the filesystem is supported you should see the files in the boot menu when it fails to boot.

You also need to have a /boot/EFI partition formatted as a vfat filesystem type.

Here's my partition layout and filesystems for my Centos 7 VM:

Code:
lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda	253:0	0   30G  0 disk
├─vda1 253:1	0  200M  0 part /boot/efi
├─vda2 253:2	0	1G  0 part /boot
├─vda3 253:3	0  9.8G  0 part /
├─vda4 253:4	0	2G  0 part
└─vda5 253:5	0   17G  0 part /data

vda4 is the swap partition

parted -l
Model: Virtio Block Device (virtblk)
Disk /dev/vda: 32.2GB
Sector size (logical/physical): 4096B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End	 Size	File system  Name				  Flags
 1	  1049kB  211MB   210MB				EFI System Partition  boot
 2	  211MB   1285MB  1074MB
 3	  1285MB  11.8GB  10.5GB
 4	  11.8GB  13.9GB  2147MB
 5	  13.9GB  32.2GB  18.3GB			   /data

cat /etc/fstab

UUID=4f48861e-de11-4992-b117-311b6365fcec /					   ext4	defaults		1 1
UUID=85126822-84c7-4c15-9275-e71301fa2c20 /boot				   ext4	defaults		1 2
UUID=6F76-424F		  /boot/efi			   vfat	umask=0077,shortname=winnt 0 0
UUID=d64078a1-eaba-4e65-90ff-274aadcea8d3 swap					swap	defaults		0 0
UUID=0ba3c837-0322-480f-a5fe-6472af269566 /data				   ext4	defaults		0 0



To have my Centos 7 boot everytime I had to copy the file as described in my previous post above. HTH.
 

bitsquirrel

Neophyte
Joined
Dec 30, 2017
Messages
6
Ah - thanks, that is helpful. I wonder if my encrypted LVM was too confusing and it could not find /boot. But exiting EFI shell and going to the Boot Manager -> Boot From File allows me to browse the EFI directories, I just can't boot from them.

Anyway, question - do you see anything particularly wrong with this?
upload_2018-4-17_23-24-55.png

fdisk on same disk:
upload_2018-4-17_23-24-31.png


I booted off CD-ROM and followed this advice:
https://wiki.debian.org/GrubEFIReinstall
But still no luck. Black screen on boot.
 

dtom10

Member
Joined
Oct 16, 2014
Messages
81
/dev/sda tells me you didn't use virtio as the disk type. Mine shows up as /dev/vda when the disk is virtio.
 

Patrick M. Hausen

Dedicated Sage
Joined
Nov 25, 2013
Messages
2,752
I'm repeating myself but I still recommend installing rEFInd - e.g. for Ubuntu:
Code:
# install rEFInd package
sudo apt-add-repository ppa:rodsmith/refind
sudo apt-get update
sudo apt-get install refind

# move rEFInd to UEFI default boot location
sudo mvrefind /boot/efi/EFI/refind /boot/efi/EFI/BOOT

Adapt appropriately for different distributions. Full installation documentation here.

HTH,
Patrick
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,219
I'm repeating myself but I still recommend installing rEFInd - e.g. for Ubuntu:
Code:
# install rEFInd package
sudo apt-add-repository ppa:rodsmith/refind
sudo apt-get update
sudo apt-get install refind

# move rEFInd to UEFI default boot location
sudo mvrefind /boot/efi/EFI/refind /boot/efi/EFI/BOOT

Adapt appropriately for different distributions. Full installation documentation here.

HTH,
Patrick
I've used the rEFind iso when necessary, but never thought about installing a ppa package and I'm not sure I'd want to. But, then I'm not an active Ubuntu user.

The changes between versions, 16.04 LTS , 17.10 and 18.04 LTS and whether you use the server install or not and choose the default disk layout with LVM will determine if you need a UEFI "fix" at all.

For example, a server install using 17.10 and 18.04 LTS with the default disk layout just works as the correct entries are created in /boot/efi/EFI/BOOT at install time.
 
Top