Why can't I boot any ISOs in Bhyve, getting frustrating.

[diskdiddler]

Can't install and test DietPi because the ISO won't boot.
Can't install and mess with Debian, ISO won't boot.
Can install Ubuntu Desktop and Ubuntu Server (shrug?)

https://saimei.ftp.acc.umu.se/debian-cd/current/i386/iso-cd/debian-10.0.0-i386-netinst.iso

Debian 10 net installer.
I get this issue
https://i.imgur.com/SLOuAdd.png
"Press ESC in X seconds to skip startuup.nsh or any other key to continue."
It then refuses to boot.
If I type exit and fiddle in the menu, still won't boot.


Am I doing something wrong here?

 

[KrisBee]

Can't test myself at the moment. IIRC bhyve suports i386/amd64, but I'd try debian 10 64bit netinst.iso and give VM at least 1024MB to install it ( can reduce memory post install) as the debian installer goes into low-memory mode otherwise.

DietPi, have you used "native PC(UEFI)" or "nativePC(BIOS/CMS)"? Seem to remember seeing another post re: DietPi UEFI iso failing to boot with bhyve, possibly a plymouth thing. Pretty sure DietPI dev would never have tested these isos with bhyve.

 

[diskdiddler]

I thought I did try Debian 10 64netinst, perhaps I missed something.

DietPi won't work as it non uefi isos don't work with VNC ? So I cant see the console.


Oh and the full uefi install is an IMG file for some reason. I don't think I can boot that either

 

[KrisBee]

Had a quick look at DietPi. Installed the ova on my desktop and the install script appears to install grub, but unless the install image itself uses grub then I can't see this working in bhyve. The image file is not directly bootable in bhyve in any case, you'd have to try dd to a zvol. Alternative workaround could be to inastall in vbox under linux/windows, then convert the vdi to something that bhyve can run. The FN Webui may not allow this, iohyve might have worked. Doesn't seem worth messing with as if it's just pre-installing a lot of std debian packages.

 

[sretalla]

Are you using this version (claims to be UEFI, but has the same link as the non-UEFI download) https://dietpi.com/downloads/images/DietPi_NativePC-BIOS-x86_64-Buster.7z ?

 

[KrisBee]

Good point, UEFI image looks same as non-UEFI. A trip down memory lane re: dealing with RPi images got the DietpI image mounted using a loopback deivce and if does appear to be using GRUB. You might get the BOIS/CSM image to work in FN by messing with the VM device order while choosing UEFI-CSM as the boot method and attaching to the serial console. However, you'd need to fix the default grub on the Dietpi image as it's not setup for serial console use.

 

[sretalla]

I notice that their website has left directories browsable, so I see that there is a different file for UEFI despite their inability to link to it...

https://dietpi.com/downloads/images/DietPi_NativePC-UEFI-x86_64-Buster.7z

Perhaps try that one.

 

[diskdiddler]

Had a quick look at DietPi. Installed the ova on my desktop and the install script appears to install grub, but unless the install image itself uses grub then I can't see this working in bhyve. The image file is not directly bootable in bhyve in any case, you'd have to try dd to a zvol. Alternative workaround could be to inastall in vbox under linux/windows, then convert the vdi to something that bhyve can run. The FN Webui may not allow this, iohyve might have worked. Doesn't seem worth messing with as if it's just pre-installing a lot of std debian packages.

Sorry to be totally dumb, but how is this possible? Can or will this be fixed? I wasn't aware the shortcomings were this bad with Bhyve, I knew some people complained from time to time but that's not ideal at all.

I do not think it's possible anymore for me to install vbox, maybe but how hard would it be to do the VDI conversion method?

Will FreeNAS simply offer an import of VDI / VMDX files in time? Surely a script can be written?

 

[diskdiddler]

I believe those files contain .IMG files which are not bootable.
I'll re-confirm but I had difficulty booting them.

(I would be well impressed if anyone can get DietPi working on their system)

 

[sretalla]

In the install notes, the image on the .iso is not bootable, it needs to be transferred to USB formatted with GPT in order to boot. I haven't yet got my head around what that means in terms of creating a bootable .iso for FreeNAS to use.

 

[KrisBee]

The DietPi_NativePC-UEFI-x86_64-Buster.iso does boot directly in VirtualBox, if UEFI is enabled. The grub menu options then invoke clonezilla live (syslinux baed) to "restore" an image which is embedded in the iso itself. Bhyve doesn't even find the "boot6x.efi" file ... hence drops to EFI shell.

 

[KrisBee]

Ok, made some progress. Installed dietpi via UEFI iso in Vbox. Converted VDI to RAW file. Copied RAW file to FN and used dd to copy RAW file to a zvol. Bhyve still drops to EFI shell but you can navigate to efi files. It falls to boot. Looks like I failed to correctly turn VDI into a useable zvol. I'll leave it for today.

 

[KrisBee]

Bingo! Not sure why dd from VDI to zvol went wrong 1st time, but this dietpi on bhyve. ( EFI boot and fdisk size mismtch to fix)

 

[sretalla]

I was able to follow your lead and get the same outcome.

I was also able to get it to boot automatically by copying /boot/efi/EFI/debian/grubx64.efi to /boot/efi/EFI/BOOT/bootx64.efi (needed to mkdir /boot/efi/EFI/BOOT first)

 

[KrisBee]

It's debian based, so use grub-install --efi-directory=/boot/efi --boot-directory=/boot --removable post-install to ensure EFI booting.

 

[diskdiddler]

Oh my goodness.

So it's possible but an awful lot of work.
I read articles, indicating I may be able to download the pre-built VM, for VBox, convert the VM to raw and indeed DD it into a ZVOL.
That's all fine and good and possible but holy crap that's scary, one wrong move on the DD and I can kiss my system goodbye.

None the less, is there, or will there be a solution to this going forward that's not too difficult for the developers to put in place?

"Import VMDK, Import VBox file" or far better, a way for me to boot ISOs without so much difficulty? I'm still baffled why so many ISOs are failing to boot for me.

End of the day, I'm still very baffled why many ISOs refuse to boot for me, is this something which can be addressed in Bhyve - is it a simple shortcoming or a much bigger issue (see also, my difficulties getting a Debian 10 64 ISO to boot)

 

[diskdiddler]

Ok, made some progress. Installed dietpi via UEFI iso in Vbox. Converted VDI to RAW file. Copied RAW file to FN and used dd to copy RAW file to a zvol. Bhyve still drops to EFI shell but you can navigate to efi files. It falls to boot. Looks like I failed to correctly turn VDI into a useable zvol. I'll leave it for today.

Hey Kris,

I've decided to try again.
I've downloaded the OVA file (importable into VBOX) and imported it - it produces a clean, simple 8GB VDI file (presumably said file contains very little data)


ddidles@hplap:~/VirtualBox VMs/DietPi_VirtualBox-x86_64-Buster$ vboxmanage clonehd DietPi_VirtualBox-x86_64-Buster-disk001.vdi dietpieq1.raw --format raw
WARNING: The character device /dev/vboxdrv does not exist.
Please install the virtualbox-dkms package and the appropriate
headers, most likely linux-headers-generic.

You will not be able to start VMs until this problem is fixed.
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone medium created in format 'raw'. UUID: 17265a18-e93f-418d-97dc-010e3d07d04f





Is that error message an issue?

 

[sretalla]

Please install the virtualbox-dkms package and the appropriate
headers, most likely linux-headers-generic.

I would be doing those recommended things if you want to run VMs on that instance of virtualbox. It's not clear that it's essential for the copy operation to have been successful, so I would just try the raw file on bhyve.

 

[diskdiddler]

So let me clarify, I can't be certain, based on the error, that the actual conversion from VDI to RAW was successful, right?

I tried to DD the file on FreeNAS but I can't boot it - but I'm probably missing something.
BTW whomever ends up helping me, I'll write a quick guide on this, it really shouldn't be this difficult :(

 

[diskdiddler]

Anyone?