ESXi Nested

[jlw52761]

Has anyone been able to get this work under the QEMU engine in TrueNAS Scale? I keep getting PSOD when booting the installer.

 

[jgreco]

Under QEMU? You're trying to run ESXi-Arm Fling or something? Or did you mean KVM?

I don't know that anyone's actually done this, but in theory it should be possible.

 

[jlw52761]

Not the ARM fling, no, standard x86. KVM uses QEMU under the hood, which really both are just libvirt under the hood.

 

[jgreco]

Conventionally, the term "QEMU" refers to the use of the emulation engine within QEMU, which is effectively a type 2 hypervisor running in userspace. The term "KVM" typically refers to the native "type 1.x-ish" kernel module that runs native code on the host CPU. They do interoperate with each other.

However, TrueNAS advertises that it offers KVM VM support, so, just for the sake of clarity, it seems best to refer to i386/amd64 VM support as "KVM". The complex interrelationships that have developed between components over time only serve to be confusing.

 

[jlw52761]

Either way, looking to see if anyone has successfully gotten a VM that has ESXi running on TrueNAS Scale.

 

[Patrick M. Hausen]

Only the other way round, sorry. But I can confirm that this runs flawlessly with 3 NVME SSDs and 2 network interfaces passed through.

 

[jlw52761]

That's unfortunate that this isn't something that is easily done. I'm sure with enough tinkering it could be figured out, but the effort vs reward just isn't there for me. I will tinker though at some point to see if this can be done, just out of my own morbid curiosity, and if I get something substantive I will return to this post and update.

 

[jgreco]

It's going to be some stupid trite issue preventing it from working. People have used KVM for ESXi before. It's known to work.

 

[soleous]

I read that you can nest ESXi on KVM, so in theory it should work on Scale, like jgreco said. However, I've just been playing with KVM on another platform using virtio and I needed to slip the drivers into Windows. So I guess depending what Scale does where virtio is concerned, storage and network drivers should not be virtio, as ESXi will need to support it, for example you'll need E1000 network cards.

 

[jlw52761]

Yeah, I figured the paravirtualized items (virtio) needed drivers, so definitely sticking with e1000 for the NIC and AHCI for the storage. I'm pretty sure it's the CPU model, and going through the different QEMU options (yes, these are QEMU CPU models) and I either get the stupid unsupported CPU message, the PSOD, or it hangs on booting the installer. I think I might switch to the kvm64 model and pass in the allow unsupported option to see if we can get it to boot.
Basically, what I'm looking to do is have a temporary swing host that I can vMotion everything to so I can patch my main box, then move them back. Once I get everything for my second host, then I won't need this. Otherwise, my old host, which is a barebones build, will still be used. no seriously, when I say barebones:

At least the new ESXi host is "proper"

He's the second from the top. Second form the bottom is the TrueNAS Scale box in all his glory, and yes, that is the basement toilet next to the rack. Sometimes, the basement bathroom, with it's exhaust fan, is the best home server room...