Proxmox - current status

[listhor]

I know it is advised to use Esxi for virtualized Truenas and currently I use it - with SATA/AHCI controller passthrough and ESxi itself in charge of PCI / nvme storage.
But I'm a bit fed up with Esxi and quite happy of Proxmox, i.e.:

• after migrating opnsense from esxi to pve in separate box - topton N100, all I/O interface errors are gone,
• windows vm is much, much faster - on less powerful hardware. And even more - in case of nested pve (pve being run in esxi), windows vm was faster in nested pve then windows vm directly in esxi,
• and in Esxi 8 - I've lost igpu passthrough.

I installed in "esxi" server, on separate nmve drive bare metal Proxmox and after importing Truenas' VM (setting passthrough and adjusting network interfaces names) all works fine.
Currently I rebooted back to Esxi and my question is: what is current status of Truenas/Proxmox compatibility? Is it only a matter of builtin VMware tools and missing qemu guest agent or something more?
BTW is there any chance to have qemu guest agent natively installed in Truenas?

 

[jgreco]

Currently I rebooted back to Esxi and my question is: what is current status of Truenas/Proxmox compatibility? Is it only a matter of builtin VMware tools and missing qemu guest agent or something more?

As explained in the virtualization guide, it's a matter of maturity -- Proxmox still lists PCIe passthru as being experimental, and it's only been stable within the last few years, and when I say stable, I don't necessarily mean reliable. ESXi has been doing PCIe passthru on everything from Sandy Bridge on forward for about a dozen years.

This is a matter of virtualization, not compatibility. Just like ESXi is the Cadillac of hypervisors, and can run almost any virtualizable workload, Proxmox is at the other end of the spectrum, it is twitchy and touchy about unusual workloads. Your apparent issues with ESXi are unusual, as a fleet of engineers have worked on high performance support for WIndows for many years under ESXi (as an example). It could be that your server isn't supported and that something is going wrong under the hood. Is your N100 listed on the HCL? It is entirely possible for ESXi to be limping along, valiantly trying to work, with some platform or device that isn't on the HCL, causing your setup to be marginal. This isn't ESXi's fault, it'd be your choice of hardware, if that's the case. Wrong driver for network or storage I/O are classic cases. When designing your ESXi system, you need to run all your hardware through the HCL and make sure you've got both the correct firmware and drivers installed as specified by the HCL.

As for Proxmox as a platform, if you can run your TrueNAS and it's stable for a month while doing burn-in testing such as memtest and the solnet-array-test script for that month, it is reasonable to believe that your platform may be stable and usable. If it can't remain stable, .... well then don't use it.

Also the qemu guest agent isn't of particular use except perhaps for shutdowns.

 

[listhor]

As explained in the virtualization guide, it's a matter of maturity -- Proxmox still lists PCIe passthru as being experimental, and it's only been stable within the last few years, and when I say stable, I don't necessarily mean reliable.

This is a matter of virtualization, not compatibility. Just like ESXi is the Cadillac of hypervisors, and can run almost any virtualizable workload, Proxmox is at the other end of the spectrum, it is twitchy and touchy about unusual workloads. Your apparent issues with ESXi are unusual, as a fleet of engineers have worked on high performance support for WIndows for many years under ESXi (as an example).


Also the qemu guest agent isn't of particular use except perhaps for shutdowns.

Thanks very much for your answer.
I've been using Esxi in my server/homelab since version 6 I think. Maybe my hardware (X11SCH-LN4F + Xeon E-2278G) is not fully compatible; and it seems like it is the case for igpu passthrough in esxi 8 - as on vmware hcl page esxi 8 is not listed for my cpu. Anyway since the beginning, windows vm's performance wasn't of the rocket speed.
So, esxi performance and capabilities of PBS started convincing me to migrate to proxmox. Seems like vmware quite quickly dumps a few years old hardware
BTW, is it better to migrate Truenas VM or install it from scratch and apply config backup?

Edit:
N100 is not even listed in vmware hcl, so I haven't tried esxi in that box. But I don't think N100 is anywhere close to E-2278G performance...

 

[jgreco]

Seems like vmware quite quickly dumps a few years old hardware

Well, that's not exactly true. Up 'til 6.7, they supported some very old hardware (Westmere/Nehalem) released in 2008. EoL for 6.7 was a year ago and EoTG is actually *today*. So that's 14 years of hardware life.

VMware started introducing some new features in 7.0 that required CPU support for certain operations; these days, they have options like encryption of VM's to help protect your assets in the cloud. So Westmere was deprecated in 7.0 and then a bunch of stuff up thru ... Broadwell? I think Skylake is the oldest currently viable CPU if you want to use current ESXi 8.0, though there may be hacks to use older CPU's. That's still pretty impressive.

Xeon E-2278G

That's Coffee Lake. That seems odd, I'd think it'd be supported.

BTW, is it better to migrate Truenas VM or install it from scratch and apply config backup?

Depends. The usual problem with TrueNAS is the potential need to reconfigure the network. If you have a strategy that allows you to avoid that, it's substantially better and less hair-ripping.

 

[listhor]

I've just tested importing Trunas esxi VM and also uploading config backup to freshly installed Truenas in Proxmox. In both cases I had to change only 2 virtual nics' settings because of different names. 2 passthrough nics didn't need to be touched at all - and thanks to them I had gui available straight away...
Like I mentioned windows before, other VMs are also more "snappier" - including Truenas interface what corresponds to: https://kb.blockbridge.com/technote/proxmox-vs-vmware-nvmetcp/
I'm not saying esxi is a bad choice but it seems like proxmox is a far more suitable for my hardware. Of course, I will be sure of that after couple of months of testing, so I will keep esxi on its drive and make backup copy of VMs I need.