State of virtualizing FreeNAS and using bhyve

[steveroch-rs]

So here's my plan.
I want to buy a DELL R720XD with 2 Xeon E5-2650s and 128GB of RAM + a seperate LSI 2308 HBA.
Since this machine is very powerful and in some sense could be a little underwhelmed with the only workload being a NAS/Database Server, I would love to use the extra CPU power for other servers I'd like running 24/7.

I am running a 2 node cluster using Proxmox (honestly just for the ease of management, nothing HA) that is serving my main gaming rig with GPU passthrough and some gameservers and Linux VMs.
I'd like to add the DELL as a third node with FreeNAS as a VM using PCIe passthrough for the HBA and CPU set to host mode. No memory ballooning or other kinds of fancy VM magic.

My question now is how "dangerous" or unrecoomended is it still to use FreeNAS in such a scenario? I mean it would have direct access to the HBA?!
I heard that bhyve is not very stable and has its issues but most of those articles and posts are 3y old... how does it perform today?

What would you recommenc? Virtualizing FreeNAS with PCIe passthrough or using FreeNAS to virtualize using bhyve?

The valuable critical data stored (like databases) will be backed up remotely (twice) and the rest is "Linux ISOs" and music.

Thanks for your advice again, this forum has been very helpful so far on my journey toward my first FreeNAS box.

 

[sretalla]

The accepted wisdom here is in two parts:

Use a Type 1 Hypervisor to do the work (ESXi provides the most tested and compatible platform) if you want to virtualize FreeNAS (https://www.ixsystems.com/community...ide-to-not-completely-losing-your-data.12714/).

Don't use Bhyve if you want to do anything fancy (like GPU passthrough).

 

[steveroch-rs]

Use a Type 1 Hypervisor to do the work (ESXi provides the most tested and compatible platform) if you want to virtualize FreeNAS (https://www.ixsystems.com/community...ide-to-not-completely-losing-your-data.12714/).

I have read the post. He seems to agree that in case one uses virtualization it has to be with PCI-passthrough.
Since Proxmox is a Type 1 hypervisor i assume it will work okay. I will however test my hardware rigorously with FreeNAS installed bare-metal before moving towards virtualizing it.

Don't use Bhyve if you want to do anything fancy (like GPU passthrough).

I won't do that. The only thing that would run inside bhyve would have been a docker host and game servers.

How about jails inside FreeNAS-as-VM? Specifically Nextcloud and Plex. They will work as expected I guess? Just install the plugin and access it via the FreeNAS IP.

 

[sretalla]

How about jails inside FreeNAS-as-VM? Specifically Nextcloud and Plex. They will work as expected I guess? Just install the plugin and access it via the FreeNAS IP.

Yes, but you will be better to use VNET and have an IP address per jail.

 

[Inxsible]

How about jails inside FreeNAS-as-VM? Specifically Nextcloud and Plex.

Why? You already have a couple of Proxmox boxes. Just install Nextcloud on that and set up the Nextcloud data folder on the FreeNAS box in case you don't have enough hard disk space on the Proxmox boxes. That's what I do.

My Proxmox box is a 1U server with just 4x500GB drives, so I had to move the NC data folder over to my bare-metal FreeNAS box.

Plex -- I can understand being on the same box as FreeNAS -- to access your "Linux ISOs" quickly

 

[steveroch-rs]

Why? You already have a couple of Proxmox boxes. Just install Nextcloud on that and set up the Nextcloud data folder on the FreeNAS box in case you don't have enough hard disk space on the Proxmox boxes. That's what I do.

The other 2 boxes are a Intel Celeron platform with my router and DNS resolver which is too weak and the other one draws 120W Idle and is only powered on when I want to game.

Therefore I would be running Nextcloud on the third proxmox host that is running FreeNAS.



Have you experimented with virtualizing FreeNAS with Proxmox?