Proxmox: Where to install TrueNAS: local-zfs or physical drive attached to HBA?

[coldplug]

Hello!

I want to know what is better between these two options, before I do something and regret later.

Let's say that I have Proxmox installed on SSD that is attached to the motherboard and also have other SSDs reserved for my local-zfs space. Apart from those drives, I have attached PCIe HBA card with plugged one SSD and two HDDs, with whole HBA being passed through for the virtualized TrueNAS.

Now, should I install TrueNAS OS onto local-zfs, or onto SSD attached to HBA? I saw only examples where people install OS onto virtual disk created on local storage, but would this lead to "ZFS on top of ZFS" problems? Should it be better to install TrueNAS OS onto dedicated SSD drive attached to HBA and that way have all disks that TrueNAS sees non-virtualized and fully separated from Proxmox? Obviously, HDDs will be used in TrueNAS for data zfs pool (mirror).

Let me know if I misundertand something.
Thank you.

 

[Basserra]

I think that it would still be recommended to install TN OS directly to a disk. But, I would assume that it wouldn't be a huge deal if a virtual disk fails and you just remake it and reload the TN config, but you would need to keep that config safe and available. I'm thinking about swapping to Proxmox on baremetal too and leaning towards using a virtual disk and just keeping the config ready. Could anyone add to this matter? Could we just disable some of the ZFS features on the boot pool in TN VM and have Proxmox manage it?

 

[Dice]

You are on the right track in terms of awareness.

Install the os on a proxmox darastore (local in your case), is fine. No benefits are derived from having the os drive on a dedicated drive in this kontext.

Make a proper pci pass through of the HBA (manual steps are required, consult proxmox documentation).

Put all drives intended for truenas storage on the hba.