Best practices new virtualized installation in Proxmox

[Mario Rossi]

Hi everyone.
I need to upgrade my home lab.
My hardware firewall with opnsense is dead and my synology nas doesn't satisfy me.

I'm assembling a PC with
- ASRock B650M Pro RS
- AMD Ryzen 5 7600
- 1x Samsung 990 PRO 1TB NVMe M.2 SSD
- 32GB Ram
- 2x 6TB WD Gold sata

Proxmox will be installed on the SSD in RAID0 ZFS
All VMs and CTs will be installed on the SSD
I would like to dedicate the two HDs in RAID1 ZFS to TrueNAS Core which will only act as a domestic NAS and will contain NextCloud inside.

I have read various of your documentation and I understand that TrueNAS should be installed in baremetal but I can't afford it due to costs, consumption and physical space.

I've read the documentation and I'm having a hard time understanding how best to install virtualized TrueNAS Core in Proxmox.

- Do I passthrough the mobo's sata controller?
- - do I configure it in RAID or SATA in UEFI?
- Do I only passthrough disks? (e.g. qm set 103 -scsi1 /dev/disk/by-id/ata-ST500DM002-1BD142-W2AV3B3T)
- Do I install and configure TrueNAS in baremetal on the PC, then export the configuration, format the SSD and install proxmox, install TrueNAS in VM and upload the baremetal configuration to it?
- - but how do I connect the two HDs? always passthrough?
- well, I accept suggestions
- do I leave TrueNAS alone and look for other software that acts as a NAS like Zamba?

In the end I need it to share the files over the network with the other PCs and act as a repository for the data of the other VMs.

Thanks in advance for the replies.

 

[HoneyBadger]

Hi @Mario Rossi

Check out our blog on "Yes, you Can Virtualize TrueNAS" here for some details:

Yes, You Can (Still) Virtualize TrueNAS

TrueNAS is a powerful storage virtualization software solution that helps businesses virtualize data. Get started today with TrueNAS.

www.truenas.com

The short version is:

Full hardware passthrough of the controller (SATA or SAS) is best. Since your SSD is NVMe, that sorts the problem of the isolated controller not being available to Proxmox.
Use AHCI/non-RAID mode for your controller in the UEFI setup. Let TrueNAS manage the mirroring/"RAID logic".

You also need a small boot device for TrueNAS to install to, as you can't use the install disk for sharing. This could be a virtual disk file on your NVMe SSD if desired, since you'll need somewhere to storage the VM configuration file anyways.

Cheers and welcome to the TrueNAS community!

 

[Mario Rossi]

Thanks for your answer.

For the installation, is it enough for me to create a 16GB/32GB disk on the SCSI BUS (VirtIO SCSI)?

What do you recommend for the remaining Proxmox parameters?

 

[HoneyBadger]

A VirtIO disk for the boot device only is fine. It doesn't need to emulate an SSD or have any special functionality.

For the remaining setup, the HBA/SATA controller will need to be assigned to the guest VM via PCI(e) Passthrough, as detailed here for Proxmox VE:

PCI(e) Passthrough - Proxmox VE

pve.proxmox.com

Some of the steps may be different depending on hardware; note that AMD processors with iGPUs sometimes have issues with IOMMU isolation and being able to break up their PCIe devices sufficiently, but this is typically more related to GPU isolation.

 

[Mario Rossi]

As soon as I have the hardware available I will update the UEFI to the latest version and then do some fine tuning. I'm researching about it.

I will check the IOMMU capabilities of the cpu/mobo and hopefully everything goes smoothly.

The iGPU will never be used, it will only be used for the first UEFI and proxmox setup and in case of emergency if the system no longer starts.

The network card will be a StarTech X710-DA4 plus the spare integrated one.

 

[Mario Rossi]

p.s. shouldn't the discard function be activated either?

 

[HoneyBadger]

If you're concerned with reclaiming space inside of the 32GB boot device then yes, you can enable SSD emulation + discard. Discard/UNMAP on your data disks would be handled inside TrueNAS since you'll be passing the entire controller through to it.