Can something like this be achieved with TrueNAS Scale?

[adityaharsh]

Can something like this be achieved with TrueNAS Scale?

Basic Idea:
1) One Mirror Pool with 3 ZVols, each containing a Windows 11 VM ( I can create 3 different Pools if need be )

2) Pass through 3 Nvidia A4000 to these three VMs respectively as the GPU

3) Output the VM i.e., Windows on an external Monitor connected to these GPUs

Plan to run 3 Windows machine on a single TrueNAS Server.

Specs:
AMD Epyc 7402P (24C/48T)
Supermicro H12SSL-I
128GB DDR4 3200 MT/s ECC RDIMM

 

[morganL]

Yes, it should work. Only 1 pool needed with 3 zvols. The testing with the 3 GPUs and displays should be done. Most users have tested with only a single GPU.

/scale/scaleuireference/virtualizationscreens/

 

[adityaharsh]

Yes, it should work. Only 1 pool needed with 3 zvols. The testing with the 3 GPUs and displays should be done. Most users have tested with only a single GPU.

/scale/scaleuireference/virtualizationscreens/

Thankyou for giving me the confidence on this, do I need to setup anything else in the BIOS or TrueNAS apart from IOMMU?

 

[morganL]

Thankyou for giving me the confidence on this, do I need to setup anything else in the BIOS or TrueNAS apart from IOMMU?

there will be things that can be set wrong in the BIOS.....it will vary by motherboard. Debug if there are issues.

 

[adityaharsh]

there will be things that can be set wrong in the BIOS.....it will vary by motherboard. Debug if there are issues.

I've seen in other posts and builds, how will I be able to pass through USB keyboard and mouse?

Also is there any scope where a config file save option be given in the future, for quicker VM setup?

 

[morganL]

I've seen in other posts and builds, how will I be able to pass through USB keyboard and mouse?

Also is there any scope where a config file save option be given in the future, for quicker VM setup?

With Angelfish, you do need a dedicated USB hub for a virtual machine.

Or you can use VNC.... perhaps a soft KVM switch might work? https://github.com/debauchee/barrier
(I don't have any experince or references on this except for reddit)

With Bluefin the granularity improved to a USB device: https://jira.ixsystems.com/browse/NAS-111020
Bluefin BETA in August. Might be available in a nightly earlier.

For enhancements, I'd suggest getting the basic system operating and then make a suggestion via Jira. Many improvements are already planned, you might find them already in the system.

 

[adityaharsh]

With Angelfish, you do need a dedicated USB hub for a virtual machine.

So, that means if I pass through 3 individual USB hubs from the usb ports on the motherboard to the 3 VMs, I can connect keyboard and mouse to each VM and run them on their individual displays, solving my original problem to run 3 Windows machine inside TrueNAS Scale?

 

[morganL]

So, that means if I pass through 3 individual USB hubs from the usb ports on the motherboard to the 3 VMs, I can connect keyboard and mouse to each VM and run them on their individual displays, solving my original problem to run 3 Windows machine inside TrueNAS Scale?

No, the USB hub has to be passed through as a PCIe device.... in Angelfish.

 

[Straafe]

@adityaharsh , I was wondering if you had an update regarding this build? I am planning something very similar.

 

[adityaharsh]

@adityaharsh , I was wondering if you had an update regarding this build? I am planning something very similar.

Hi, After some failed attempts on my orginal plan, I droped the idea of using TrueNAS Scale as my hypervisor.
Instead now I'm using Proxmox VE as my primary Hypervisor and TrueNAS Scale as VM inside Proxmox with PCIe passthrough of NVME drives and HBA Cards.

Hope you found the answer early on, saw this reply today itself

 

[adityaharsh]

Leaving this thread open just case if anyone else come around here for some guidance

 

[Straafe]

Since then I did go ahead with a multi-GPU multi-VM build and everything is working really well for me. Currently on Cobia with it. I have a P2000 for a media server (plex) and then 2x 3090's and an A4000 which I switch between VM's as needed. Everything works pretty well.