Split IOMMU Group for GPU Passthrough

[ryanw]

Hi,
I've tried isolating an NVIDIA 1030 in TrueNAS-SCALE-22.02.1 and then passing it through to an Ubuntu VM.
Unfortunately it seems the GPU shares and IOMMU group with the LSI2308 SAS controller meaning that the VM will not boot and complains about the fact that all devices in the same group must be passed through.
I can't switch the GPU to another slot as there are no others. :)
How can I split out the GPU to another IOMMU group in order that is can be successfully passed through to the VM?

PS. I've tried adding "intel_iommu=on pcie_acs_override=downstream" to the end of the GRUB_CMDLINE_LINUX_DEFAULT="quiet" line and update-grub, but this didn't work.

Any ideas?
Thanks.

 

[Adnan]

This works to edit GRUB but I had no result of splitting the group.

How to edit GRUB_CMDLINE_LINUX_DEFAULT?

I was running Proxmox on my home server and now I am trying to switch to TrueNAS Scale. I did run into a problem with passing devices into the VM. [EFAULT] internal error: qemu unexpectedly closed the monitor: 2021-09-11T14:34:10.839083Z qemu-system-x86_64: -device...

www.truenas.com

 

[Dough29]

Hello,

I'm facing the same issue passing through a GTX 1060 GPU.

Other PCIe lines come in the same IOMMU Group :

Code:

IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 05)
IOMMU Group 1 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) [8086:1905] (rev 05)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106 [GeForce GTX 1060 6GB] [10de:1c03] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation GP106 High Definition Audio Controller [10de:10f1] (rev a1)
IOMMU Group 1 02:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)

I tried adding "pcie_acs_override=downstream" boot option without any effect.

Do someone got this sorted out ?

Thanks.

 

[Sasquatch]

i found that you cannot isolate evice connected to chipset, only devices directly connected to cpu can be isolated...at least using gui to isolate gpu wont work unless its fitted in gpu slot or through adapter to first m.2 slot. on amd system at least