PCI Passthrough Headache

[DavidLiptrot]

Morning all,

To begin, I'm pretty new to all this, trying to pick it up as a figure out what I want to do with my machine. Have previously been using Raspberry Pis for everything plus a Synology box as my NAS.

TrueNAS-SCALE-22.02.0.1
Intel 8700k
Intel Z390 MAXIMUS XI HERO
No name eBay special PCIe USB controller

I've been trying to passthrough the USB controller listed above to an Ubuntu VM. I have managed to assign the VFIO-PCI driver using the usual method in the past although couldn't get the VM to boot once it was assigned. In trying to get past this issue, I'm now stuck with the USB controller using the XHCI_HCD driver.

The device is in its own IOMMU group and the controller is assigned to the VFIO-PCI driver in the grub file.

There were no error messages when starting the Ubuntu VM when I did have the VFIO-PCI assigned. I've got a RTX 3080 assigned to a Windows VM at the moment which is working fine so I'm fairly happy that everything else is set up properly.

Would anyone be able to point me in the right direction please?

Cheers

 

[DavidLiptrot]

Bump

I've managed to get it working great on UNRAID but I'm preferring the TrueNAS platform

 

[DavidLiptrot]

So, Update...

I've had the VM ticking along nicely with the USB Controller passed through, no troubles until I've put the GPU back in the system.

I've dived into the log and hopefully I've pulled out the correct sections. On starting the VM now, I'm getting the following in the kernel log...

Jul 3 03:11:15 truenas kernel: audit: type=1400 audit(1656843075.405:76): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="libvirt-d13c81fb-5225-4a45-8e6d-2431e79bcca4" pid=1601074 comm="apparmor_parser"
Jul 3 03:11:16 truenas kernel: vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Jul 3 03:11:16 truenas kernel: vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Jul 3 03:11:17 truenas kernel: audit: type=1400 audit(1656843077.777:77): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-f7375bb1-0d29-49b2-ad4c-c44787670093" pid=1601287 comm="apparmor_parser"
Jul 3 03:11:17 truenas kernel: audit: type=1400 audit(1656843077.893:78): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="libvirt-f7375bb1-0d29-49b2-ad4c-c44787670093" pid=1601290 comm="apparmor_parser"
Jul 3 03:11:17 truenas kernel: audit: type=1400 audit(1656843077.977:79): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="libvirt-f7375bb1-0d29-49b2-ad4c-c44787670093" pid=1601294 comm="apparmor_parser"
Jul 3 03:11:18 truenas kernel: audit: type=1400 audit(1656843078.061:80): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="libvirt-f7375bb1-0d29-49b2-ad4c-c44787670093" pid=1601298 comm="apparmor_parser"
Jul 3 03:11:18 truenas kernel: audit: type=1400 audit(1656843078.217:81): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="libvirt-f7375bb1-0d29-49b2-ad4c-c44787670093" pid=1601310 comm="apparmor_parser"
Jul 3 03:13:07 truenas kernel: audit: type=1400 audit(1656843187.130:82): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="libvirt-f7375bb1-0d29-49b2-ad4c-c44787670093" pid=1605609 comm="apparmor_parser"
Jul 3 03:13:07 truenas kernel: xhci_hcd 0000:04:00.0: xHCI Host Controller
Jul 3 03:13:07 truenas kernel: xhci_hcd 0000:04:00.0: new USB bus registered, assigned bus number 3
Jul 3 03:13:07 truenas kernel: xhci_hcd 0000:04:00.0: Host halt failed, -19
Jul 3 03:13:07 truenas kernel: xhci_hcd 0000:04:00.0: can't setup: -19
Jul 3 03:13:07 truenas kernel: xhci_hcd 0000:04:00.0: USB bus 3 deregistered
Jul 3 03:13:07 truenas kernel: xhci_hcd 0000:04:00.0: init 0000:04:00.0 fail, -19
Jul 3 03:13:12 truenas kernel: xhci_hcd 0000:04:00.0: xHCI Host Controller
Jul 3 03:13:12 truenas kernel: xhci_hcd 0000:04:00.0: new USB bus registered, assigned bus number 3
Jul 3 03:13:12 truenas kernel: xhci_hcd 0000:04:00.0: Host halt failed, -19
Jul 3 03:13:12 truenas kernel: xhci_hcd 0000:04:00.0: can't setup: -19
Jul 3 03:13:12 truenas kernel: xhci_hcd 0000:04:00.0: USB bus 3 deregistered
Jul 3 03:13:12 truenas kernel: xhci_hcd 0000:04:00.0: init 0000:04:00.0 fail, -19

Does that mean anything to anyone? When doing lspci in Ubuntu, I can see that the USB controller is passed through but any connected devices don't show up in lsusb and plugging in a device whilst the VM is running crashes the VM.

Thanks in advance.

Dave