USB/PCI passthrough finally possible?

[Patrick M. Hausen]

Hi all,

I was surprised by this dialog in Virtual Machines -> <Machine> -> Devices:


Does that imply that I could probably pass my xhci controller to my Windows VM to connect and control my UPS with its Windows software? That would be awesome!
How do I go about it?

Code:

xhci0: <Intel Lynx Point USB 3.0 controller> mem 0xfb700000-0xfb70ffff irq 19 at device 20.0 on pci1
xhci0: 32 bytes context size, 64-bit DMA

Thanks!
Patrick

 

[waqarahmed]

Hi @Patrick M. Hausen yes it is indeed possible. Can you please confirm your TN version ? Are you using TN Core or TN Scale ?
In either cases you can select the pci devices available and use them for your virtual machine.

 

[Patrick M. Hausen]

TN Core but I do not get to select anything in that drop down. See screenshot. So my assumption was that I need to set some tunables to keep the FreeBSD base OS from "grabbing" the device.

 

[waqarahmed]

Yes that is accurate, these steps should work for TN Core https://wiki.freebsd.org/bhyve/pci_passthru, for SCALE we need to use vfio-pci driver.

Please let me know if you run into issues. Thank you

 

[Patrick M. Hausen]

Booting Windows with device 0/20/0 passed through crashed my FreeNAS.
So postponed for now - thanks a lot for these hints, anyway!

Plus - my IPMI console lost its keyboard. But I expected that after checking the mainboard architecture ;-)

 

[Yorick]

I was able to pass USB through to Windows VM without crashing TN Core. YT about this is here: https://youtu.be/i0MAZBX7P-U

 

[Patrick M. Hausen]

DMAR - check
MSI - check

And guess what? The second attempt worked ... weird.

 

[warllo]

I was able to pass USB through to Windows VM without crashing TN Core. YT about this is here: https://www.youtube.com/watch?v=YLJ5t1kUVMM

Thanks for that YT by the way. It was very helpful.

 

[Yorick]

I just realized I left out the part where I actually show it working lol. I may delete that video and re-upload.
Edit: Done, tacked that part onto the end, deleted the old video, uploaded a new one.

 

[Patrick M. Hausen]

And I just lost the reason why I started this mini project in the first place. I bought a Powercom UPS a year ago because it was supposedly "5 stars - green" in the NUT compatibility list. Unfortunately the usbhdi driver only works on Linux, not on FreeBSD, so I never properly connected it to my FreeNAS. It probably eliminated the occasional surge but I never implemented orderly shutdown.

Now with the proprietary software running in Windows I intentionally pulled the plug for the first time.

Result:
- Battery from 100% to 30% immediately
- Hard power off of the UPS after less than two minutes even after I quickly restored power

Seems like this unit is done for. I will probably get something with an Ethernet interface this time. Just need to make sure to give the network equipment UPS power, too.

But nice to know that it works now

 

[ChrisChros]

I was able to pass USB through to Windows VM without crashing TN Core. YT about this is here: https://youtu.be/i0MAZBX7P-U

Thanks for this tutorial.
I tried to passthrough my usb to a Debian VM but it worked not as expected.
With lsusb I can see my USB controller but when I plug something in it looks like everything freeze, even TureNAS. Problem looks like the same @Patrick M. Hausen had with his Windows VM.

Code:

dmesg |grep xhc
[    0.857321] xhci_hcd 0000:00:05.0: xHCI Host Controller
[    0.857325] xhci_hcd 0000:00:05.0: new USB bus registered, assigned bus number 1
[    0.858388] xhci_hcd 0000:00:05.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0000000001109810
[    0.858426] xhci_hcd 0000:00:05.0: cache line size of 64 is not supported
[    0.859290] usb usb1: Manufacturer: Linux 4.19.0-13-amd64 xhci-hcd
[    0.859859] xhci_hcd 0000:00:05.0: xHCI Host Controller
[    0.859861] xhci_hcd 0000:00:05.0: new USB bus registered, assigned bus number 2
[    0.859862] xhci_hcd 0000:00:05.0: Host supports USB 3.0 SuperSpeed
[    0.859885] usb usb2: Manufacturer: Linux 4.19.0-13-amd64 xhci-hcd
[    0.865968] xhci_hcd 0000:00:1e.0: xHCI Host Controller
[    0.865971] xhci_hcd 0000:00:1e.0: new USB bus registered, assigned bus number 3
[    0.867417] xhci_hcd 0000:00:1e.0: hcc params 0x03401280 hci version 0x100 quirks 0x000000000000b930
[    0.868970] usb usb3: Manufacturer: Linux 4.19.0-13-amd64 xhci-hcd
[    0.869399] xhci_hcd 0000:00:1e.0: xHCI Host Controller
[    0.869401] xhci_hcd 0000:00:1e.0: new USB bus registered, assigned bus number 4
[    0.869402] xhci_hcd 0000:00:1e.0: Host supports USB 3.0 SuperSpeed
[    0.869496] usb usb4: Manufacturer: Linux 4.19.0-13-amd64 xhci-hcd
[    1.098546] usb 4-1: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd

the driver is active and supplied by the kernel.

Code:

lspci -v | grep xhci
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci

Any suggestions how to fix?

Regards Chris

 

[seanm]

So do I understand correctly that you can't pass through individual USB devices? Only the entire USB controller? Wouldn't that mean losing the keyboard on the TrueNAS computer?

 

[Patrick M. Hausen]

@seanm, yes. You cannot pass USB devices. Only PCI devices, like network cards, NVME SSDs or in our case entire XHCI controllers. If the system in question features multiple separate USB controllers, you can possibly keep keyboard and mouse.

In my particular case the emulated keyboard and mouse provided by IPMI are connected to the single XHCI I would need for the USB ports, too ...

 

[Yorick]

but when I plug something in it looks like everything freeze, even TureNAS.

Are you using USB for anything with TrueNAS? Boot? Mouse? Keyboard? IPMI?

You are passing the entire controller through which means it cannot be used for anything in TrueNAS.

 

[ChrisChros]

I don't use USB for anything within TrueNAS when it's running. Also is no mouse or keyboard connected during normal operation.
As far as I understood you can only pass the complete USB controller to e.g. a VM. At the moment there is no possibility to pass only the "front USB" to a VM.

But, also with this in mind, a canceled the project to pass USB to one of my VM's, because then I can't enter the TrueNAS in an emergency, I don't habe a IPMI interface on my mainboard.

 

[jace92]

Hey all! I know I'm digging up an old thread but I'm new here and my issue is related to the YouTube link @Yorick posted. If this doesn't belong here let me know!

I was able to get the PCI USB card passed through (at least it looks like it in the Windows VM) but the flash drive I connected doesn't show. Anyone know why?