Unable to Pass PCI Device to VM

[theprez]

Morning -

I'm trying to pass through a PCI Intel Network Card to a specific virtual machine. To do that, I:

1) confirmed that IOMMU is enabled via:
dmesg | grep -e DMAR -e IOMMU

2) Identified the PCI device in question using lspci

3) Edited the VM and added the PCI device passthrough (having already identified it via lspci) and saved my changes. Attempting to relaunch the VM generates the following error:

"[EFAULT] internal error: qemu unexpectedly closed the monitor: 2022-02-17T17:34:27.195899Z qemu-system-x86_64: -device vfio-pci,host=0000:02:00.1,id=hostdev0,bus=pci.0,addr=0x5: vfio 0000:02:00.1: group 15 is not viable Please ensure all devices within the iommu_group are bound to their vfio bus driver."

I thought I read on here (maybe it was CORE and not SCALE) that there shouldn't be any manual loading of drivers or modules but it seems like something isn't working correctly here. Any ideas?

Thanks

 

[theprez]

As an update in case this helps others - you have to select both PCI addresses with in a given group. In my case, my network adapter was a dual port adapter and I was incorrectly selecting only once PCI address. Going back and adding a second PCI address as a new entry resolved the issue.

 

[dffvb]

Yes thats an issue, you can only passthrough full IOMMU groups...

 

[waqarahmed]

@theprez in some cases this is dependent on the PCI devices in question. Like for GPU passthrough, we want to the GPU devices from the host as soon as system boots as otherwise we are not able to do so later when the system has booted. Similarly, in some cases PCI devices which do not have reset mechanism defined - we are unable to properly isolate them from the host on the fly as these devices have different behaviors with some isolating but when we stop the VM, they should be given back to the host but that does not happen whereas for some other devices stopping the VM hangs the VM indefinitely as it did not have a reset mechanism defined.

Generally this is not required that you isolate all of the devices in your IOMMU group as the system usually does this automatically but some devices can be picky. We have a suggestion request open which allows you to isolate devices from the host on boot automatically and keep them isolated similar to how system does for GPU devices. However seeing this case, it might be nice if you create a suggestion ticket to somehow perhaps allow isolating all PCI devices in a particular IOMMU group clarifying how you think the feature should work.

 

[Beliall]

As an update in case this helps others - you have to select both PCI addresses with in a given group. In my case, my network adapter was a dual port adapter and I was incorrectly selecting only once PCI address. Going back and adding a second PCI address as a new entry resolved the issue.

Hello :)
Im tying to achieved to passthrough my usb controller to a vm
Can you give me some detail about your step 3 : 3) Edited the VM and added the PCI device passthrough ?

When im reading your error message i understand i must seek around qemu vfio and immou group.
I w'ill dig in this direction and hope and answer :)
Regards

PS: i open my own post here : https://www.truenas.com/community/threads/usb-passthrough-via-pci-passthrough.99456/

 

[Richard.T]

Hello theprtz,

I'm experiencing the same issue as Beliall, even though I confirmed that my CPU support IOMMU. I don't understand how you "Edited the VM and added the PCI device passthrough ...": I simply cannot find any passthru device in the list when I try to add a device into a VM.

My hardware is HP Gen8.

Could you please help to clarify a little bit?

Regards

Richard

 

[theprez]

Sorry for the delay, not sure why but I didn't/don't get notifications when I get replies but maybe this will help others still...

Step 3 - you're basically telling the host what device you want to pass through to the associated virtual machine. It doesn't display things in the drop down in an easily understood human recognizable manner so you have to fill in the blank - essentially you have to tell it the PCI address of the device/adapter/dongle/card so it can then pass it along to the VM. For any of this this to work, your CPU has to support IOMMU (not sure what AMD boards call that feature, sorry)

Anyways, the easiest way I've determined is to go about identifying the device in one of two ways:

Way 1:

1) Go to a shell prompt (I use SCALE, so its under System Settings -> Shell) and type in lspci and observe the output.
2) If you are able to recognize the device based on the description, make note of the information in the far left (such as 7f:0d.0) as you'll need that for step 3.
3) Back under your virtual machine - go to devices -> add. For type select PCI pass through device, allow a few moments for the second dropdown to populate. Select the appropriate item that matches what you found in step 2. Note: there may be preceding zeros. So following the same example as I mentioned in step 2, in my case it shows in the drop down menu pci_0000_7f_0d_0. That's the one I selected.
4) Change the order if desired, otherwise click save.

Way 2:

1) Observe the console log and insert the desired device (such as a USB drive or other peripheral) and observe what appears in the console.
2) In my case it shows a new USB device was found, the vendor of the device, and the PCI slot information . Take note of this, it's needed for the next step. In my example, it showed: 00:1a.0

Hint: You can also drop to a shell and run: lspci | grep USB if you're using a USB device.
3)Follow Step 3 from Way 1.


Note: Some devices require both PCI device IDs to be passed - such as the case of my dual NIC intel card. Had to identity and pass both PCI addresses.