PCI nic Passthrough

[mikegleasonjr]

I have trouble getting a passthrough nic to be recognized in a guest VM (both BSD and Linux).

Running TrueNas release 12. I have 3 nics, 1 onboard and a dual nic card. All Intels.

So I have em0 (onboard), em1 (dual nic 1), em2 (dual nic 2). I am trying to passthrough em2.

What I did:

1. Followed the steps here to ensure I have virtualization enabled:
https://wiki.freebsd.org/bhyve/pci_passthru

2. Added relevant loader configs after determining the PCI device id:

3. Verified that the device is available as a passthrough (em2 became ppt0):

Code:

% pciconf -lc
...
em1@pci0:1:0:0:    class=0x020000 card=0x7044103c chip=0x105e8086 rev=0x06 hdr=0x00
    cap 01[c8] = powerspec 2  supports D0 D3  current D0
    cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
    cap 10[e0] = PCI-Express 1 endpoint max data 128(256)
                 link x4(x4) speed 2.5(2.5) ASPM disabled(L0s)
    ecap 0001[100] = AER 1 0 fatal 1 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 002655ffffd4d732
ppt0@pci0:1:0:1:    class=0x020000 card=0x7044103c chip=0x105e8086 rev=0x06 hdr=0x00
    cap 01[c8] = powerspec 2  supports D0 D3  current D0
    cap 05[d0] = MSI supports 1 message, 64 bit
    cap 10[e0] = PCI-Express 1 endpoint max data 128(256)
                 link x4(x4) speed 2.5(2.5) ASPM disabled(L0s)
    ecap 0001[100] = AER 1 0 fatal 1 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 002655ffffd4d732

4. Created a VM (Tried pfSense and a generic Linux Distro) and assigned the passthrough device (of course, not at the same time):

5. Booted the VM
I can't see a nic.

One thing I suspect is that I see an error in the pciconf -lc output for the passthrough device:

Code:

   ...
    ecap 0001[100] = AER 1 0 fatal 1 non-fatal 0 corrected
   ...

But I don't know how to debug that one out.

Thanks for your help!

 

[Kennyvb8]

got same issue, only with VGA/GPU, the wiki is old, and alot of freebsd forums claim they bhyve can do vga/gpu now anyway, but just tried nic, same results as you

 

[Ed Adame]

Same result for me on a PowerEdge T420 (Broadcom BCM5720 chipset). Tried both BSD and Linux, neither was able to see the passthrough device.

I just re-tested after updating to release 12 U1... same result...

 

[Ed Adame]

Tested again with TrueNAS 12 U2 and pfsense 2.5. Same result... PCI passthrough device is not detected in the virtual machine.

 

[filipejps]

same here with
TrueNAS-12.0-U4

 

[Samuel Tai]

It's usually not possible to pass single ports on dual-port NICs, because they're all downstream of the same PCI host. The error message means the VM wasn't able to grab the PCI path 1/0, as that's in use in the host TrueNAS system. Try a separate PCI NIC, and passing that through, or passing through both ports.

 

[shachar]

i had the same problem and the solution that ive found on the tuf x79 sabertooth is reset the bios and enable iommu (vt-d)