PCI nic Passthrough

mikegleasonjr

Dabbler
Joined
Sep 14, 2020
Messages
10
I have trouble getting a passthrough nic to be recognized in a guest VM (both BSD and Linux).

Running TrueNas release 12. I have 3 nics, 1 onboard and a dual nic card. All Intels.

So I have em0 (onboard), em1 (dual nic 1), em2 (dual nic 2). I am trying to passthrough em2.

What I did:

1. Followed the steps here to ensure I have virtualization enabled:
https://wiki.freebsd.org/bhyve/pci_passthru

2. Added relevant loader configs after determining the PCI device id:
Screen Shot 2020-10-23 at 2.36.16 PM.png


3. Verified that the device is available as a passthrough (em2 became ppt0):
Code:
% pciconf -lc
...
em1@pci0:1:0:0:    class=0x020000 card=0x7044103c chip=0x105e8086 rev=0x06 hdr=0x00
    cap 01[c8] = powerspec 2  supports D0 D3  current D0
    cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
    cap 10[e0] = PCI-Express 1 endpoint max data 128(256)
                 link x4(x4) speed 2.5(2.5) ASPM disabled(L0s)
    ecap 0001[100] = AER 1 0 fatal 1 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 002655ffffd4d732
ppt0@pci0:1:0:1:    class=0x020000 card=0x7044103c chip=0x105e8086 rev=0x06 hdr=0x00
    cap 01[c8] = powerspec 2  supports D0 D3  current D0
    cap 05[d0] = MSI supports 1 message, 64 bit
    cap 10[e0] = PCI-Express 1 endpoint max data 128(256)
                 link x4(x4) speed 2.5(2.5) ASPM disabled(L0s)
    ecap 0001[100] = AER 1 0 fatal 1 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 002655ffffd4d732


4. Created a VM (Tried pfSense and a generic Linux Distro) and assigned the passthrough device (of course, not at the same time):
Screen Shot 2020-10-23 at 2.44.10 PM.png


5. Booted the VM
I can't see a nic.

One thing I suspect is that I see an error in the pciconf -lc output for the passthrough device:

Code:
   ...
    ecap 0001[100] = AER 1 0 fatal 1 non-fatal 0 corrected
   ...


But I don't know how to debug that one out.

Thanks for your help!
 

Kennyvb8

Contributor
Joined
Mar 18, 2017
Messages
112
got same issue, only with VGA/GPU, the wiki is old, and alot of freebsd forums claim they bhyve can do vga/gpu now anyway, but just tried nic, same results as you
 

Ed Adame

Cadet
Joined
Apr 14, 2017
Messages
2
Same result for me on a PowerEdge T420 (Broadcom BCM5720 chipset). Tried both BSD and Linux, neither was able to see the passthrough device.

I just re-tested after updating to release 12 U1... same result...
 

Ed Adame

Cadet
Joined
Apr 14, 2017
Messages
2
Tested again with TrueNAS 12 U2 and pfsense 2.5. Same result... PCI passthrough device is not detected in the virtual machine.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
It's usually not possible to pass single ports on dual-port NICs, because they're all downstream of the same PCI host. The error message means the VM wasn't able to grab the PCI path 1/0, as that's in use in the host TrueNAS system. Try a separate PCI NIC, and passing that through, or passing through both ports.
 
Last edited:

shachar

Cadet
Joined
Dec 16, 2021
Messages
2
i had the same problem and the solution that ive found on the tuf x79 sabertooth is reset the bios and enable iommu (vt-d)
 
Top