VM unable to see passthrough PCIe nic

[iambenmitchell]

Hello all,

I am new to TrueNas, the last time I used this was back in 2014 or so and it was called FreeNAS I believe. Anyways, it has changed a lot and I'd like to use it again.

I am looking to host my PFsense or OPNsense firewall as a VM in TrueNAS. To do this I have gone and bought a Quad 2.5G NIC and I am looking to pass it into the VM without TrueNAS touching it.

I think I have done everything correctly by going to VM - Devices - Add - PCI Passthrough and selecting the device.

Weirdly it shows up as 4 separate devices, I think each port might have its own controller, so I have added 4 different PCI devices to correspond with each port. These are the ones I have added:

Code:

pci_0000_05_00_0
pci_0000_06_00_0
pci_0000_07_00_0
pci_0000_08_00_0

And this is the output of lspci (cropped to the NIC)

Code:

05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
08:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)

However, upon booting OPNsense (and I have tried PFsense too), I am getting something along the lines of "No network interfaces detected". So my question is, is there something I am missing? I couldn't see the PCIe devices earlier, turns out VT-D was disabled, so I enabled it and then I was able to add them. Perhaps because TrueNAS was setup with VT-D disabled, I need to enable some kernel module or something? I am not sure as to why this is not working as PFSense and OPNsense both support the RTL8125.

My System:

Code:

CPU: Intel(R) Pentium(R) Gold G6405 CPU @ 4.10GHz
RAM: 8GB
TrueNAS: TrueNAS-SCALE-22.12.2

 

[jgreco]

With only 8GB of RAM, you barely have enough memory to run TrueNAS itself, you don't have enough for VM's.

There have been some problems with Realtek ethernets and PCIe passthru in the past, which is one of the reasons we do not recommend the Realtek stuff.

Additionally, you haven't identified what kind of mainboard you're using. This can impact your ability to do PCIe passthru if the board doesn't handle setup correctly.

 

[iambenmitchell]

With only 8GB of RAM, you barely have enough memory to run TrueNAS itself, you don't have enough for VM's.

There have been some problems with Realtek ethernets and PCIe passthru in the past, which is one of the reasons we do not recommend the Realtek stuff.

Additionally, you haven't identified what kind of mainboard you're using. This can impact your ability to do PCIe passthru if the board doesn't handle setup correctly.

I plan to upgrade the ram soon.

My motherboard is a Z590I VISION D which uses the Z590 chipset so it should have all the top end features.

 

[Ericloewe]

My motherboard is a Z590I VISION D which uses the Z590 chipset so it should have all the top end features.

Your optimism is palpable, but misplaced. Beyond Intel's famously-aggressive segmentation efforts, most vendors these days have a hard enough time shipping a motherboard that works in their limited testing, much less stuff the average desktop user is clueless about such as PCI pass-through.

Weirdly it shows up as 4 separate devices

The RTL8125 is a single-port controller, so someone stuck four of them behind a PCIe switch. That's valid, but the RTL8125 is pretty terrible, and having four of them doesn't mitigate that.

More to the point of your question, you should investigate whether the driver has been intentionally disabled, as it was in TrueNAS, following the discovery of a bug that corrupted data transmitted over iSCSI.