10G NIC disconnected at boot

[bigdweeb]

Server Setup

• Software: TrueNAS Core 13.0-U4
• Motherboard: Supermicro X10SLL-F
• CPU: Intel Xeon E3-1240 v3
• RAM: 32GB ECC
• Boot Disk: mirrored 256G SSD
• Disk Controllers: 2x LSI SAS2308
• Pool 1: 6x 8TB RAIDZ2
• Pool 2: 6x 1TB Flash RAIDZ2
• NICs (builtin): Intel I210, I217-LM
• NIC (PCIe): Supermicro AOC-STGN-I2S
• Optic: Finisar FTLX8571D3BCL (SFP-10G-SR)

I have had this setup running for several years using one of the builtin Intel NICs. Everything works as expected connected to a copper port on my Unifi US-48-G1 switch. I recently installed the Supermicro AOC-STGN-I2S in the system. It looks like from the specs that card is a PCI-E 2.0 x8. I currently have it slotted in the PCIe 2.0 x4 (in x8 slot) which I don't think is ideal.

When I power the system up, the lights on the card light up and the link on the 10G port of the Unifi US-48-G1 switch links, but the TrueNAS interface and the underlying FreeBSD ifconfig both say the interface ix0 is disconnected. If I reboot the system with a Debian Linux live USB and keep everything else the same, the interface links up and is accessible over the 10G port. If I reboot back into TrueNAS, then ssh into the shell and type ifconfig ix0 up, the interface links and I can use it.

Can anyone explain how I can resolve this?

 

[difggy]

Hi I have a similar issue as you, but at boot, one port shuts off. I've also seen both ports lit while TrueNAS says "Link state down". Once I configured one of hte interfaces with an ip and then rebooted, the link went up, but only on one port. I know both ports are good because I tested them with other devices. Something about my unifi switch that Truenas doesnt like. Sorry I couldnt provide any solution to your issue, but thought I'd share my experience.