10Gb Adaptor Error

[jea001]

I encountered an error I have been unable to solve today. While troubleshooting a failed flash drive (used for my jails volume), both the 10Gb ports on my board stopped working. When I connect a network cable to either 10Gb port, I receive the following error:

ix0: Error handling LASI interrupt: -3

I receive the error on both ix0 & ix1, my 10Gb adaptors.

I have rebooted my server several times, including powering it off and leaving it unplugged for several minutes before restarting. I rebooted my router between one of the server power cycles. I also attempted booting into FreeNAS-11.1-U2 instead of my normal FreeNAS-11.1-U4. In all cases, I received the error shown above when either booting with a network cable connected or upon connecting a network cable to one of the 10Gb ports.

I have searched both these forums and Google for additional troubleshooting ideas and not found anything that I can understand or use. The error is not critical as I am not running a 10Gb network & my 1Gb adaptors are working normally, so I have shifted my network connection to one of those connectors and everything is working fine. However, I would eventually like to run a 10Gb network, it is one of the reasons I purchased my motherboard and I would like to resolve this problem.

Any assistance troubleshooting this problem would be greatly appreciated.

My FreeNAS build specs:
Motherboard: SuperMicro X10SDV-4C+-TLN4F
CPU: Intel Xeon D-1518
RAM: 32GB (2x 16 GB Micron MTA36ASF2G72PZ-2G1A2 DDR4 RDIMM)
Boot Drive: Corsair Force MP500 M.2 2280 120GB PCI-E SSD
Storage Volume Hard Drives: 6x WD Red 4TB (WD40EFRX)
Jails Volume Storage: 2x SanDisk 32GB Ultra Fit (1 currently installed, 1 faulty & removed, replacement on order)
No separate storage or network controllers
FreeNAS Version: 11.1-U4

 

[Ericloewe]

I assume the interfaces in question are the ones from the SoC and not some discrete card. They should be well-supported in 11.1, so we have a bit of a mystery here. The first step is to figure out what the -3 return code stands for and what the interrupt in question is.

 

[jea001]

They are indeed the SoC interfaces.

My Google searching has led me to the source code for the Intel 10Gbe drivers for FreeBSD, but it hasn't helped me with figuring out the -3 return code. My coding experience is rudimentary and mainly limited to Matlab and Fortran from engineering school. I've only had a little practice with C and definitely not for programming NIC drivers.

The one thing I noticed is that this error appears as an elseif clause in an if statement with an overheat error. I know that it may have nothing to do with the overheat error, but my thought is, can't hurt to try. So, I've shut my server down and will leave it shut down for about 16 hours while I'm asleep and at work. This should allow plenty of time for everything to cool off and I'll try again tomorrow afternoon. My previous shutdowns were under 10 minutes, so, let's see what effect time has.

Anyways, time for me to go to sleep, work will be upon me too soon.

 

[jea001]

Well, my experiment worked. After being off for around 17 hours, I plugged the server in, booted up, and had no issues getting a connection on the 10Gb connection. While it may not be the causation, based on the code I found yesterday, a 10Gb interface that already runs warm, higher temperatures than I have experienced since I built my server in December, and fan curves optimized to keep noise down and my wife happy, I think that it was running a little too warm. I'll work on optimizing cooling to try and keep this happening again.

Ericloewe, thanks for your help, it prompted me to dig around in the code a little more than I had previously.

 

[Ericloewe]

Interesting. I would've hoped for a more explicit warning in case of overheating...

Let's hope that solves it.

 

[MrToddsFriends]

I'll work on optimizing cooling to try and keep this happening again.

Could you please drop a line (or two) about the case and the cooling solution in use?

 

[jea001]

Could you please drop a line (or two) about the case and the cooling solution in use?

Absolutely. I am using a Fractal Design Node 304. I have replaced the stock fans with 2x Noctua NF-A9 PWM in the front and 1x Noctua NF-A14 PWM in the rear. My build is very similar to what Stux is running in his Node 304 build, I even used his fan control script.

I know that this board was designed for rack mounting with significant front to back airflow. That definitely is not the case in my build, so I may need to increase the speed of my fans when it is summer and the house is warmer. The other thought I had is finding a way to mount another fan to provide cooling on the 10Gb adaptors.

I eventually want to add an HBA to shift my jail storage to an extra SATA SSD I have, but I know they have a tendency to run hot, so I need to consider how I'm going to keep everything cool in this case before I add even more heat to it.

 

[MrToddsFriends]

The other thought I had is finding a way to mount another fan to provide cooling on the 10Gb adaptors.

Most likely I would try to find a way to mount a Noctua NF-A6x25 (PWM or FLX) above these parts, at least as long no other sources of heat come into play.