Supermicro X8SIL-F network issue (Watchdog timeout -- resetting)

[Thomymaster]

Hi

I have a Supermicro X8IL-F mainboard vrunning the latest FreeNAS 9.3
The mainboard has 2 NICs with the following specs:

dmesg | grep em0
em0: <Intel(R) PRO/1000 Network Connection 7.4.2> port 0xdc00-0xdc1f mem 0xfb5e0000-0xfb5fffff,0xfb5dc000-0xfb5dffff irq 16 at device 0.0 on pci4
em0: Using MSIX interrupts with 3 vectors
em0: Ethernet address: 00:25:90:01:0d:a0

pciconf -lcv
em0@pci0:4:0:0: class=0x020000 card=0x060515d9 chip=0x10d38086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82574L Gigabit Network Connection'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
speed 2.5(2.5) ASPM disabled(L0s/L1)
cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
ecap 0001100 = AER 1 1 fatal 0 non-fatal 6 corrected
ecap 0003140 = Serial 1 002590ffff010da0

em1@pci0:5:0:0: class=0x020000 card=0x060515d9 chip=0x10d38086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82574L Gigabit Network Connection'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
speed 2.5(2.5) ASPM disabled(L0s/L1)
cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
ecap 0001100 = AER 1 0 fatal 0 non-fatal 6 corrected
ecap 0003140 = Serial 1 002590ffff010da1

vmstat -i after 5 minutes of operation

[root@storage] ~# vmstat -i
interrupt total rate
irq0: attimer0 1 0
irq21: ehci0 1507 1
irq23: ehci1 46114 48
irq256: hpet0:t0 339154 354
irq257: hpet0:t1 315863 330
irq258: hpet0:t2 355758 371
irq259: hpet0:t3 423055 442
irq264: em0:rx 0 653078 682
irq265: em0:tx 0 583423 609
irq266: em0:link 4832 5
irq267: em1:rx 0 838948 876
irq268: em1:tx 0 749606 783
irq269: em1:link 5 0
irq270: ahci0 544546 569
Total 4855890 5074

From time to time the network connection is dropped and on the console i see:

em1: Watchdog timeout -- resetting
em1: Queue(0) tdh = 0, hw tdt = 994
em1: TX desc avail = 30,Next TX to Clean = 0
em1: link state changed to DOWN
em1: link state changed to UP
em1: Watchdog timeout -- resetting
em1: Queue(0) tdh = 0, hw tdt = 994
em1: TX desc avail = 30,Next TX to Clean = 0
em1: link state changed to DOWN
em1: link state changed to UP

Has anybody of you have a problem similar to mine? I already opened a Bug (9390) and set the tuneable "hw.em.enable_msi" to 0, but it seems that its not honored (still the same output as above).

Best,

Thomas

 

[dlavigne]

That specific error usually means it's a cable error: https://www.freebsd.org/cgi/man.cgi?query=em.

 

[Thomymaster]

The errors still exist. I tried with the following tuneable "hw.em.enable_msix" -> 0 and dmesg shows me that MSI-X is now disabled but the errors still occur.
Also strange, when the errors start and i replug the cable, i get a "link UP" in dmesg but i can't even ping my local gateway. The errors the continue (every minute or so).

 

[Thomymaster]

Hi

I buyed a pcie intel 1000pt dual networking card and installed it. Under heavyload i still get the error messagers with this card and FreeNAS has to be rebooted :(