Solarflare 10Gb, LACP and dumping core

[aufalien]

Hi,

I've configured an LACP link using a duel 10Gb SolarFlare adapter (model #SFN5162F).

I'm able to use the link w/o issue so far. However when I reboot FreeNAS, the system dumps core and when it finally boots, checks USB that the FreeNAS OS is running from as it was uncleanly mounted.

I'm assuming that its dumping as the screen attached to it goes ballistic with hex values/alpha numeric text for a good few minutes. I think it was flipping me off but I digress :)

Under View Interfaces, the media column for lagg0 shows Unknown.

My switch is configured for 802.3ad on the trunk/bond.

Configuring non LCAP as individual ports has no issues restarting.

Systems specs;
FreeNAS v9.2.1.3
42x3TB Seagate ES.2 in 4xIntel JBOD 2000 chassis
256GB mem
2xIntel DC3700 - ZIL
Duel 10Gb SolareFlare Server NICs
LSI 92016-16e
Intel 2430x2 Hexcore CPUs on Intel 2400SC2 main board (24 threads total)
Running off 220V
Connecting to a Foundry SuperX
No sysctl or tunables enabled of any kind

Herr is ifconfig -m lagg0 output;

lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
options=c07ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE>
capabilities=c07ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE>
ether 00:0f:53:08:44:7c
inet 10.0.10.99 netmask 0xffff0000 broadcast 10.0.255.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
status: active
supported media:
media autoselect
laggproto lacp lagghash l2,l3,l4
laggport: sfxge1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport: sfxge0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>

Should I file a bug?

* UPDATE;
Unsure if its a core dump as there are no cores in the system dataset pool. In other words the cores directory is empty.

 

[aufalien]

Hi all,

An update;

https://bugs.freenas.org/issues/4803

And dumps of this nature are kept in /data/crash just in case any one is wondering.

 

[cyberjock]

I'm not sure what the final result of the ticket will be.. but I'd say you should expect it to be one of the following:

1. Close to driver update and hope it works.
2. Close to "Don't use that hardware silly! That's not server grade."
3. Not closed for months or years.

Not sure how true #2 is. I've never heard of Solarflare until this post(or at least the brand has never caught my eye), so I tend to think they're some weird knock-off brand that is just a poor choice for a file server running FreeBSD/FreeNAS.

 

[aufalien]

They've OEM'd to HP for years. You'll also see em in high end compute clusters. Its not marketed to the masses for sure, mopre of e behind the scene company.

For now I'm doing individual addies and mounting some clients off one and other clients off the other. I was going to test under FreeBSD 9.2 and 10 but got held up. I'll post with my results soon though.

Forgot to add that they also seem to have the market share in equity trading platforms. They are a pretty serious company, don't let mainstream or lack there of fool you.

My view on popular, well Microsoft is an example of this. Are they the best? Well depends on what view you take but over all I'd say hardly in general.

 

[cyberjock]

Hmm.. is Solarflare a rebranded Intel or something? There's a possibilty it's a poor choice for a file server running FreeBSD/FreeNAS due to poor driver support. Many popular brands in Windows/Mac are crap on FreeBSD. Just look at SATA controllers or NIC brands for some excellent examples of stuff that works great in Windows but doesn't work at all(or works very poorly) in FreeBSD. ;)

What driver is it using? Does it show up in dmesg?

 

[aufalien]

Hmm.. is Solarflare a rebranded Intel or something? There's a possibilty it's a poor choice for a file server running FreeBSD/FreeNAS due to poor driver support. Many popular brands in Windows/Mac are crap on FreeBSD. Just look at SATA controllers or NIC brands for some excellent examples of stuff that works great in Windows but doesn't work at all(or works very poorly) in FreeBSD. ;)

What driver is it using? Does it show up in dmesg?

Its not a re-branded Intel, they roll there own and use FPGAs which are very fast. Driver support has been around for a while now in FreeBSD and use the SolarStorm driver;

sfxge0: <Solarflare SFC9000 family> port 0x5100-0x51ff mem 0xd1000000-0xd1ffffff,0xd2850000-0xd285ffff irq 27 at device 0.0 on pci4
sfxge0: Using MSI-X interrupts
sfxge0: Ethernet address: 00:0f:53:08:44:7c
sfxge0: Solarstorm SFN5162F SFP+ Server Adapter
sfxge1: <Solarflare SFC9000 family> port 0x5000-0x50ff mem 0xd0000000-0xd0ffffff,0xd2840000-0xd284ffff irq 30 at device 0.1 on pci4
sfxge1: Using MSI-X interrupts
sfxge1: Ethernet address: 00:0f:53:08:44:7d
sfxge1: Solarstorm SFN5162F SFP+ Server Adapter
sfxge0: promiscuous mode enabled

 

[cyberjock]

Wow... this is the only thread where sfxge is mentioned! I don't know what to say. The easy answer would be to buy an Intel NIC or something that is well known to work for FreeNAS.

One thing I will say, that may or may not help... I saw that you are using MTU=9000. Does it work with a standard MTU?

 

[aufalien]

Same issue regardless of frame size. The card does work in LACP mode, no packet errors etc... Its just during reboot or shutdown when core dumps. This causes a dirty start up and forces an FS check and fix on the USB boot device.

In terms of being the only thread were sfxge is mentioned, well most of the material on this forum is geared towards home use or light biz use. Now I'm not arguing the valid point of using mainstream fromage to ensure success. But I've always taken a diff approach; sure its growing pains but more so then not I've reaped benefits in the end and so will others due to my blazing trails of sorts.

 

[cyberjock]

Oh, I'm not saying your choice is bad. I think there's room for that discussion, but that's not the most ideal when you own the card and it costs more to replace it(and even more if it's not local). There is always a real cost of going with different hardware. It may be 2 minutes of your time and a card sitting on a shelf collecting dust, or it might be a flight around the world and a very expensive card that you have to buy with money you don't want to spend.

I'm just looking at the simplest solution to your problem.. and I think that is currently 'use a different NIC'. :(

 

[aufalien]

I'm here to report that Solarflare works properly in FreeNAS. A network dev on the FreeBSD forums fixed it and FreeNAS promptly applied the patch in 9.2.1.4.1.

Stoked!

I have to thank the FreeNAS dev team for getting this going. It was done very quickly!

 

[spjsdm]

HI,

I have installed a 10G Solarflare network card, but the drivers are not included in FreeNAS 11.1, but they do exist on FreeBSD 11.1.

Any chance of having them included for the next release.

Thanking you in anticipation
Stephen

Details below:

Entry in dmesg.boot

Code:

pci5: <network, ethernet> at device 0.0 (no driver attached)

pciconf -vl

Code:

none5@pci0:5:0:0:	   class=0x020000 card=0x020110b8 chip=0x07101924 rev=0x02 hdr=0x00
	vendor	 = 'Solarflare Communications'
	device	 = 'SFC4000 rev B [Solarstorm]'
	class	  = network
	subclass   = ethernet

lspci | grep Ethernet[cmd/]

Code:

05:00.0 Ethernet controller: Solarflare Communications SFC4000 rev B [Solarstorm] (rev 02)

 

[nightshade00013]

HI,

I have installed a 10G Solarflare network card, but the drivers are not included in FreeNAS 11.1, but they do exist on FreeBSD 11.1.

Any chance of having them included for the next release.

Thanking you in anticipation
Stephen

Details below:

Entry in dmesg.boot

Code:

pci5: <network, ethernet> at device 0.0 (no driver attached)

pciconf -vl

Code:

none5@pci0:5:0:0:	   class=0x020000 card=0x020110b8 chip=0x07101924 rev=0x02 hdr=0x00
	vendor	 = 'Solarflare Communications'
	device	 = 'SFC4000 rev B [Solarstorm]'
	class	  = network
	subclass   = ethernet

lspci | grep Ethernet[cmd/]

Code:

05:00.0 Ethernet controller: Solarflare Communications SFC4000 rev B [Solarstorm] (rev 02)

You would be better suited to create a new thread.

However the support baked in to FreeBSD is not for your SFC4000

The sfxge(4) driver supports all 10Gb Ethernet adapters based on Solarflare SFC9000 family controllers.

So your controller is not supported in FreeBSD. You will be better suited finding a different NIC.