Issue with X710 10GbE SFP+

[Ericloewe]

You won't find a 10GBase-T SFP+ module. They do not exist.

There aren't even any decent (price, quality, size) SFP+ to twisted pair adapters, which is going to be a real headache as 10GBase-T gains popularity (Xeon-D should help).

 

[pclausen]

I'll probably just go with FTP then.

Ok, so no 10GBaseT SFP+ modules, I guess than means I'll just have to roll the dice with the Intel E10G41AT2 then unless there is a know supported 10G Ethernet card with at least one 10GBase-T port? What I might do it try one of his cards first, but I'd hate to drive the 2+ hours to his house, dragging my 3 846 servers with me, only to find that it won't work.

 

[j_r0dd]

Just wanted to report back that the X710 is working fine for me so far on build 201506042008. Hopefully the driver kinks have been worked out.

 

[jgreco]

There aren't even any decent (price, quality, size) SFP+ to twisted pair adapters, which is going to be a real headache as 10GBase-T gains popularity (Xeon-D should help).

An SFP+ module for 10GBase-T probably just isn't going to happen, because the specification for SFP+ specifies a maximum of 1.5 watts for module power consumption (see Page 17), and 10GBase-T has taken a lot more than that so far (see Page 7). The other problem that you're fighting is that for 10GBase-T, the power consumption increases somewhat with distance. While there's a possibility that improvements in PHY technology will get us down into the sub-1-Watt range for power burned by the actual transceiver, the problem is that you're still sending a significant amount of power into the copper, which would need to be sourced by the SFP+ slot.

We'll be stuck with crappy media converters. There are several of those available, but you really don't want to use them unless you absolutely positively have to. Media converters are kinda like PoE power injector midspans or anything else that sits external to your switch and adds another bit of gear to your rack. An example would be http://estore.perle.com/smi-10gt-sfp.aspx - and just so we're clear, I'm not saying that any specific manufacturer's device is crappy, but rather that the entire concept of needing to add a translational device is crappy. I would hope that for $1200, the Perle gadget is awesome, and it appears to be, since it has management capabilities and supports "TACACS+, RADIUS, LDAP, Kerberos, NIS and RSA. Manage via SNMP, CLI- Telnet/SSH or HTTPS."

 

[Ericloewe]

An SFP+ module for 10GBase-T probably just isn't going to happen, because the specification for SFP+ specifies a maximum of 1.5 watts for module power consumption (see Page 17), and 10GBase-T has taken a lot more than that so far (see Page 7). The other problem that you're fighting is that for 10GBase-T, the power consumption increases somewhat with distance. While there's a possibility that improvements in PHY technology will get us down into the sub-1-Watt range for power burned by the actual transceiver, the problem is that you're still sending a significant amount of power into the copper, which would need to be sourced by the SFP+ slot.

We'll be stuck with crappy media converters. There are several of those available, but you really don't want to use them unless you absolutely positively have to. Media converters are kinda like PoE power injector midspans or anything else that sits external to your switch and adds another bit of gear to your rack. An example would be http://estore.perle.com/smi-10gt-sfp.aspx - and just so we're clear, I'm not saying that any specific manufacturer's device is crappy, but rather that the entire concept of needing to add a translational device is crappy. I would hope that for $1200, the Perle gadget is awesome, and it appears to be, since it has management capabilities and supports "TACACS+, RADIUS, LDAP, Kerberos, NIS and RSA. Manage via SNMP, CLI- Telnet/SSH or HTTPS."

Hey, post #6000

 

[jgreco]

Hey, post #6000

Yes, yes, yes, I know, I gotta get a life. Couldn't you have at least posted something fun to read!! >:-( <grinch />

 

[anodos]

Yes, yes, yes, I know, I gotta get a life. Couldn't you have at least posted something fun to read!! >:-( <grinch />

You still only have about 1/3 the posts of cyberjock, which you can take to mean that you have 3x more of a life than him. :D

 

[pclausen]

Looks like this one is fully supported according to Datakeeper in his built thread. So when the times comes, that's likely the one I'll go with.

125 posts! Whohoo. :D

 

[jgreco]

You still only have about 1/3 the posts of cyberjock, which you can take to mean that you have 3x more of a life than him. :D

Now, see, that, there, that was funny! That's what I was talkin' about!

 

[jgreco]

Looks like this one is fully supported according to Datakeeper in his built thread. So when the times comes, that's likely the one I'll go with.

125 posts! Whohoo. :D

Yeah, I haven't heard anything bad about the X540 except for the pre-9.3 driver issue. The Intels are apparently a bit below the iX-preferred Chelsios in terms of performance, which may be the other factor to consider.

 

[BenjaminHFJ]

Just wanted to report back that the X710 is working fine for me so far on build 201506042008. Hopefully the driver kinks have been worked out.

Hi everyone, after reading up a bit about the X710 issues with FreeNAS 9.3 I stumbled on this statement.

Since I have one of the (relatively) new Intel X710-DA2 SFP+ 10GbE cards coming in, I just wanted to know if anyone else has those up and running. Some of the initial driver issues / flow control bug reports in the bug tracking system have not changed for the last two months and are still not resolved at this point. Now (obviously AFTER ordering the parts) I am a bit worried about driver issues and so on.

It is not a "production" or mission critical system, but I have a pretty nice 8x4 TB Home NAS for all kinds of stuff (Xeon E3-1225 v3, 32 GB ECC) been running for the last 1,5 years. At this point and with the ever increasing sizes of media data the 110 MB/s bottleneck is becoming too annoying, so I decided to upgrade to 10 GbE. Also, with the new D-Link Switches the prices for 10 GbE have dropped dramatically.

Anyway, it would be nice to have some sort of "2nd confirmation" that the main driver issues of the Intel X710 cards are resolved. I'm otherwise pretty happy with FreeNAS and a lot of the manuals ans setup guides in this forum were extremely helpful back when I first put together the system - so some belated thanks.

 

[j_r0dd]

Hi everyone, after reading up a bit about the X710 issues with FreeNAS 9.3 I stumbled on this statement.

Since I have one of the (relatively) new Intel X710-DA2 SFP+ 10GbE cards coming in, I just wanted to know if anyone else has those up and running. Some of the initial driver issues / flow control bug reports in the bug tracking system have not changed for the last two months and are still not resolved at this point. Now (obviously AFTER ordering the parts) I am a bit worried about driver issues and so on.

It is not a "production" or mission critical system, but I have a pretty nice 8x4 TB Home NAS for all kinds of stuff (Xeon E3-1225 v3, 32 GB ECC) been running for the last 1,5 years. At this point and with the ever increasing sizes of media data the 110 MB/s bottleneck is becoming too annoying, so I decided to upgrade to 10 GbE. Also, with the new D-Link Switches the prices for 10 GbE have dropped dramatically.

Anyway, it would be nice to have some sort of "2nd confirmation" that the main driver issues of the Intel X710 cards are resolved. I'm otherwise pretty happy with FreeNAS and a lot of the manuals ans setup guides in this forum were extremely helpful back when I first put together the system - so some belated thanks.

It's working until a driver update breaks it again. Maybe the kinks have been worked out. I should probably give a confirmation that it's working in the bug report. I'm using an E10GSFPSR at both ends, for what it's worth. I bought 4 from here http://pages.ebay.com/link/?nav=item.view&alt=web&id=221771732396&globalID=EBAY-US at a decent price. Not 100% sure if they are legit. The ad says made in Malaysia and the units say made in China . Intel rep said they can't verify optics for some odd reason but they said that they do make them in China as well. Whatever, they work in the card and my switch detects it as an Intel optic. Maybe I'm just paranoid dealing with people on eBay.

 

[BenjaminHFJ]

Hi guys, just wanted to report back: The cards and cables are finally here.

First issue: Intel X710 cards are SFP+ Intel vendor locked in... means that the cards only work with a genuine Intel cable.
I was pretty pissed finding out about this issue. I didn't even remotely think about something like this when ordering the cables and after that I initially felt that this only applies to optical or active connection, not to f****ng direct attach cables.
Very long face, as it basically delayed the setup for another day.

Other than the cable issue, it's been working great. FreeNAS immediately recognized the new cards, link went up and everything was/is working fine (running the latest 201506292130-stable).

It appears that the initial problems seem to have been resolved.

 

[wpirobotbuilder]

run windows ;)

Short answer is NFS with sync disabled can pretty much max out 10GBe. For a non-critical seed type move it may be viable. ISCSI can do better than cifs but also has implications. You may be able to use a tool to open up multiple threads and get throughput up as well. Even FTP is lightweight and can fly. Multi-channel is coming but in the meantime Samba is a little rough when trying to fill a large pipe.

I would imagine that writing to an all-flash device would get you there with NFS sync, no? Assuming the device can support 10G write speeds.

 

[xnaron]

Can a chelsio sr-310e-cr sfp+ card talk to an intel x710 sfp+ card? Acknowledging the post above where the x710 cards can be locked in to intel sfp+ cables. So the scenario assumes the use of an x710 supported sfp+ cable. The reason I am curious is I have a s310e-cr in a freenas machine and would like to connect it to a vsphere esxi 6u1 server. The s310e-cr is not a supported card (I haven't tried to see if it would work though)

 

[Ericloewe]

Can a chelsio sr-310e-cr sfp+ card talk to an intel x710 sfp+ card? Acknowledging the post above where the x710 cards can be locked in to intel sfp+ cables. So the scenario assumes the use of an x710 supported sfp+ cable. The reason I am curious is I have a s310e-cr in a freenas machine and would like to connect it to a vsphere esxi 6u1 server. The s310e-cr is not a supported card (I haven't tried to see if it would work though)

All cards can talk to each other. It's direct-attach cables and SFP/SFP+ modules that may have vendor lock-in.

 

[rogerh]

All cards can talk to each other. It's direct-attach cables and SFP/SFP+ modules that may have vendor lock-in.

It may sound like a silly question (from someone who has never handled this stuff), but can you easily put different SFP or SFP+ modules at each end of a fibre, and does it work?

 

[xnaron]

All cards can talk to each other. It's direct-attach cables and SFP/SFP+ modules that may have vendor lock-in.

So the direct attach sfp/sfp+ cables have the lock in an not the card? BenjaminHFJ was saying his Intel x710 cards had a lock in for only intel sfp/sfp+ direct attach cables.

 

[Ericloewe]

So the direct attach sfp/sfp+ cables have the lock in an not the card? BenjaminHFJ was saying his Intel x710 cards had a lock in for only intel sfp/sfp+ direct attach cables.

If you connect two cards via fiber, any supported configuration will work between vendors.
Unfortunately, you will need the manufacturer-recommended SFP+ modules. Since twinax direct attach cables are passive things, you often end up requiring the same vendor's cards on both ends. The cable itself is just a cable, but an Intel card might not be capable/willing to talk to a Chelsio card over direct-attach.

 

[Ericloewe]

It may sound like a silly question (from someone who has never handled this stuff), but can you easily put different SFP or SFP+ modules at each end of a fibre, and does it work?

Yes, as long as they're using the same technology (multimode versus singlemode fiber, SR versus LR optics, ...).
Not sure about SFP to SFP+, but I'd imagine the fiber layer also allows for backward compatibility, like twisted pair Ethernet.