Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.
10 Gig Networking Primer

10 Gig Networking Primer

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
15,851
What you do is you look for information about the seller.

The seller who has a name like shiningrainbow2021 and is selling x520 cards alongside sandals, electric foot massagers, and fashionable purses is a questionable seller who may be getting knockoff crap shipped from Asia by the containerful. They will have usually very positive feedback figures in like the five or six digits.

The seller who has lots of computer parts being sold by "recycleit" and has used Supermicro, Dell, HP chassis available in large quantity that appear to be of a 4-to-8-year vintage, where they've clearly taken a server and broken it up into parts, and are selling the chassis, the memory, the hard drives, the hard drive trays, the add-in cards, cables, maybe network switches and other data center miscellanea. These people are parts'ing up used data center gear, and the stuff is going to be the real deal. They typically have only been on eBay a year or two, don't have a lot of feedback, but the feedback that's there is often less than stellar. This is because those of us who buy professionally in the used market often don't leave feedback, we're just happy with what we get, but there are morons who buy stuff that they don't understand, it's perfectly good stuff but they are not able to use it. So you get this weird effect where they seem to have unreasonable feedback rates until you actually look at the feedback, where I typically roll through and go "moron ... moron ... idjit ... seriously?" I do take lucid, cogent, plausible, properly spelled feedback into account, especially where there is some sort of "DOA warranty" that isn't being honored, or stuff is shipped badly, because that is a red flag.

I'm sorry if that sounds cynical or insulting to eBay buyers, but quite frankly I have had pretty good luck with this sort of strategy. We buy lots of stuff used. Sometimes it doesn't work out, but vetting your sellers to be legitimate recyclers is one of the best ways to avoid fakes.
 

Riddick

Junior Member
Joined
Mar 4, 2020
Messages
14
Can someone help me?
I recently ordered the Intel Ethernet X710DA2 SVR with 2 Ubiquiti UF-MM-10G sfp+
Freenas told me the media type is not supported.

I'm not sure, can i use fiber optics with this card? if yes, which sfp+ i should buy? Maybe i just ordered the wrong.
 

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
15,851
Intel makes the Intel X710DA2, so one would expect Intel optics to work with it.

From Intel's tech support site:

https://www.intel.com/content/www/u...007045/network-and-i-o/ethernet-products.html

It says, in part,
  • Intel® Ethernet SFP+ SR Optics and Intel® Ethernet SFP+ LR Optics are the only 10-Gbps optical modules supported. You can purchase these modules separately.
  • You can't use other brands of SFP+ (10 Gbps) modules with these adapters.
  • Tested modules are listed in the Intel® Product Compatibility Tool.

  • Other brands of SFP+ optical modules don't work with the Intel® Ethernet Converged Network Adapter (Intel® Ethernet CNA) X710 Series. You must use the listed Intel® SFP+ SR or LR Optics.
The Product Compatibility Tool can be used to check for compatibility. I would also expect that Intel-coded generic optics from someplace like fs.com would work, as would used eBay optics, both of which should be substantially less expensive than new Intel optics.
 

Riddick

Junior Member
Joined
Mar 4, 2020
Messages
14
Intel makes the Intel X710DA2, so one would expect Intel optics to work with it.

From Intel's tech support site:

https://www.intel.com/content/www/u...007045/network-and-i-o/ethernet-products.html

It says, in part,
  • Intel® Ethernet SFP+ SR Optics and Intel® Ethernet SFP+ LR Optics are the only 10-Gbps optical modules supported. You can purchase these modules separately.
  • You can't use other brands of SFP+ (10 Gbps) modules with these adapters.
  • Tested modules are listed in the Intel® Product Compatibility Tool.

  • Other brands of SFP+ optical modules don't work with the Intel® Ethernet Converged Network Adapter (Intel® Ethernet CNA) X710 Series. You must use the listed Intel® SFP+ SR or LR Optics.
The Product Compatibility Tool can be used to check for compatibility. I would also expect that Intel-coded generic optics from someplace like fs.com would work, as would used eBay optics, both of which should be substantially less expensive than new Intel optics.

Thank you! Other question, if i use the Intel SFP+ and on the other side of the fiber the Ubiquiti 10g SFP+ should it work?
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,477
Thank you! Other question, if i use the Intel SFP+ and on the other side of the fiber the Ubiquiti 10g SFP+ should it work?
Yes. AFAIK, the vendor lock non-sense only applies to the transceiver. So as long as a "intel" transceiver is shoved into a "Intel" SFP+ slot on one end and a "Mikrotik" transceiver is shoved into a "Mikrotik" SFP+ slot on the other end, the connection should be fine. That's what I have here.

The only complication I have encountered is SFP+ transceivers not playing nice with SFP slots (which I mistakenly thought were SFP+). That was a dumb oversight on my part. In other words, do not assume that a 10GbE transceiver can work in a SFP slot (it won't) nor that a 10GbE transceiver in a SFP+ slot can "throttle down" to a 1GbE connection speed (many don't, and those that do, advertise it). I have some Intel transcievers here that can run at 1 or 10GbE, which is a nice-to-have.

Plenty of folk can sell you either a used genuine Intel transceiver or faked new ones that have a Intel vendor ID burned into them.
 
Last edited:

danb35

Wizened Sage
Joined
Aug 16, 2011
Messages
12,743
I headed over to STH to try to come up with a suitable mezzanine dual-10G NIC for my Proxmox hosts on nodes of a Dell PowerEdge C6220 II, and while I was there, ran across this:

It's looking like a single ICX6610 could replace my Dell X1052, my Unifi US-8-60W, and my two Mikrotik units. A number of other units are listed there as well, and there are what look like pretty good docs for flashing to the latest firmware and at least basic configuration. Look like some nice options for 10G (and even 40G) connectivity.
 

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
15,851
Yeah, the Brocade ICX stuff is good, some of it originally came from Foundry (hence the Iron moniker). Would buy it without a second thought, but be careful about licensing. You can probably find cheap licenses as this stuff is all EOL as far as I can tell.
 

danb35

Wizened Sage
Joined
Aug 16, 2011
Messages
12,743

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
15,851
Possibly not legal, but perhaps the licensing cops aren't going to beat down your door.

Actually I'm really tempted. One of my first gig switches was a Foundry FastIron Workgroup Switch 24+2G which is what we used to aggregate 100Mbps ethernets on Intel ISP1100's into two 1G fiber uplinks. It was a great switch. They started to feel the pressure in the mid-2000's and sold themselves off at some point, alas.

Now that some of the better name brand gear is becoming available, I'm a bit torn. I could really use higher port count 10G switches like the ICX6650 to replace several of the core switches here, most of which are 10G or 10/40. But the availability of cheap 25/100 stuff is probably just a handful of years away. I'm not sure that I mind being "limited" to 10/40 though given the vast choices of available gear, and really network utilization here doesn't tend to be super-heavy.
 

danb35

Wizened Sage
Joined
Aug 16, 2011
Messages
12,743
I expect the legality is more of a gray area than might be expected, but as this is well outside my specialty it probably wouldn't be wise for me to opine further. But if the gear isn't being made any more, nor are the licenses being sold, it doesn't seem there'd be a whole lot of reason for any extant company (I guess Ruckus is the current owner of the brand?) to pursue any recourse here.

BTW, I'd appreciate if you could give any input into my thread in OT where I pinged you--I think the question has come down to (1) whether these NICs can be made to do Ethernet, and (2) whether they'd allow splitting the QSFP+ into individual SFP+ lanes (thus allowing the use of a QSFP+-to-SFP+ adapter). The Intel 10G NICs with the proper mezzanine bracket appear to be unobtainium.
 

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
15,851
BTW, I'd appreciate if you could give any input into my thread in OT where I pinged you

You did?

I think the question has come down to (1) whether these NICs can be made to do Ethernet, and (2) whether they'd allow splitting the QSFP+ into individual SFP+ lanes (thus allowing the use of a QSFP+-to-SFP+ adapter).

I may not be finding the right search thingys for that particular card. Or it may just be crappy support docs.

If we look at a similar generation Mellanox card for Dell, we see

https://web.archive.org/web/2022021...etworkcard_MC[GQ]H29B-XCC_user_manual_1_2.pdf

These cards can usually be configured for either/or Infiniband or Ethernet. The specifics of the card types are a bit opaque to me as I have not been a Mellanox fan; I don't have any insight into this particular card. At that price point, it's the sort of thing that's not worth a lot of time looking into, and I would be tempted to just buy one and see what happens(*)

The ConnectX-2 cards have been slowly losing support in various OS's as they are very late in their lifecycle. Be certain your ${stuff} supports them. I noticed a post over on Level1Techs just so you don't screw yourself. I got hung by the ESXi deprecation of the Linux driver compatibility shim and now I have a bunch of Solarflare cards sitting around doing nothing. At some point I should probably hold a TrueNAS virtual garage sale.

Breakout cables should be a possibility if the driver in question supports it. However, there's examples of where that hasn't been true. It looks like the chipset supports it.

(*)This is how I usually get into trouble. ;-)
 

danb35

Wizened Sage
Joined
Aug 16, 2011
Messages
12,743
Yep, Saturday morning. Maybe notifications didn't work like I'd expected.
I may not be finding the right search thingys for that particular card.
That's validating--at least it isn't just me. And while the PDF you found isn't for exactly the card I'm looking at, it still seems encouraging.
I would be tempted to just buy one and see what happens(*)
That's pretty much where I was at anyway. Ordered one, I'll see what happens with it and hopefully order a few more once I can confirm it works.
very late in their lifecycle.
Doubtless that's why they, like the Chelsio S310 cards, are so cheap.
 

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
15,851
That's validating--at least it isn't just me.

No. This is a common problem with vendor gear. These bits are designed to be sold along with systems and to come from the factory all configured and deployed, so there may be minimal documentation or no documentation; the entirety of what features are actually supported may have existed only on one web page that described the card as an option for the system with a line that vaguely said "Infiniband/Ethernet support".

People wonder why I create a local cache of files I've found I need...
 
Top