Network Setup Help

[Fastline]

Hey guys,

I'll soon setup the network for my server and the machines i use. Here's a quick summary for it

Server: 25GbE SFP28 (Intel XXV710) > 25G SFP28 module > LC > Mikrotik CRS518-16XS-2XQ-RM (Port 1)
Client A: 10GbE SFP+ (Intel X710) > 10G SFP+ module > LC > Mikrotik CRS518-16XS-2XQ-RM (Port 2)
Client B: 10GbE SFP+ (Intel X710) > 10G SFP+ module > LC > Mikrotik CRS518-16XS-2XQ-RM (Port 3)

Will this all work correctly? I'm concerned about the compatibility. Can i use like this (interconnecting 10G and 25G) or both the switch end and NIC end has to be the same module must have the same wavelength and speed? Of course, the client end will have a compatible 10G SFP+ module and the server will have 25G SFP28 module. I read that SFP+ and SFP28 have the same form factor and pinouts. Let me know, you guys are the expert here ;)

 

[Fastline]

Anyone?

 

[Arwen]

I can't answer compatibility with either XXV710 or X710, don't know.

However, having computers connect with different speeds is exactly what a network switch does.

Obviously you have to use the appropriate SFP+ / SFP28 modules in each link. I can't tell you if the Intel cards or Mikrotik switch have special requirements, (like some Cisco products), for specific branded SFP+ or SFP28.

 

[Fastline]

I can't answer compatibility with either XXV710 or X710, don't know.

Hmm

However, having computers connect with different speeds is exactly what a network switch does.

Yes, yes. I think it will work as the pinout for the SFP28 and SFP+ are same and the LC cable can do that much of speed. Although, not 100% sure that's why the thread. xD

Obviously you have to use the appropriate SFP+ / SFP28 modules in each link.

Of course of course, that's the requirement.

I can't tell you if the Intel cards or Mikrotik switch have special requirements, (like some Cisco products), for specific branded SFP+ or SFP28

Umm, I'm not sure about that either. HPE and Dell require their own modules for the link. But as far as I'm aware, Mikrotik doesn't have such special requirements.

Looks like need help of some network expert here.

 

[Patrick M. Hausen]

You can buy vendor branded SFP+ modules from fs.com and I am confident they will assist you in picking the right ones.

 

[Fastline]

You can buy vendor branded SFP+ modules from fs.com and I am confident they will assist you in picking the right ones.

Yes, that's what i'm doing right now. xD

But can i interconnect 25G and 10G with the help of a switch? The switch end runs 25G and the client side runs 10G. Provided that each NIC will have a compatible module installed and the LC fiber.

 

[Patrick M. Hausen]

I have never had gear past 10G ... ask fs.com support?

 

[Fastline]

I have never had gear past 10G ... ask fs.com support?

Yes, i'm with them. Thank you!

 

[danb35]

ask fs.com support?

I'd think the better option would be Mikrotik support, since it's their switch. But I'd be very surprised if it didn't work--that's literally what switches do, all the way back to the bad old days of "fast Ethernet".

 

[Fastline]

I'd think the better option would be Mikrotik support, since it's their switch. But I'd be very surprised if it didn't work--that's literally what switches do, all the way back to the bad old days of "fast Ethernet".

Have mailed them too. Awaiting on their response. Yes, i also think it should cause the same is written in the 10G primer resource.

 

[Arwen]

Ah, I have just looked at the switch specifications, and it has 16 x 25Gbit/ps ports AND 2 x 100Gbit/ps ports. (Plus a 10/100Mbit/ps port, probably suitable for management purposes.)

Originally I assumed it would have been a 10Gbit/ps switch with a some uplink ports at 25Gbit/ps. (And we all know what assumptions bring .

Further work with the switch vendor Mikrotik and the SFP+ & SFP28 vendor FS.com is suggested. However, the switch vendor's User Manual for CRS518-16XS-2XQ-RM has this;

* 100G QSFP28 supported speeds: 1x 100G; 1x 40G; 4x 25G; 4x 10G; 4x 1G. Not supported: 2x 50G; 1x 50G, 2x 40G
* 25G SFP28 supported speeds: 1x 25G; 1x 10G; 1x 1G. Not supported: 1x 5G; 1x 2.5G

Seems pretty definitive. And you don't seem to need any adapter octopus cables, like for either of the 2 x QSFP28 ports, for 4 x 25Gbits/ps or 4 x 10Gbits/ps. Though those octopus cables do get you another 4 or 8 10Gbits/ps or 25Gbits/ps.

Based on this, I would definitely consider buying Mikrotik 10Gbits/ps module for that switch. That way the switch vendor is on the hook for compatibility on of the fiber comms, which MUST be normal 10Gbits/ps light. (But, you do need same light frequency and distance on both sides, as well as either Multi-Mode fiber or Single Mode fiber support.)

 

[danb35]

But, you do need same light frequency and distance on both sides, as well as either Multi-Mode fiber or Single Mode fiber support.

...and just to be clear on this (Arwen, I know you know this, but I'm not sure OP does): you need to have the same wavelength and everything on a given connection. But it's perfectly valid to have one NIC with a SR optic and OM4 fiber connecting to a SR optic in the switch, and the next NIC with a LR optic and 300m of single-mode fiber connecting to a LR optic in the switch.

 

[Arwen]

What @danb35 said, better than me.

(Hey, I have an excuse, I am sick today... so a bit fuzzy headed.)

 

[Fastline]

Ah, I have just looked at the switch specifications, and it has 16 x 25Gbit/ps ports AND 2 x 100Gbit/ps ports. (Plus a 10/100Mbit/ps port, probably suitable for management purposes.)

Originally I assumed it would have been a 10Gbit/ps switch with a some uplink ports at 25Gbit/ps. (And we all know what assumptions bring .

Further work with the switch vendor Mikrotik and the SFP+ & SFP28 vendor FS.com is suggested. However, the switch vendor's User Manual for CRS518-16XS-2XQ-RM has this;

Yes, i checked the manual today and i can confirm the same. Its sad that only one port does full 100G and the other can be only splitter.

Seems pretty definitive. And you don't seem to need any adapter octopus cables, like for either of the 2 x QSFP28 ports, for 4 x 25Gbits/ps or 4 x 10Gbits/ps. Though those octopus cables do get you another 4 or 8 10Gbits/ps or 25Gbits/ps.

Umm, what are adapter octopus cables here?

But, you do need same light frequency and distance on both sides, as well as either Multi-Mode fiber or Single Mode fiber support

Hmm. I knew this but was not sure. What does the same light frequency and distance mean here?

 

[Arwen]

Yes, i checked the manual today and i can confirm the same. Its sad that only one port does full 100G and the other can be only splitter.

I am not sure what you mean by this. From what little I read, both 100Gbits/ps ports support both 100Gbits/ps and various lower speeds.

Umm, what are adapter octopus cables here?

It's something like this;

XQ+BC0003-XS+ | MikroTik

A QSFP28 to 4x SFP28 break-out cable. Allows connecting multiple 25 Gigabit devices to your CCR2216/CRS504 devices.

mikrotik.com

Hmm. I knew this but was not sure. What does the same light frequency and distance mean here?

@danb35 said it better. You have to use the same type of fiber modules on each end of a single link. (Different links can have a different pair of modules.) So, if you use SR, (Short Range), multi-mode fiber, you need modules at each end that support it.

I can't explain it any better right now.

 

[Fastline]

...and just to be clear on this (Arwen, I know you know this, but I'm not sure OP does

Well, while researching on this topic, i got to know this.

you need to have the same wavelength and everything on a given connection.

What does the same wavelength mean here?

But it's perfectly valid to have one NIC with a SR optic and OM4 fiber connecting to a SR optic in the switch, and the next NIC with a LR optic and 300m of single-mode fiber connecting to a LR optic in the switch.

Cool cool!

 

[Fastline]

What @danb35 said, better than me.

(Hey, I have an excuse, I am sick today... so a bit fuzzy headed.)

Get some rest. No rush at all :)

 

[danb35]

What does the same wavelength mean here?

Same wavelength of light, usually indicated in nanometers or nm. It'll be listed as part of the specs for the module. I think that SR optics (which would go up to 300 meters of cable length) are always 850nm; at least I haven't seen any that are different. LR optics could vary.

 

[Fastline]

I am not sure what you mean by this. From what little I read, both 100Gbits/ps ports support both 100Gbits/ps and various lower speeds.

Oh, i was confused. It's all good now. Yes, both the ports support 100G. :)

It's something like this;

Is this AOC?

@danb35 said it better. You have to use the same type of fiber modules on each end of a single link. (Different links can have a different pair of modules.) So, if you use SR, (Short Range), multi-mode fiber, you need modules at each end that support it.

Umm, do you guys mean both the end points should have same kind of module for e.g. SR/LR optics?

 

[Fastline]

Same wavelength of light, usually indicated in nanometers or nm. It'll be listed as part of the specs for the module. I think that SR optics (which would go up to 300 meters of cable length) are always 850nm; at least I haven't seen any that are different. LR optics could vary.

So that basically means both the NIC and switch end should have the SR/LR optics right? Is that what you mean?