How to use 1Gb network for some machines, 40Gb for others

[friolator]

This is probably a basic networking question, but I'm finding myself kind of lost in the admin interface, and the manual isn't really addressing this as far as I can tell. I couldn't find anything in the forum either, but I may not be searching for it correctly...

What we have is a 1Gb network that all machines in the office are connected to, and and that is connected to a router, so each of those machines also have internet access. It's a mix of Windows, Mac, Linux and FreeBSD machines. We primarily use SMB/CIFS for point-to-point file sharing. Of the dozen or so PCs in the office, all of them are on this network. We use a generic windows workgroup for sharing, no domain controller.

In addition, we have a 40GbE switch, which our FreeNAS box and (for now) one Windows machine are connected to. Soon, 2 more Window machines and one Linux box will be added.

Performance on the 40GbE network is the most important thing here. We don't want to have a 1Gb bottleneck, because we'll be moving terabytes of data around on the 40Gb network, but right now, our FreeNAS box is only communicating with those machines on the gigabit network. The 40GbE switch is physically isolated from the 1Gb switch, and we don't want to be using the high performance network for things like internet traffic, so I want it to stay separate. However, some machines (such as the iMac I'm typing this on) will need to be able to access the shares on the FreeNAS box, and those machines will never have 40 or even 10Gb interfaces.

So all that background to ask this: How do we go about configuring FreeNAS so that:

1) Older machines with only 1Gb interfaces can access the FreeNAS box via SMB/CIFS
2) Some machines remain on the 1Gb network for general internet access, but always access the FreeNAS box via the 40Gb network. Again, via SMB/CIFS

I'm looking for a step by step guide if there is one. I've been googling all afternoon but I'm coming up short.

Thanks!

 

[Mlovelace]

You need to use separate subnets for the different freeNAS interfaces, and it would be best to use separate vlans for those subnets as well. I don't know if your networking gear supports vlans or vlan tagging but if it does, bob's your uncle.

I'm looking for a step by step guide if there is one. I've been googling all afternoon but I'm coming up short.

As for a step-by-step instructions there are probably walk throughs out there, but it's not something I'm going to write up on a forum post.

 

[depasseg]

The 40GbE switch is physically isolated from the 1Gb switch, and we don't want to be using the high performance network for things like internet traffic, so I want it to stay separate.

You might want to rethink this. If you keep this separated configuration, then you will be forced into managing 2 ip spaces, which can be a pain. If instead, you connect your 40GbE switch to your 1 GbE switch then you can keep a single ip address space, and just move machines to the faster switch when ready and needed. As for internet browsing affecting your 40GbE switch, I wouldn't worry unless I was connected to the 100GbE Internet2 network.

 

[friolator]

You might want to rethink this. If you keep this separated configuration, then you will be forced into managing 2 ip spaces, which can be a pain. If instead, you connect your 40GbE switch to your 1 GbE switch then you can keep a single ip address space, and just move machines to the faster switch when ready and needed. As for internet browsing affecting your 40GbE switch, I wouldn't worry unless I was connected to the 100GbE Internet2 network.

So if we were to do this, it seems we'd need a new gigabit switch, because our current Cisco switch doesn't have 10GbE uplink ports. I would rather not buy a new switch if I don't have to, because our 1Gb network is very stable and has been working well on that Cisco switch for the past 5 years. Can you (or anyone) recommend a device to bridge the two switches?

From our IBM 40GbE switch, we can use breakout cables to get 4x 10GbE connections on one 40Gb port. So I think all we'd need is a small device that has a few 1Gb RJ45 ports and a passive SFP+ port for the 10Gbe uplink. Does such a thing exist? I can only find switches with 10Gb uplinks in larger, more expensive models. Someone must make something like this, no?

 

[depasseg]

I didn't realize your 40GbE couldn't get an SFP that would handle 1GbE. That does make things a bit more difficult. If you look in the 10G networking thread (or search the forums) I think there is a Dell 54xx switch that has 24x1GbE and 2x10GbE ports for ~$250. There was a post about this just the other day, but I can't find it ATM.

 

[danb35]

That would be the Dell PowerConnect 5524. You can pretty easily find them on eBay for $250-300, and might be able to do better than that if you're lucky. You can also consider the Dell X1052, which is a newer model, 48 ports of GbE and 4 SFP+. I think I've seen that one around $500 on eBay, but haven't looked too closely. The 5524 is discussed in the 10G primer here in the Networking forum.

 

[depasseg]

Thanks @danb35, I thought you posted about this recently.

 

[Stux]

Net gear also have a range of smaller gb switches with 10gbe uplinks

 

[Stux]

Or you could just plug one of the 10gbe cables into the gbit switch. If it's 10gbaseT

 

[danb35]

Actually, turns out the X1052 is only $629 new from Dell.

 

[friolator]

Or you could just plug one of the 10gbe cables into the gbit switch. If it's 10gbaseT

Not quite so simple: One switch is 40Gb and the other is 1Gb. The 1Gb switch has 1Gb miniGBIC ports. But the 40 can only use 40->10GB copper DAC breakout cables. So I don't think I'd be able to connect them. Unless I'm missing something. Most of the switches I'm seeing with 10Gb uplink ports are fibre optic.

What I need is a 24+ port gigabit switch with a copper 10Gb uplink port that takes a passive DAC cable.

...or just live with two subnets and use IP addresses/mapped network drives for the 40Gb. But I'd prefer it to be more seamless from the end user perspective.

 

[Stux]

What I need is a 24+ port gigabit switch with a copper 10Gb uplink port that takes a passive DAC cable.

http://www.netgear.com.au/business/...le-smart-switch-10g-uplinks.aspx#tab-overview

2x Sfp+ uplinks + 2 x 10gbaset uplinks. 24-48 ports

So it can also act as a bridge between your Sfp+ network and 10gbaset

 

[friolator]

Thanks. But are these any good? My experience with Netgear stuff in the past was so-so - the old 10/100s were really good, but the first gb switch we bought was a netgear and had a tendency to burn out ports. Like a lot of them.

Our current Cisco switch (an SG300-28) has been absolutely rock solid for years.

 

[depasseg]

10GB copper DAC breakout cable

Most 10GbE switches are SFP+ which will accommodate this connector. The fiber is only because that type of SFP+ is inserted.

 

[CookiesLikeWhoa]

Why spend all this money on Another switch when you can just use a seperate subnet for the 40gbe needs? Seems simple enough


Sent from my iPhone using Tapatalk

 

[Mlovelace]

Why spend all this money on Another switch when you can just use a seperate subnet for the 40gbe needs? Seems simple enough


Sent from my iPhone using Tapatalk

I'm pretty sure someone else suggested that too :D

 

[Stux]

For simplicity ;)

 

[Mlovelace]

For simplicity ;)

If you have 1G ports and 40G ports on the same subnet you're still connecting by IP to make sure you're connecting to the correct interface. I don't see how adding another piece of hardware is "simplicity"

 

[friolator]

As of right now we have it set up as a separate subnet. I would prefer that, from the end user's perspective, everything appears to be a single network. That is - shared volumes on the FreeNAS box would show up along side other machine-to-machine shares in the network browser. Right now we can set up mapped drives using the FreeNAS IP address, but I'd prefer human-readable, browsable shares.

An extra piece of hardware is fine with me, if it works reliably. If we were to get something like a 24 port (or larger) gigabit switch with 10G uplinks, we'd just replace our Cisco, so the hardware footprint would be the same as it is now.

The simplicity I'm after is on the end-user side. I don't mind an extra bit of hardware if we need to go that route though.

 

[CookiesLikeWhoa]

Realistically you're going to want individual subnets for security and simplicity sake. Shares will remain the same, literally everything will look the same from the users perspective it's just how they connect to the NAS.

If you want browsable shares cifs can provide those for you with ease


Sent from my iPhone using Tapatalk