Eliminate my switch?

[300cpilot]

Here is the equipment:
3 Dell R610's
1 HP380p gen8 + (2) disktrays 56Tb total drive space
all use single 10 gig nic's.

Currently they are all running happily through a cisco switch, but it only has 4 10 gig ports. I need to add 2 more R610's. Each R610 is running 35-40 vm's. The HP380 (Freenas) is the only storage via iscsi. Hypervisor on the R610's is Xenserver 7.2.

What I would like to know is there a way to put 3 dual 10gig nic's in the HP380 and have it act as a switch using iptables or bsd equivalent? I have very low knowledge of BSD and am not sure if it will act the same way as Linux? BSD isn't Linux, right? The reason for this is that it saves me from buying the new switch, until later and it will save us space and power charges at the colo.

Thank you for your insights!

 

[Elliot Dierksen]

FreeBSD is certainly capable of bridging. https://www.freebsd.org/doc/handbook/network-bridging.html

The question in my mind is if the DL380 G8 is up to having 6 10G ports. I don't see a limitation mentioned in the quick specs https://h20195.www2.hpe.com/v2/getdocument.aspx?docname=c04123238 but it does say you need 2G of server memory per 10G NIC.

 

[300cpilot]

Thanks for the link, going through it now. The HP has 64 gigs in it, and currently only has a one dual 10 gig in it. Looking at matching the HP nic that is in it.
Will the bridging config survive a reboot? Just watched the video's on advanced network setup and they mention that anything not done through the Freenas interface, will not survive a reboot.

 

[Elliot Dierksen]

I have seen some references to "epairs" in other forum posts surrounding virtual machines, so it should be roughly the same thing. I have not had occasion to do it since I have plenty of 10G switch ports. If nothing else, you could assign the bridge info in startup scripts. I found a link https://forums.freenas.org/index.ph...-without-10gb-switch.25259/page-3#post-161894 that talks about doing some of that. It has to be done in the GUI, or in a startup script to survive a reboot. You would assign the IP address to the bridge interface, not the physical interfaces.

 

[Elliot Dierksen]

Again, I haven't done it but it does appear that this can be done by creating a link aggregation and tuning a few parameters. https://www.ixsystems.com/documentation/freenas/11.2/network.html?highlight=bridge

 

[kdragon75]

You can add all the 10gb interfaces to a bridge and that's it. The bridge is essentially a "switch". The performance will be TERRIBLE but it's technically possible. If you need the ports, teh NetApp CN1610 can be found for 100-150 USD and has 16 10gbe sfp+ ports. I sick with Cisco DAC cables and it has been nothing but perfect. VLANs are fully supported too. With 8 ports active I draw just over 100 watts and it's not insanely loud like the Cisco Nexus 5k stuff.

 

[300cpilot]

It isn't the cost of the switch to buy, it is the rental of the colo space that we were looking to reduce. I don't have to listen to it so a Nexus would be on the horizon. It is just what we use everywhere else. By reducing 1u of space we could go from two cabinets to one. Which is about $800 a month savings. I will be trying this as a bridge as soon I can. Curious if the speed will be effected?

 

[kdragon75]

Curious if the speed will be effected?

It will. Feel free to try it and benchmark. I would be curious to see the results.