Any cost efficient methods to get greater than 1Gbps transfers

[Black Ninja]

You probably don't need OM3, but my suggestion is to shoot for OM2.

What kind of card and switch are you planning to get?

I'll get the OM3. I read this from http://www.lanshack.com:

"The 50/125 µm core size comes in three different classifications (OM2, OM3 and OM4). Please note that OM3 is usually just referred to as 10GIG since it is generally the best choice for 10 Gigabit Ethernet over Multimode fiber and was designed specifically for that purpose (unless you need the extra distance provided by OM4).

OM3 and OM4 fibers will support upcoming 40 and 100 Gb/s speeds. Only OM3 and OM4 Multimode fibers are included in the IEEE 802.3ba standard for next-generation 40 and 100 Gb/s Ethernet. OM1 and OM2 fibers are not supported media types for these applications."

Those you point's make it OM3 if go the fiber road. I don't know what card is best, I was hoping to find out from you. Let me tell you my setup maybe you can suugest best for my case:

I have 5-6 workstations on the second floor and servers in the basement with switch there. In order to have non-blocking from second floor to basement I ran 6 cables (cat6) from each PC to the switch in that basement so no daisy chaining switches.

I was hoping I can upgrade the switch in the basemen with 10gb capabilities so I can use existing cat6 between second floor and basement. Any server already in the basement is next to the switch so I can ruining any sort of connection to it, but wanted to have non-blocking 5x1gb client and 1x10gb workstation from second floor without running fiber in the walls ?

 

[Black Ninja]

Yes. Buy two used Intel 10Gb LAN cards on ebay and direct connect the two. I did this for $325 including the networking cable about 16 months ago. Its even cheaper now.

Perhaps I am repeating what the other user is asking , but since it's not answered I hope this will underline the importance of the question:
Can you be more specific on the hardware needed for this ? Card, modules or optics and cable ?

Thanks in advance

 

[cyberjock]

I'm not at home, and won't be for a while, so I can't be more specific. :(

I have posted it somewhere though, with links to exactly what I bought.

 

[Black Ninja]

I'm not at home, and won't be for a while, so I can't be more specific. :(

I have posted it somewhere though, with links to exactly what I bought.

I am desperately trying to find it. Let me know if you did.Thanks:)

 

[Darren Myers]

I have info for the Chelsio cards (fiber) : 10Gb cards , the modules you can get the seller down to $25 each (only need 2, 1 per card) , and the fiber , those are ALL the parts im using other then the XFP's for my 10Gb switch. I am also using the 2M cables because they're very short runs.

 

[Black Ninja]

I have info for the Chelsio cards (fiber) : 10Gb cards , the modules you can get the seller down to $25 each (only need 2, 1 per card) , and the fiber , those are ALL the parts im using other then the XFP's for my 10Gb switch. I am also using the 2M cables because they're very short runs.

Thanks a lot this is what I was looking for. How good is it particular Chelsio card , I heard few time in a forum that chelsio card are the best supporter card for Freenas , what is your opinion. I buid my nas using the most recommended hardware and everything work flawlessly I just need a 10gb that is really proven to work best ? Fiber or cooper al long as is the most reliable solution ? What switch you have ?

 

[Darren Myers]

I picked up a old 10Gb switch, its ALL 10Gb the only copper is for management. It's a switch from 2006-2007 but it does one hell of a good job for what i need. I picked up the Chelsio cards due to price and the support that FreeNAS offers for it, i also use the SFP+'s recommended by Chelsio and branded Chelsio; so far everything has worked flawlessly at first i had my servers direct connected until i added a SAN....2 ESXi servers, and my FreeNAS server...then i realized all the daisy chaining was getting out of hand and not wanting to out of pocket 2000$-6000$ for a Cisco N5k switch...i choose a route that got me a switch for 400$ granted it is indeed older...it does an amazing job

 

[Black Ninja]

I picked up a old 10Gb switch, its ALL 10Gb the only copper is for management. It's a switch from 2006-2007 but it does one hell of a good job for what i need. I picked up the Chelsio cards due to price and the support that FreeNAS offers for it, i also use the SFP+'s recommended by Chelsio and branded Chelsio; so far everything has worked flawlessly at first i had my servers direct connected until i added a SAN....2 ESXi servers, and my FreeNAS server...then i realized all the daisy chaining was getting out of hand and not wanting to out of pocket 2000$-6000$ for a Cisco N5k switch...i choose a route that got me a switch for 400$ granted it is indeed older...it does an amazing job

I was really hoping to get a specific model I can search for and buy it. I can't search in ebay for "old switch for good job" You see my point ? I am on a market for switch and need a exact model to help my search , you know what i mean ?

 

[Darren Myers]

I was really hoping to get a specific model I can search for and buy it. I can't search in ebay for "old switch for good job" You see my point ? I am on a market for switch and need a exact model to help my search , you know what i mean ?

It took me a full day of research, and some tinkering with google to yield the results i wanted....i even contacted Fujistsu about compatible XFP's and they were almost completely useless. Here is the 10Gb swtich and these are the XFP's that i have verified working, the biggest issue i had was getting consoled into the switch as it requires a NULL modem cable,what a pain in the ass i searched through 4 big boxes of cables at my datacenter to find it at the bottom of the last box.

 

[Black Ninja]

It took me a full day of research, and some tinkering with google to yield the results i wanted....i even contacted Fujistsu about compatible XFP's and they were almost completely useless. Here is the 10Gb swtich and these are the XFP's that i have verified working, the biggest issue i had was getting consoled into the switch as it requires a NULL modem cable,what a pain in the ass i searched through 4 big boxes of cables at my datacenter to find it at the bottom of the last box.

That sound really tough. You really wanted to make this switch to work. I guess you needed all 10gb ports ? Otherwise I would wonder why not get a 1gb switch with 4x10gb SPF+ uplinks. I like the idea of putting the nas on 10gb uplink port and then hope 10 clients on 1gb port can suck data from it simultaneously without the need of stupid LACP struggle.

 

[Darren Myers]

I really wanted a 10Gb backbone for my data, pretty much all my servers communicate through the 10Gb network now, and i'll be adding my Mac to that list this week. I thought about a switch with 10Gb uplinks but some of the uplinks are for switch to switch only and don't work how i'd want. So i figure i'd do the research, see if it will do what i want,and sure enough it does what i want and its perfect for my use case.

 

[Black Ninja]

I really wanted a 10Gb backbone for my data, pretty much all my servers communicate through the 10Gb network now, and i'll be adding my Mac to that list this week. I thought about a switch with 10Gb uplinks but some of the uplinks are for switch to switch only and don't work how i'd want. So i figure i'd do the research, see if it will do what i want,and sure enough it does what i want and its perfect for my use case.

Well from what i know those who are for switch to switch are stack able ports , I was talking about that one you can connect workstation to it. Then HOW you connect your 10gb network to 1gb network ?

 

[Darren Myers]

My 10Gb network doesnt interface with my 1Gb LAN other then the management port, my 10Gb stuff is all self contained, on its own IP range and subnet and has no internet access. The servers access each other through 10Gb, and if something is needed outside the 10Gb switch then it uses the 1Gb. Such as me watching plex, that comes from the 1Gb LAN, if my ESXi servers need there VM's they use the 10Gb backbone, same goes for if the VM's are accessing FreeNAS its all through 10Gb

 

[Black Ninja]

My 10Gb network doesnt interface with my 1Gb LAN other then the management port, my 10Gb stuff is all self contained, on its own IP range and subnet and has no internet access. The servers access each other through 10Gb, and if something is needed outside the 10Gb switch then it uses the 1Gb. Such as me watching plex, that comes from the 1Gb LAN, if my ESXi servers need there VM's they use the 10Gb backbone, same goes for if the VM's are accessing FreeNAS its all through 10Gb

I am trying to imagine it: You must have 2 switches ? What is connected to 1gb switch ? Is your freenas on 2 separate subnets 10gb and 1gb on build in port ?

 

[Darren Myers]

2 separate switched 2 different IP ranges and subnets. The 1Gb traffic has access to the Internet as well as every pc in my network, access points, and some stuff accessible outside the network. The 10Gb switch is purely data between my servers, VM's and SAN. My 2 ESXi servers use the 10Gb to communicate to the SAN for VM storage, access the file server (FreeNAS), and for various VMs to communicate with each other. So each VM has 2 NICs one that has Internet on the 1Gb and the other NIC that is strictly data on the 10Gb side. 90% of my traffic is done through the 10Gb because that's what is easiest for me and moving data, the other 10% is done over the 1Gb and includes streaming plex internally or externally, moving files from my laptop or my gfs laptop to FreeNAS, or cloud backup. But yes my FreeNAS server is on 2 different subnets and IP ranges

 

[Black Ninja]

2 separate switched 2 different IP ranges and subnets. The 1Gb traffic has access to the Internet as well as every pc in my network, access points, and some stuff accessible outside the network. The 10Gb switch is purely data between my servers, VM's and SAN. My 2 ESXi servers use the 10Gb to communicate to the SAN for VM storage, access the file server (FreeNAS), and for various VMs to communicate with each other. So each VM has 2 NICs one that has Internet on the 1Gb and the other NIC that is strictly data on the 10Gb side. 90% of my traffic is done through the 10Gb because that's what is easiest for me and moving data, the other 10% is done over the 1Gb and includes streaming plex internally or externally, moving files from my laptop or my gfs laptop to FreeNAS, or cloud backup. But yes my FreeNAS server is on 2 different subnets and IP ranges

So let's say your workstation is on 10gb and 1gb networks, when you want to access you freenas I suppose you that to happen on 10gb subnet not chose the 1gb path ? How you define that ?

 

[Darren Myers]

Let's say VM called "VMServer1" needs access to FreeNAS I would simply map the drive over the 10Gb IP which is 172.0.0.x. Now I do not have a DHCP server on the 10Gb switch so all the IPs are staticly assigned by me, if I need to access FreeNAS over the 1Gb I just put in either \\freenas or 10.1.10.x and there it is. Atm my MacBook isn't on 10Gb for obviously reasons (it being a laptop so no 10Gb card) so when I need a share from FreeNAS I point my MacBook to 10.1.10.x, now once I get my Mac Pro (tower) fired up with the 10Gb card and connected to the 10Gb switch I'll put in the network path 172.0.0.x and away I go accessing FreeNAS via 10Gb and any of my virtual machines via a 10Gb link, and even my SAN if needed.

 

[Black Ninja]

Let's say VM called "VMServer1" needs access to FreeNAS I would simply map the drive over the 10Gb IP which is 172.0.0.x. Now I do not have a DHCP server on the 10Gb switch so all the IPs are staticly assigned by me, if I need to access FreeNAS over the 1Gb I just put in either \\freenas or 10.1.10.x and there it is. Atm my MacBook isn't on 10Gb for obviously reasons (it being a laptop so no 10Gb card) so when I need a share from FreeNAS I point my MacBook to 10.1.10.x, now once I get my Mac Pro (tower) fired up with the 10Gb card and connected to the 10Gb switch I'll put in the network path 172.0.0.x and away I go accessing FreeNAS via 10Gb and any of my virtual machines via a 10Gb link, and even my SAN if needed.

I see , you are accessing it by ip on different subnets.
I tried something like that on dual 1gb links on my freenas on the same subnet same switch and my dhcp gave freenas 2 ip's but when I tried accesing freenas from 2 different pcs on each freenas ip still end up using one link instead of both simulations. I guess has something to to with them being on same subnet , I never really figure out. The idea was not using LACP , to make backup machine use 1 link to freenas while workstation use other simultaneously.

 

[Darren Myers]

Yes i access FreeNAS via 2 different IP addresses and subnets, i dont think LACP will do what you want....i've read alot of posts and it just doesnt seem to do what most people want.

 

[Black Ninja]

Yes i access FreeNAS via 2 different IP addresses and subnets, i dont think LACP will do what you want....i've read alot of posts and it just doesnt seem to do what most people want.

It will not gives a 2gb speed for a single client, that's already understood. But it will not always balance 2 client by utilizing both links , it does balance sometime wheever it wants, mostly not when needed. You are right i don't want lacp. It is setup as lacp but i'll change it, as soon as i can come up but better configuration.

I am trying to understand if 2 subnets can exist on the same switch(both 1gb nics) ? Does it need vlan separation to work ?