MPIO Question

[KPc]

Hi
My company just purchase 12 NAS loaded with FreeNAS
I have been given the task to configure them

Configuring the nas is not hard i have work with storage from different vendos ( EMC .. NETAPP, etc )

We only purchase the NAS to do 2 simple task . Local Vranger BACKUPS via CIFS and ISCSI LUNs to one Virtual machine .
We have one ESX Host with a few VMs , We will configure one VM to used to mapped 2 LUNs , one used to save WDS Images and the Other to save WSUS database
VM is configured to support MPIO .

My question is
Given the fact that ISCSI only uses one TCP stream ( session) what will be the best way to configure the Network Interfaces in the freenas ?

I originally setup the 4 Nics as load balance on 2 Swicth stack and noticed everytime a sesssion was opened the 4 switchports flapped ..
I have now Setup one interface in one Vlan and another on another Vlan

VM has 2 extra vNics configured on each vlans for iSCSI traffic , One Session going over one Vlan and another going throug another VLAN.

Is it safe to assume that with this setup MPIO and Loadbalance is covered or did i missed somthing ?

Thanks for your replies

 

[zambanini]

so you bought 12 freenas maschines. and you did not read the faq, the manual and the stickies? that is rude.

why do I know that you did not read the stickies? because you did not specify your hw in the post.

 

[KPc]

hey Zambani
Thanks for your comments .

Basically 1 U Server made by Asus. 4 X 1 TB 72 rpm HDD , 4 Gb Ram, 4 NIC Interfaces ,, Regular Server HW.
Running Lastest FreeNas Release.
Yes I did read as much as posible , But since FreeBSD kind of has its own rules for Network Bahavior , I decided to question my setup ..

So Here is the question again .
What will be the best Setup for both Performace and availability?

1- Freenas 4 NICs on Load balance , , MPIO Setup on the VM Windows 2012 R2
2 - Freenas 2 Nics configured with LB on one Subnet and 2 On another Subnet. ( IP portal will include both IPs). MPIO ON VM
3. 1 Nic on one Subnet and another Nin on another Subnet - No LB ( IP portal will include both IPs)- MPIO On VM

Running Cisco 3850X as Stack of 2

I noticed that with 4 Nics in LB the Interfaces flap on every TCP Session.

 

[pirateghost]

4gb ram?

Is that a typo or are you serious?

 

[jgreco]

Name some names. What company sold you Asus 1U servers with 4GB of RAM and preloaded FreeNAS on there for you?

 

[KPc]

4gb ram?

Is that a typo or are you serious?

i might be wrong ,

The Box comes with 8 GB of RAM - Just double checked

 

[KPc]

Name some names. What company sold you Asus 1U servers with 4GB of RAM and preloaded FreeNAS on there for you?

Dont want you guys to DDOS their website , So no names . LOL.
Was a pretty good deal compared to QNAP.

The Box comes with 8 GB ram . Just confirmed!!

 

[zambanini]

lets guess. no ecc ram. forget lacp. also mpio will not help you for write bursts. what exactly do you expect?

how much data will you put on each box? and why are we discussing it here. ixsystems has a sales team..scnr.

 

[jgreco]

Dont want you guys to DDOS their website , So no names . LOL.
Was a pretty good deal compared to QNAP.

The Box comes with 8 GB ram . Just confirmed!!

So half as much RAM as the minimum needed for iSCSI. For a small platform with just four drives, you will probably be able to get away with 32GB but usually more is needed.

 

[KPc]

lets guess. no ecc ram. forget lacp. also mpio will not help you for write bursts. what exactly do you expect?

how much data will you put on each box? and why are we discussing it here. ixsystems has a sales team..scnr.

Yes has ecc mem

So half as much RAM as the minimum needed for iSCSI. For a small platform with just four drives, you will probably be able to get away with 32GB but usually more is needed.

The ISCSI LUN will not be used Heavilly .. In Fact the Entire BOX will not be used heavely , I will store some Vrangers Backups via CIFS for local recovery . we dont want to recover 120 GB of Data over wan on aa network that 80 % traffic is Voice ..
ISCSI is Just to store Windos Images, and WSUS updates. So more reads than writes ..

 

[jgreco]

so, tell us a little bit more about what you plan to do. For example, how much data do you plan on storing on these machines?

 

[KPc]

so, tell us a little bit more about what you plan to do. For example, how much data do you plan on storing on these machines?

WSD Images Via ISCSI = Max 50 GB
WSUS DB - Via ESCSI = MAX 100 GB
vRanager backup = 800 GB to keep a week history .

Traffic . WDS once images are in place . we image one computer once a week., WSUS Lun . about 500 Pcs downloading WSUS updates. Not very frequently ..
Vranger.= Full Backup once a week of max 5 Vms .

 

[jgreco]

How much disk are you planning to put on each machine?

 

[KPc]

How much disk are you planning to put on each machine?

the NAS has 4 X 1TB

Partition as RAIDZ
WDS = 50GB
WSUS = 100
Vranger = 800
Maybe sone local shares for the Local techs ..

 

[jgreco]

Just so I have a better idea how badly they screwed you, is that 1 8GB module or 2 4GB modules?...

 

[KPc]

Just so I have a better idea how badly they screwed you, is that 1 8GB module or 2 4GB modules?...

2 X 4

 

[jgreco]

I'll provide a lengthier response when I'm not on a mobile device...

 

[KPc]

I'll provide a lengthier response when I'm not on a mobile device...

Much appreciated .

Just want to make sure the desing is as good as it can get with what we have !

 

[jgreco]

Okay, so, kinda crap.

1) Your vendor screwed you by selling you the 2 x 4's. Despite the fact that it gives you better memory bandwidth, that's something you don't need on a NAS. When you inevitably find out that your performance is suffering because you have insufficient RAM, instead of merely adding another 1 x 8 alongside an existing 1 x 8 (which still leaves two slots free) you have bad choices depending on what the mainboard supports - probably need to add two more 4's or 8's (16GB or 24GB config). Anyone selling you a NAS should know that the limiting factors are network and disk I/O, and that even the slowest memory you can stuff in there is way fast enough. I don't think your vendor has a clue.

2) Your vendor screwed you by selling you 1TB's. A ZFS pool used for random block access services like iSCSI really should not be pushed beyond 60% capacity, and may run into fragmentation issues even well below 60%. RAIDZ is not appropriate to block access service either, so what you have is best set up as two vdevs at 1TB each, and you pretty much cannot go past 1.2TB without killing performance. Fragmentation over time will take what initially seems like "this is pretty good" performance and put it in the crapper. https://forums.freenas.org/index.ph...res-more-resources-for-the-same-result.28178/ Again, a knowledgeable vendor wouldn't have sold you hardware that's barely capable of handling your immediate needs, and that will degrade with time.

3) The minimum for iSCSI is 16GB as noted in the manual. This isn't a technical limitation, but rather a practical one - performance will be extremely variable on an under-resourced system. As the stresses of daily operation build up, and fragmentation increases, memory is the thing that offsets it. You basically have none to spare. So what will happen is even with 16GB, you are likely to get to a point where the thing just goes over the edge of a cliff and suddenly things are slower.

4) If performance isn't a primary concern, which maybe it isn't, you can probably make what you have work for the time being, but you can totally forget about MPIO and be prepared for pain down the road. It's out there. With only 8GB, performance will never be spectacular and may get substantially worse.

5) If performance is a primary concern, it isn't even clear that the selected hardware is suitable to the task, but it'd be a lot closer to suitable if you returned all the RAM, put 32GB in each box, returned the 1TB disks, ordered 2TB disks instead, and then tried to keep pool utilization down under 30-40%.

 

[cyberjock]

Prime example of a company (whoever they are) that were solely interested in selling you a product and not actually valuing your data.

I just wish you had done enough research and asked enough intelligent questions to realize the bone-job you were about to get yourself into by buying the product. :/