Sharing 56TB in FreeNAS

[Carlos Owusu]

Hello,

We are currently upgrading our old 32TB NAS (E3-1230v3, Supermicro X9, 32GB RAM, FreeNas 9.3, 8x4TB WD REDs) to 56TB (6x8TB and 2x4TB WD REDs). The storage will be shared among three departments - Traffic, Programs & Production. These will appear as Windows Shares with permissions on our network. If we were to split the storage equally, what is the best way to configure this storage so that will offer us redundancy in case one of the drives fails?

Thanks in advance!

 

[toadman]

I would think it depends in part on the use case for each department. i.e. frequency and volume of traffic. The two 4TB drives are an outlier, so if you want to use them I would probably mirror those in their own pool, then to 6x8TB of RAIDZ2 in another pool. But this will leave you with a 4TB pool and a 32TB pool (usable). Is this tolerable? i.e if all departments need 12TB then someone has to have a 4TB share on it's own pool and a separate 8TB on the other pool, shared out in two different network share locations.

I think the easier answer is to buy two more 8TB drives and then do a pool with an 8x8TB in RAIDZ2 for 48TB (usable). Then a dataset for each department.

What was your redundancy before?

 

[Carlos Owusu]

First setup was 32TB RAIDZ2 with I think 28GB usable.

Programming has to have the highest share as they will rewriting to the drive constantly followed by Production and then Traffic. Traffic may not need to have any redundancy as files are usually stored here temporarily and then moved to either Programming or Production.

 

[depasseg]

You don't assign the data protection level based on the share.

Start with how much data is needed to be supported and what amount of disk failure can be tolerated, then factor in performance needs, this will dictate your vdev setup which will provide you with 1 or more pools. You could have one pool for high performance that is comprised of 10 vdevs each with a mirrored set of drives, and another that has a single vdev with 12 drives running RAIDZ3. Then you would create teh appropriate shares on each of those pools corresponding to the user needs.

 

[toadman]

The traffic one depends further on whether the only copy of their files exists on the NAS, even if temporarily. If so, I would think they would need redundancy or a single drive failure will destroy their files.

I don't see a better way to set it up than 6x8TB RAIDZ2 (I would not do RAIDZ with 8TB drives, no way) and 2x4TB mirror. I suppose you could technically put both of those vdevs in a single pool, but it's not at all recommended. I suspect it would work, but the performance would be inconsistent and I'd have to calculate what it would do to the MTBF of the pool. If that doesn't matter you could consider doing it. Personally I would do two pools each with one vdev as described, or buy two more 8TB drives. :)

 

[toadman]

corresponding to the user needs.

Yup. Gotta know what the customer needs first. :)

 

[Carlos Owusu]

You don't assign the data protection level based on the share.

Start with how much data is needed to be supported and what amount of disk failure can be tolerated, then factor in performance needs, this will dictate your vdev setup which will provide you with 1 or more pools. You could have one pool for high performance that is comprised of 10 vdevs each with a mirrored set of drives, and another that has a single vdev with 12 drives running RAIDZ3. Then you would create teh appropriate shares on each of those pools corresponding to the user needs.

K got it!

We are looking to use the largest amount of storage available to us with full redundancy. Forgot to mention that we are storing media files (video, audio clips, presets, etc...) so performance is key.

 

[Carlos Owusu]

The traffic one depends further on whether the only copy of their files exists on the NAS, even if temporarily. If so, I would think they would need redundancy or a single drive failure will destroy their files.

I don't see a better way to set it up than 6x8TB RAIDZ2 (I would not do RAIDZ with 8TB drives, no way) and 2x4TB mirror. I suppose you could technically put both of those vdevs in a single pool, but it's not at all recommended. I suspect it would work, but the performance would be inconsistent and I'd have to calculate what it would do to the MTBF of the pool. If that doesn't matter you could consider doing it. Personally I would do two pools each with one vdev as described, or buy two more 8TB drives. :)

Yes, two extra 8TB drives would have been ideal but the budget didn't allow for that. We might add those in the future though ;)
That's why we wanted to carry two of our older 4TB WD drives into the new setup just to expand the storage a little bit.

 

[snaptec]

If Performance is important you should go with mirrored vdevs instead of one raidz2 vdev


Gesendet von iPhone mit Tapatalk

 

[depasseg]

largest amount of storage available to us with full redundancy

what does full redundancy mean to you?

 

[snaptec]

Overall you want best redundancy, largest amount of storage and best performance...

Pick one !


Gesendet von iPhone mit Tapatalk

 

[Carlos Owusu]

Sorry if I'm not making myself too clear.

I would like to have the largest amount of storage with redundancy. Our RAID Z2 we have now performs optimally but we were just looking for storage expansion to
cater for the other departments needs.

 

[snaptec]

If perfomance is ok you have just the Choice with more or less redundancy (z2 vs. Z3).
For availaiability z2 is normally enough.
In case of Desaster recovery you should have a Backup (or 2 or 3).


Gesendet von iPhone mit Tapatalk

 

[danb35]

For your storage layout, you have two decent options I can see:

• Set up a pool with all eight drives in RAIDZ2, with a plan to replace the 2 x 4 TB drives with 8 TB drives in the fairly-near future. This will give you a net capacity of about 24 TB now, which will expand to 48 TB once you replace the two drives.
• Set up two pools, the first with the 6 x 8 TB drives in RAIDZ2, and the second with the 2 x 4 TB drives mirrored. The first pool will have a capacity of about 32 TB; the second, 4 TB. This gives you more capacity in the short term, but less in the long term (i.e., once you've replaced the 2 x 4 TB drives with 8 TB drives). It also fragments your storage--it will always be in two separate volumes.

If you think you'll be able to buy the two more 8 TB drives fairly soon, and you don't need the extra capacity before then, I'd vote for the first option.

 

[Mirfster]

We are currently upgrading our old 32TB NAS (E3-1230v3, Supermicro X9, 32GB RAM, FreeNas 9.3, 8x4TB WD REDs) to 56TB (6x8TB and 2x4TB WD REDs).

Out of curiosity, are you saying that you have 32TB of drives in the old NAS and are getting 56TB of new/additional space? If that is the case, may I ask why not consider just sticking the new drives in a JBOD and expanding the existing Pool(s)/Volume(s) you already have, or replacing some of them and letting that "auto-expand". At least consider those drives in your calculations as well. No need to dump the good 8x4TB WD REDs.

If I am incorrect on this, I apologize and you should just ignore my mindless babble. ;)

 

[Carlos Owusu]

Out of curiosity, are you saying that you have 32TB of drives in the old NAS and are getting 56TB of new/additional space? If that is the case, may I ask why not consider just sticking the new drives in a JBOD and expanding the existing Pool(s)/Volume(s) you already have, or replacing some of them and letting that "auto-expand". At least consider those drives in your calculations as well. No need to dump the good 8x4TB WD REDs.

If I am incorrect on this, I apologize and you should just ignore my mindless babble. ;)

Well we are using a Supermicro 825TQ server case which holds only 8 drives. Would be neat if we could get the same setup so we don't have to toss the 32TB pool out. :(

 

[Mirfster]

Well we are using a Supermicro 825TQ server case which holds only 8 drives. Would be neat if we could get the same setup so we don't have to toss the 32TB pool out. :(

All you basically need is a SuperMicro JBOD box and a LSI 9200-8E. I am no SuperMicro expert (I use all Dell C2100/FS12-TY Servers); but I am sure the others can easily provide you with that info.

Guessing the box would be ~$150.00, LSI 9200-8E ~$50.00; thinking total ~$250.00 (Shipping, Cables, etc)....

Would be a shame to dump those drives...

Another idea would be just to get a larger SuperMicro case, like a 4U and swap the Motherboard, CPU, RAM etc... Then you could accommodate more drives. FreeNAS doesn't care, it will gladly handle that.

 

[Robert Trevellyan]

Would be neat if we could get the same setup so we don't have to toss the 32TB pool out.

Does everything have to be in one box?

 

[snaptec]

If you want the new one besides the old, it would really be a option to just change the case, f.e. with a 846 4u or use a jbod.
Only disadvantage is you can't get over 32gb with your mobo.

 

[Carlos Owusu]

If you want the new one besides the old, it would really be a option to just change the case, f.e. with a 846 4u or use a jbod.
Only disadvantage is you can't get over 32gb with your mobo.

So if we were to get a case like this
https://www.amazon.com/dp/CSE846TQR9/?tag=ozlp-20

We could just expand the storage with our six 8TB drives? What would be a good setup with 8 4TBs and 6 8TBs that
would offer some sort of redundancy and maximum storage capacity.