Advice on a FreeNAS build

[asinghania82]

Hello All,
I'm a newbie and this is my first FreeNAS build. I work for a small geospatial research lab. The NAS is mostly for storing/backing up data and connecting it to an internal network so that lab members can access it as and when they need it. It wont be used for any kind of data processing, just storage of large volume of data. I've gone through the recommendations on the FreeNAS website and the forums here and come up with some system specs. I wanted you all's advice on:

Case: Rosewill 12 BAY HotSwap RSV-L4412
MotherBoard: Supermicro - X11SSL - CF - O - mostly for the 14 drives capability
CPU : Intel Pentium G4560
RAM: 32 GB - 2 x Crucial 16GB DDR4-2133 CT16G4WFD8213
HDD: 3 or 4 - 12TB Seagate Ironwolf
PSU: Seasonic FOCUS FM series 750watts 80+ Gold - SSR-750FM
Boot: Crucial MX500 250GB 3D NAND SATA 2.5 Inch Internal SSD - CT250MX500SSD1(Z)

I'm starting with 3/4 drives with the highest capacity available and 32 GB RAM for budgetary constraints. Idea is to get the NAS started and then every 8 months to a year add more 12 TB drives as the need arises, basically spreading the drive and ram cost over a year or two. I like the case for the ease of drive installation. Also the NAS will be basically sitting in our lab over a computer desk.

Let me know what you guys think.
Thanks.

 

[danb35]

add more 12 TB drives as the need arises,

How are you planning to do this? If you have, say, your currently-planned four drives in RAIDZ2, you can't just add another one to make a five-drive RAIDZ2.

 

[asinghania82]

Actually I wont be adding 1 drive at a time rather 3-4 at a time and create a new RAIDZ2. We generally create a new budget at a start of a financial year. So depending on the prevailing prices and the biggest TB sizes available I could budget in the drives and use them to create a new RaidZ2.

 

[loch_nas]

The 750W Power Supply seems a bit over-dimensioned. I think 550W or 650W for 12 drives should be enough.

If you know now already that you're filling the chassis with 12 drives at some time or other, then I would make now a vdev (RAIDZ2) with 6 drives and add another 6 drives in future.

 

[MatthewSteinhoff]

Case: Rosewill 12 BAY HotSwap RSV-L4412

I'd consider a case with more bays. Geospatial research seems like something with large data sets. Getting a larger case now costs a trivial amount more compared to having to dump this case and get a larger case later. Also, instead of having to purchase massive, cutting-edge, really expensive 12TB drives, you might be able to get by with more smaller drives. More drives typically gives better performance than fewer drives.

rather 3-4 at a time and create a new RAIDZ2.

If you're building RAIDZ2 pools using only four drives, it would be just as space efficient to use a stripe of mirrors. Not only would that give you better performance (double on a four-drive group versus RAIDZ2), you could expand your pool two drives at a time instead of four drives a time. This gives you a bit more flexibility.

Also, when you do get to the point where you have maxed out your bays with drives, you can backup your data, convert from striped mirrors to RAIDZ2 then restore the data and get a huge amount of disk space back without spending a dime. (Assuming you have a place to backup the data during the migration.)

Cheers,
Matt

 

[LIGISTX]

I would actually stick with the 750 watt. A beefy PSU is not a bad thing, and if you eventually plan on having 12 7200 rpm drives, I think that’s a good option. The wattage number is possibly “overkill”, but I guess the way to describe it is hope it has a beefy 5v delivery that can support the current draw of that many 7200 rpm drives. Should be fine.

Also, I second the idea about “how will you go about adding more storage”. How many users at a time will be hitting it, and over what networking? If you do RAIDZ2 vdevs with 4 drives, your losing 1/2 of your available space to redundancy which just makes the dollar/usable space figure go up QUICK. If you can get away with getting 6 drives to start so you can add another 6 later, your cost per usable space will be much lower and your total space in the end will be higher. Obviously, budgetary issues are a factor and you have to deploy this system within a set dollar number, but just food for thought. For instance, quick amazon search shows 8TB ironwolfs for 245 bucks where a 12 tb is 430. You can do 6 8TB drives in Z2 which will net you 32TB of space for ~1500 bucks. If you go with 4 12’s, you get 24 TB for ~1800 bucks.


Sent from my iPhone using Tapatalk

 

[tvsjr]

Do you have the flexibility to buy used gear? There's a ton of systems out there, either on eBay or through reputable surplus vendors like The Server Store, UNIX Surplus, etc. Since you're already looking at rack mount, a 4U, Supermicro 24-bay or 36-bay system would be an excellent solution for you.

I haven't bought from this company specifically, but others have and have been happy... https://unixsurplus.com/collections...-2x-e5-2680-2-8ghz-192gb-2-port-10gbe-sfp-nic
Far more speed, 4x the RAM, and 3x the expandability. There are four "fixed" bays inside that can be used to install boot drives, etc. with connections directly to the motherboard.

Your existing config, with 4 drives in RAIDZ2, gives you 24TB (minus overhead, minus free space). You could get the same from 6 6TB drives in RAIDZ2... looking at Ironwolfs on Amazon, your price goes from $427.08 * 4 = $1,708.32 to $189.79 * 6 = $1,138.74, and you get better performance as a side benefit. You could then add up to 5 additional 6-drive RAIDZ2 vdevs - either into your existing pool, or into a new one.

 

[MatthewSteinhoff]

We've bought from UnixSurplus and are happy. They also go by the names MrRackables and ITRecycleNow on eBay. Buying used on this sort of equipment is a no-brainer if you have enough lead time to give it a thorough testing. (We don't think used is more likely to fail than new but, just to cover one's ass with upper management, we tend to test used more heavily just to give us some cover if there are problems.)

Cheers,
Matt

 

[asinghania82]

Thanks a lot for all the replies.
@danb35 Thanks for your post. The link to Uncle Fester's Basic FreeNAS 9.10 Configuration Guide was very informative

@MatthewSteinhoff @LIGISTX .....The system is mostly storage and data archiving. I expect at most 3-4 users trying to download data from it or copy data to it simultaneously over a Gigabit ethernet line and even that would be a rarity.

@tvsjr ...The vendors (specially online only) we can buy from are pretty limited and unfortunately I cannot buy used gear. For e.g. we are strongly discouraged to use even EBAY.

I like the idea of smaller 6 -8TB/ or 8-6TB drives. It does seem more cost and space effective as well as better performance.
Before I was thinking that once the case fills up I will simply add an LSI controller to the original system and use it with a JBOD/DAS enclosures with SAS connections. However it turns out they are actually pretty expensive.
Options that might be possible for larger bay case, depending on where I can cut back and how persuasive I am (to the boss of course) are the NORCO RPC-4020 and NORCO RPC -4024. If not, then I might have to go for the 15 bay Rosewill RSV-L4500 with preinstalled 7 fans. Any other suggestion for the cases are most welcome.

I did have a couple of questions:

a) What do you guys thinks about the 5900 RPM Seagate Archive ST8000AS0002 8TB. These are a little cheaper than the Ironwolfs and can help free up money for the case.

b) Secondly, I was using the ZFS/RaidZ capacity calculator and it shows having 8-6TB drives gives a usable space of 68.6% and 6-8TB drives a usable space of 64.3% . Would there be any reason as far and performance and reliability of the system is concerned to go with the 6-8TB configuration?

Thanks again guys for putting time and effort into the replies. It is highly appreciated.

 

[tvsjr]

Stay away from the "Archive" drives... most are SMR, which means they have horrendous performance and low write durability.

Less drives usually means less power consumption/heat generation, plus you aren't filling your box up as fast (burning bays, connectors, etc.) 6-drive RAIDZ2 vdevs are the most common configuration, IMO.

I would still consider pricing out one of the 24- or 36- bay Supermicro chassis, even if you have to buy new. They Just Work(tm).