BUILD 10bay FreeNAS box (first build)

[cyris212]

Hi guys,

I've been using FreeNAS on an older PC for a couple of months.
Now I want to build my own production box to store media files and personal data on it.

Currently I'm using 4*4TB WD RED's, but I want it to be expendable to a maximum of 10 disks.
So using 4TB or 6TB disks, it will need around 40GB to 60GB ECC RAM.
It should also be capable to run 3-4 Jails (PLEX, PyLoad, OwnCloud, Syslog/Logstash).
It will also backup my pfSense box and couple of DigitalOcean Droplets through rsync tasks.
There are only a handfull of people accessing it, so it will run most of the time idle.

What I've seen/read, there are 2 possibilites that both support upto 64GB ECC RAM.
The First one is going with an Supermicro X9/X10 board using a LGA2011/LGA2011r3 socket.
The Second would be to use one of the Supermicro's awesome Avoton boards...

I've tested FreeNAS using an i7 860 for the last few months, which is only a bit better then the Atom C2750.
The only CPU intensive task was compilling the PyLoad port... So I don't really see the point in buying a Xeon... HBA and mainboard provide 10 SATA3 and 4 SATA2 Ports which is more then enough...

Build A:

• Mainboard: A1SAM-2750F
• RAM: 4 * Intelligent Memory 16 GB ECC
• Power Supply: Seasonic 360W Gold
• HBA: IBM ServeRaid M1015
• Case: Cooler Master Centurion
• HotSwap Bays: 2 * Supermicro CSE-M35T-1 (5 * 3.5" each)

But the avotons UDIMM's are just to pricy I will go with Build B:

• Mainboard: Supermicro X10SRL-F
• RAM: 4 * 16GB DDR4 RDIMM Samsung M393A2G40DB0-CPB (on the QVL)
• CPU: Intel Xeon E5-1620v3
  
• Power Supply: Seasonic G-550
• HBA: IBM ServeRaid M1015
• Case: Cooler Master Centurion
• HotSwap Bays: 2 * Supermicro CSE-M35T-1 (5 * 3.5" each)

What do you think?

 

[marbus90]

If you plan above 32GB RAM, get a Xeon E5 system. An Avoton 64GB system costs 30% more than a Xeon E5 64GB system (the latter being equipped with 4x16GB RDIMMs with expansion to 128GB possible and tons more of CPU power).

(all prices including 19% tax, roughly translates to USD without tax)
X10SRL-F for 280EUR
E5-1620 v3 (it's a Quadcore, it's still good enough. you only need more ram, not more CPU) for ~300EUR
4x16GB Crucial (probably not QVL) for ~700EUR total or 1x16GB Samsung (QVL) for ~180EUR each.

 

[stefanb]

I agree to marbus90, with an additional info:
The X10SRL-F has 10x SATA3 (6Gbps) ports - no need to buy the IBM ServeRaid M1015 :)
Need more SATA? Maybe the Supermicro MBD-X10SL7-F-O is a choice - up to 14xSATA with IT Mode-Patch - but only 32GB RAM max...

S.

 

[sremick]

Are we sure this system really needs 64GB RAM?

 

[stefanb]

16GB + 1GB for each TB of HDD space.
16GB + 4*6 = 40GB RAM recommended.

 

[marbus90]

I'd say more like 1GB per TB, but at least 8GB for whatever you're running. Since most boards will prefer dual over single dimms and 4GB DIMMs are a waste of money, 16GB is the recommended minimum. With s.2011-3 the best DIMM size would be 16GB RDIMMs, so start with 32 on those.

 

[david kennedy]

16GB + 1GB for each TB of HDD space.
16GB + 4*6 = 40GB RAM recommended.

Isnt that one of those inaccurate statements everyone makes about ZFS, similiar to the "optimal" number of disks to have in a pool?

 

[Ericloewe]

Isnt that one of those inaccurate statements everyone makes about ZFS, similiar to the "optimal" number of disks to have in a pool?

I wouldn't put it that way.

It's not a strict rule. It's a nice rule of thumb that gives you approximately where your memory should be. You can safely stray from it, particularly once you're into 64+ GB of RAM. You might also need a crapton more RAM, if you have a nasty workload.

As for optimal pool sizes, I wouldn't call it inaccurate, but that's a discussion for another topic.

32GB of RAM can be limiting. If someone says "I want a crapton of storage and am willing to pay for 64GB of RAM", I'm not going to argue they might get away with 32GB - it might bite them in the ass rather soon.

 

[cyris212]

Thanks a lot for your answers guys!

I did some math on the pricing and marbus90 is totaly right...
The Avoton boards are almost everytime around 30% more expensive and you get way less computing power.
I will go with a LGA2011 r3 board an the E5-1620v3.