BUILD New Build

[ZakMckracken]

I currently have a non-ECC, 5x 3TB RaidZ1 FreeNAS build I built a couple of years ago, before I did a lot more research into FreeNAS. It's at almost 80% capacity, so I've been researching for a new build, and this time I want to do it right.

I'm planning to use the new server primarily for media storage/serving with Plex with 1-3 clients streaming most of the time. I'm already planning to use ECC and RaidZ2 this time around. I've been searching through the forums, reading the guides, and still have some questions about how to go about this build. I know 1GB RAM for 1TB is the general guideline, but it's also a significant jump to get past 32GB.

With all setups, I'd like to use 10 4TB HGST NAS drives and either an NZXT Source 210 or a Fractal R4/R5, and PSU: SeaSonic G Series SSR-550RM

1st Build (32GB RAM)
Mainboard: Supermicro X10SL7-F
CPU: Xeon E3-1231v3
RAM: 2x Crucial 16GB (2x 8GB) CT2KIT102472BD160B

2nd Build option (>32GB RAM)
I've seen two different mainboard suggestions, and I'm not sure which is a better choice. In the hardware guide, the X9SRH-7F is suggested, but I've seen several newer builds using the X10SRL-F instead. Either way, I'd pair it with a Xeon E5-1620. Is the X10SRL-F supported well enough in 9.3 that it's a better choice?

I'm sorry if some of this was answered in other threads that I missed. I tried, but I've been reading threads for days, and my have missed some.

 

[ZakMckracken]

So, I guess my questions are:
1) If I'm using 10 4TB's, should I make the jump past 32GB? Would the 32GB RAM be a limitation on performance at that size?
2) If I do go over 32GB of RAM, which of those two boards is the better choice?

 

[marbus90]

I'd rather run a 11disk z3 or 2x 6disk z2 vdevs. ;) You can always start on a Xeon E3/32GB platform, but do plan ahead for a fullsize ATX mobo which takes a Xeon E5 with RDIMMs if you plan to keep that system around for a while, meaning troughout the next capacity jump. Then again, at that time a new platform with then-reasonably priced 64GB is possible... other than that, the current Xeon selection seems nice.

If you want to jump on the E5 train right now, take the X10SRL-F one. It would support up to 1TB RAM with future 128GB LRDIMMs and right now already 256GB on 32GB LRDIMMs. Reasonably priced are the 16GB RDIMMs tho.

 

[cwxc818]

Wow, it looks like we're going to have very similar builds (referring to your 1st build option). :)

I have 3 4TB HGST NAS and plan on getting 3 more and running RaidZ2. I ended up going with the NZXT Source 210. I was going to use the Fractal R4 when it was on sale for ~$80 but I didn't pull the trigger and now it's back up in price. I'll probably get the 450W SeaSonic G Series PSU and I was just looking at the same RAM and deciding on which Xeon E3 to go with.

If you're going to buy a bunch of the HGST NAS, make sure to do a "combo" at Newegg (I don't think this applies automatically if you add a bunch to your cart). You can filter or search by Keyword or Item #. There are also some combos that include the X10SL7-F and various Xeon processors to save you a few $.

 

[ZakMckracken]

I'd rather run a 11disk z3 or 2x 6disk z2 vdevs. ;) You can always start on a Xeon E3/32GB platform, but do plan ahead for a fullsize ATX mobo which takes a Xeon E5 with RDIMMs if you plan to keep that system around for a while, meaning troughout the next capacity jump. Then again, at that time a new platform with then-reasonably priced 64GB is possible... other than that, the current Xeon selection seems nice.

If you want to jump on the E5 train right now, take the X10SRL-F one. It would support up to 1TB RAM with future 128GB LRDIMMs and right now already 256GB on 32GB LRDIMMs. Reasonably priced are the 16GB RDIMMs tho.

I'm not against going the Z3 route, though the system is going to be an expense even with the Z2 setup. I didn't want to do the separate 6 disk vdevs due to the space loss and additional drive cost, but it would make any vdev operations (drive replacements & resilvering) or expansions easier & faster.

Thank you for recommending the X10 for >32GB of RAM. I wasn't certain on its support, but I'd seen others using it in their builds.

 

[ZakMckracken]

Wow, it looks like we're going to have very similar builds (referring to your 1st build option). :)

I have 3 4TB HGST NAS and plan on getting 3 more and running RaidZ2. I ended up going with the NZXT Source 210. I was going to use the Fractal R4 when it was on sale for ~$80 but I didn't pull the trigger and now it's back up in price. I'll probably get the 450W SeaSonic G Series PSU and I was just looking at the same RAM and deciding on which Xeon E3 to go with.

If you're going to buy a bunch of the HGST NAS, make sure to do a "combo" at Newegg (I don't think this applies automatically if you add a bunch to your cart). You can filter or search by Keyword or Item #. There are also some combos that include the X10SL7-F and various Xeon processors to save you a few $.

Thank you for that, I completely overlooked the combos. I have a friend who recently put 10 drives in an NZXT Source 210 and he said it was a good case for it. The rest of his hardware setup is quite different, though.

 

[ZakMckracken]

In further research, the X10SRL-F only has 10 SATA 3 ports. If I were to go with 11x Z3 or with 2 vdevs of 6x Z2, I'd need either a different mainboard, or an add-on card to get another port. Is the M1015 still the best way to go for that, or would it be better to just switch mainboards (to the X10SRH-CF perhaps)?

It's about $130 increase from the X10SRL-F to the X10SRH-CF, but the M1115/M1015 cards are also about $100.

 

[Ericloewe]

In further research, the X10SRL-F only has 10 SATA 3 ports. If I were to go with 11x Z3 or with 2 vdevs of 6x Z2, I'd need either a different mainboard, or an add-on card to get another port. Is the M1015 still the best way to go for that, or would it be better to just switch mainboards (to the X10SRH-CF perhaps)?

It's about $130 increase from the X10SRL-F to the X10SRH-CF, but the M1115/M1015 cards are also about $100.

Right now, M1015s/LSI SAS 9211s/LSI SAS 9207s are a bit more stable than LSI SAS 3008-based stuff, like the X10SRH-CF.

 

[ZakMckracken]

Thank you guys for your input. I've still got a lot of learning to do before I assemble this build (never used LSI based parts before so flashing to IT will be a first).

 

[jgreco]

So, I guess my questions are:
1) If I'm using 10 4TB's, should I make the jump past 32GB? Would the 32GB RAM be a limitation on performance at that size?
2) If I do go over 32GB of RAM, which of those two boards is the better choice?

The rules of thumb get looser as the amount of RAM goes up. 32GB is almost certainly fine for your intended use case with a 32TB pool, and very likely still fine even if you had 10 6TB's (48TB). If you are buying a reasonably-sized system for your needs in the foreseeable future, it may be wiser to go E3 now and then buy a "better" E5 later if needed.

If you do go E5, be sure to be looking at the E5-1600 parts. The 1650 is a fantastic CPU and is not outrageously priced (compared to many of the 2600 parts).

 

[Rilo Ravestein]

With all setups, I'd like to use 10 4TB HGST NAS drives

I have the 3TB variants of those, great performance so far. Low temperatures and quiet too.
I took the advice from some of the more experienced users on this forum and got drives from different batches (bought from several different stores), to minimize the chance of multiple drives failing at once (or too short after eacht other to be able to replace).