BUILD First Build! Micro-ATX with 8+ Drives. Advice Appreciated!

Status
Not open for further replies.

ninjarobert

Dabbler
Joined
Dec 13, 2014
Messages
30
Hi all, total NAS noob here. This is going to be a bit of a rambling, knowledge dump from info that I've gathered in my brain over the past few days from various webpages and these forums (btw, the stickies are amazing). The need for a NAS was recognized recently and I need to make purchases before the end of the year. Yikes! Any assistance is greatly appreciated!

tl;dr
Usage: Photography. Backup/store image files from at least two Windows 7 computers
MicroATX Build:
Hard drives: 8x – 3TB WD Red
Motherboard Option #1: SuperMicro X10SLM-F-O
Motherboard Option #2: ASRock C226M (no dual LAN)
Motherboard Option #3: ASUS H97M-Plus (no dual LAN)
MicroATX Case: Fractal Node 804
Motherboard Option #4: X10SAE-O (will need ATX case)
ATX Case: ?
RAM: 16 or 32 GB compatible ECC RAM
CPU: i3-4330 or G3220
Power Supply: SeaSonic 550
USB stick: 8 or 16 GB SanDisk
SATA Card: ?
Switch: ?

***Update: Final Build
Thanks to everyone that gave feedback to arrive to my final build!
Motherboard: SuperMicro X10SL7-F-O
CPU: Intel Core i3-4330
Memory: 2x - 16GB Crucial CT2KIT102472BD160B
Hard Drives: 6x - 4TB HGST Deskstar NAS H3IKNAS40003272SN
Case: Fractal Node 804
Power Supply: SeaSonic 550
USB: SanDisk 16G Cruzer Ultra Fit
Switch: NETGEAR 5 Port Gigabit Business-Class Desktop Switch - GS105
Backup Power Supply: APC Back-UPS Pro 1500

Questions:
1. Will the server/network be fast enough to store images on the NAS and edit on computer(s) over the network?
2. The drives are spec'd at 6 Gb/s, but I keep reading that a mobo with SATA 3 Gb/s is sufficient. Is that because of network bottleneck?
3. I don't expect to use many of the drives to start. At our current rate, we fill 1 TB per year. In ZRAID-2 does data get 'spread' across the whole array of drives or does it fill up sequentially? If it's sequential, will the 'unused' drives get worn down while not being really used?
4. How painful will it be to eventually migrate to bigger drives?
5. With the 8 WD Red drives, could I add more non-Red drives?
6. Should I bother with a fan for the top of the fractal node 804 case?
7. Any recommendations on a PCIe SATA card?
8. Any experience here with the SuperMicro X10 SAE board?
9. I’ve read the G3220 is sufficient for simple data servers. Is there any reason to get the i3 for my use case? 10. Any recommendations for a good switch?
11. 8 or 16 GB USB stick?
12. USB stick hanging out of my case seems… odd. Are there any other options for the boot drive?
13. Did I miss any major components?

Usage of Server
Backup/store image files from a Windows PC (v7 Ultimate, 64-bit). More details in my intro: https://forums.freenas.org/index.php?threads/howdy-howdy.25657/

Question 1: Will the server/network be fast enough to store images on the NAS and edit on computer(s) over the network?

Hardware
Hard drives: I've settled on WD Red drives- I can be convinced otherwise. The WD website says 8 of these drives can be used in NAS before requiring the PRO drives. http://www.newegg.com/Product/Product.aspx?Item=N82E16822236344

Question 2: The drives are spec'd at 6 Gb/s, but I keep reading that a mobo with SATA 3 Gb/s is sufficient. Is that because of network bottleneck?

I read the system needs to be reconfigured when adding new drives, so I'd rather just configure with all 8 drives to start. Therefore, the price point at 3TB is manageable for 8 drives.

RAID Configuration: RAIDZ-1 should suffice, but will probably go with RAIDZ-2 to protect against multiple disc failures. The ~16GB of usable space should last quite a while.

Question 3: I don't expect to use many of the drives to start. At our current rate, we fill 1 TB per year. In ZRAID-2 does data get 'spread' across the whole array of drives or does it fill up sequentially? If it's sequential, will the 'unused' drives get worn down while not being really used?
Question 4: How painful will it be to eventually migrate to bigger drives?

Form Factor: No preference really. Seems like micro-ATX allows for more free space in the case and room for expansion as compared to mini-ITX

Question 5: With the 8 WD Red drives, could I add more non-Red drives?

Case: For micro-ATX, Fractal 804. Pretty easy choice. Only one I can find that holds more than 8 3.5” HDD. Luckily, it’s well reviewed. http://www.newegg.com/Product/Product.aspx?Item=N82E16811352047

Question 6: Should I bother with a fan for the top of the case?

Motherboard: As I expect to one day expand storage capacity, I’d like a mobo capable of 32 GB of memory. Seems like a favorite here are the SUPERMICRO X10’s. Never heard of SuperMicro before coming here, so I’ll put my trust in you experts! Need to do side-by-side comparison to see which fits my needs. Seems like all models have only 6 SATA ports though.

Question 7: Any recommendations on a SATA card?

Also considering the ASRock C226M and ASUS H97M-Plus. Have to do a more detailed comparison, but they don’t have dual LAN.
http://www.newegg.com/Product/Product.aspx?Item=N82E16813157561
http://www.newegg.com/Product/Product.aspx?Item=N82E16813132119

The SUPERMICRO MBD-X10SAE-O is tempting because it has 8 SATA ports. It’s a full ATX board though. Not a huge deal because I have the space to put the tower.

Question 8:
Any experience here with the SuperMicro X10 SAE board?

CPU: Since I’m just storing data, I don’t see much need for processing power. Would like one with decent/supported integrated graphics so I won’t need a gfx card though. Intel Core i3-4330 or G3220.
http://www.newegg.com/Product/Product.aspx?Item=N82E16819116945
http://www.newegg.com/Product/Product.aspx?Item=N82E16819116950

Question 9: I’ve read the G3220 is sufficient for simple data servers. Is there any reason to get the i3 for my use case?

Memory: Didn’t realize ECC memory was so pricey, may start with 16 GB. If I go with the SuperMicro board, will follow this sticky: https://forums.freenas.org/index.php?threads/ram-recommendations-for-supermicro-x10-lga1150-motherboards.23291/

Power Supply: Someone on this forum had an X10 build with the SeaSonic 550. Looks good to me!
http://www.newegg.com/Product/Product.aspx?Item=N82E16817151119

Router: I just know a switch is preferable to a hub.

Question 10: Any recommendations for a good switch?

Boot Drive: I must admit that running an OS off a USB stick seems ‘crazy’ to me. Nevertheless, I’m a fan of SanDisk. Never had a card fail in my camera. I vaguely recall reading an 8 GB stick is sufficient.
http://www.newegg.com/Product/Product.aspx?Item=N82E16820171651

Question 11: 8 or 16 GB USB stick?
Question 12: USB stick hanging out of the case seems… odd. Are there any other options for the boot drive?
Final Question: Did I miss any major components?

Thanks again for taking the time to look through this!
 
Last edited:

DKarnov

Dabbler
Joined
Nov 25, 2014
Messages
44
1) You're probably best off editing the photo files on a local SSD and using rsync / syncthing / something similar to keep the NAS copy up to date. This is out of my realm of expertise so I can't give details. Photo editing over 1GBe is going to be cumbersome regardless of what's on the NAS end.

2) Consumer-level spinning media like WD Reds tend to max read/write rates around the 100 megabytes/sec rate, which is well under max transfer rate for SATA2, let alone 3. SATA transfer rates only really come into play when you're talking very fast SSDs, which is not a concern in your build.

3) Data will be 'spread' across all drives and all drives will be used equally from day 1. If your data demands are low, maybe you're better off with just six drives in RAIDZ2 instead of eight.

4) Adding space involves either adding another set of drives (either as a new zdev or an entirely separate pool) or replacing ALL the drives, one at a time, with ones of a larger capacity. I.e. you have your 6x3TB array and, one at a time, swap out the 3TBs with 5TBs, and letting the array rebuild. A final option is to build a new, bigger pool, copy the data over, and chuck / repurpose the old drives. Each method has upsides/downsides. It's a little less elegant than, say, Drobo / Synology Hybrid RAID and it's not like regular RAID where you can just add drives, but it's not particularly vigorous and it certainly can be done without needing to move off all your data. Just make sure to read up on it before you try it.

5) You can add non-Red drives, with several caveats that will probably mean you don't want to. You want all the drives in a given vdev to be the same size, and preferably the same kind of drive. This all gets complicated and beyond the scope of simple home / SOHO use, and I recommend reading Cyberjock's guide linked all over the forum, or Googling around for ZFS and zpool / vdev info.

6) I don't know about the top fan because I don't have a Node 804, but I imagine it will be semi-obvious whether you need it or not after building the box. I do recommend throwing an intake fan or two on the HDD/PSU side of the case.

7) You may or may not need a SATA card depending on your mobo and final decision on drive number. However the goto card here for simple applications is the IBM 1015, bought off eBay for ~100 and flashed to IT mode (guides on the forum.)

Motherboards:
The SuperMicro X10SL7-F-O (http://www.newegg.com/Product/Product.aspx?Item=N82E16813182821) is favored for >6 drive mATX because you can drive up to 14 (6 SATA + 8 off the SAS expander) drives without an additional card.

The Asrock board is ok, but it doesn't have IPMI, and IPMI is a very nice thing to have with FreeNAS type stuff. Asrock doesn't have any super mATX options right now; you can go with the E3C224D4I-14S (http://www.newegg.com/Product/Product.aspx?Item=N82E16813157486) which is technically extended Mini-ITX (it'll fit in the 804 just fine.)

The ASUS board you link is no good because it doesn't support ECC memory.

With IPMI you don't need a graphics card (or monitor, or keyboard and mouse) to operate the box; just another ethernet cable and an open port on your switch to plug it into. Much, much better for servers you don't touch very often.

8) No idea about that particular SM board, see above for recommendations.

9) CPU - Integrated graphics are irrelevant - See note above about IPMI, and FreeBSD (which means FreeNAS) doesn't use the GPU for acceleration. I think the i3 gives you a little more headroom if you find you want to use the box for somewhat more stuff, but the Pentium would probably cut it.

10) Unmanaged GBE switches are cheap, get one with enough ports. Make sure you're behind a firewall, but your network layout is beyond the scope of this thread (and forum.)

11 & 12) Get a stubby USB stick like this one: http://www.newegg.com/Product/Product.aspx?Item=9SIA12K16C8558 and you will never notice it barely sticking out the back where your ethernet cables etc come out anyway. It's also common for server-class boards (including I think both the ones I linked above) to have an internal USB port specifically for boot drives like this.
 

ninjarobert

Dabbler
Joined
Dec 13, 2014
Messages
30
Extremely helpful Dkarnov! Feel a bit more confident about the whole thing :)
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
For 8+ drives, the X10SL7-F is, by far, the easiest and most reliable option you have. Forget non-server boards.
There's also an ASrock C224 board that's like a compact X10SL7-F, but it's less tested around here.

For anything business-related, go with an i3 at least. The cost difference is small, but the extra threads can come in handy. iGPU is completely irrelevant, as it is simply not used in server boards (the BMC has its own "good enough" video controller that can also send its output via IPMI).

The Seasonic G-550 is an excellent PSU for the price. It's not the best thing money can buy, but it's close enough for a server.

I strongly recommend you try out RAIDZ2 before committing to it - performance may be too slow for your use case. If that should be the case, use mirrors instead. For ZFS vdev layout details, read Cyberjock's guide (link in my sig) - everything you need to know is there.

For booting, you can either get real drives (you can mirror the boot drive starting with FreeNAS 9.3, for extra reliability) or USB drives. 4GB is the minimum, 8GB is better, more is better. More capacity = more snapshots of boot environments can be stored.

Once you've settled on a final config, be sure to run it by us again. It's easy to miss details in such long posts.
 

ninjarobert

Dabbler
Joined
Dec 13, 2014
Messages
30
Thanks Ericloewe! (for your guides and numerous useful posts too).

Made a pass through Cyberjock's amazing guide. It's a great reference.

This is my config. Going to let it stew for a couple days before making purchases:
Motherboard: SuperMicro X10SL7-F-O
CPU: Intel Core i3-4330
Memory: 2x - 16GB Crucial CT2KIT102472BD160B
Hard Drives: 8x - WD30EFRX 3TB (may get 4x to start, then add the other 4 to the zpool as a separate vdev when needed for expansion)
Case: Fractal Node 804
Power Supply: SeaSonic 550
USB: SanDisk 16G Cruzer Ultra Fit
Switch: NETGEAR 5 Port Gigabit Business-Class Desktop Switch - GS105
Backup Power Supply: APC Back-UPS Pro 1500 (want to plug main desktop into it as well)
 
Last edited:

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
Hard Drives: 8x - WD30EFRX 3TB (may get 4x to start, then add the other 4 to the zpool as a separate vdev when needed for expansion)
If you're considering starting with four disks, give thought to whether you'd want to do RAIDZ2 or mirrors. Your capacity will be the same either way. Your redundancy will be a bit worse with mirrors, but performance will be better. You may want to try out both configurations with some test data to determine if the performance with RAIDZ2 will be acceptable.
 

Z300M

Guru
Joined
Sep 9, 2011
Messages
882
2) Consumer-level spinning media like WD Reds tend to max read/write rates around the 100 megabytes/sec rate, which is well under max transfer rate for SATA2, let alone 3. SATA transfer rates only really come into play when you're talking very fast SSDs, which is not a concern in your build.

HGST NAS drives are scoring better than the WD Reds on NewEgg.com -- based on user feedback.
11 & 12) Get a stubby USB stick like this one: http://www.newegg.com/Product/Product.aspx?Item=9SIA12K16C8558 and you will never notice it barely sticking out the back where your ethernet cables etc come out anyway. It's also common for server-class boards (including I think both the ones I linked above) to have an internal USB port specifically for boot drives like this.

The Sandisk Cruzer Fit is even smaller (e.g., http://www.newegg.com/Product/Produ...12504&cm_re=cruzer_fit-_-20-171-588-_-Product ) -- and the SuperMicro does have an internal USB socket; it's USB3, which FreeNAS doesn't support with the Intel Haswell chips, but as long as you don't try enabling USB3 in FreeNAS that socket works fine as USB2. There are also adapters that convert a USB 2.0 header to sockets for a USB drive: http://www.newegg.com/Product/Produ..._re=usb_header_adapter-_-12-200-474-_-Product . Of course there are external USB2 sockets as well, but there may be good reasons for keeping your boot device out of the way inside the case.
 

ninjarobert

Dabbler
Joined
Dec 13, 2014
Messages
30
@danb35: I'm pretty sold on the redundancy offered by RAIDZ2. Thanks for the idea though. I'll be sure to do a comparison when I build the box.

@Z300M:
I've had great luck with WD Drives (mostly blues) ever since I built my first computer over 10 years ago, so this was a no-brainer for me. However, the NewEgg reviews are a bit startling. Seems to be a high rate of DOA. Weirdly the reviews are much better on Amazon. The HGST is 7200 RPM is guaranteed to have better read/write performance-- more heat though? In my case, I'm more interested in reliability though. Been searching for some objective reliability tests and this is the closest one I've found so far. *sigh*

I scored a SanDisk 16GB Cruzer Ultra Fit from an Amazon Gold Box deal yesterday. Woot! Thanks for the USB mobo information. Using the USB3 socket seems most straightforward.
 

ninjarobert

Dabbler
Joined
Dec 13, 2014
Messages
30
I'm rethinking my usage now. If the NAS is used for 'unique data' storage, I'll need a backup for the NAS. Instead, I will increase the hard drive capacity on my workstations and use the NAS strictly for backing up. I'm in a pretty paranoid mood right now after reading hard drive reviews over the past couple of hours, so I'll also continue to do my 'dumb' manual backups to external hard drives via a docking station too.

***** Start Brainstorming *****

Workstation:
3x - 3TB WD BLACK SERIES WD3003FZEX
or
3x - 4TB WD GREEN WD40EZRX

NAS:
4x - 4TB HGST Deskstar NAS H3IKNAS40003272SN (increase from 3TB to 4TB is only an additional $15/drive)
or
4x - 4TB WD Red WD40EFRX (up from 3TB to match price of option #1... 3TBs would be $200 cheaper)

In RAIDZ2, I'll have ~7.3 TB of usable storage. I'm sitting on 3TB of data now, so I should be good for 1.5 to 2 years.

After I finally decide, I'll have 6 1TB WD Blues freed up. Maybe another NAS in RAIDZ1 to act as a straight data dump while transitioning all my data to the new setup?

***** End Brainstorming *****
 

Jailer

Not strong, but bad
Joined
Sep 12, 2014
Messages
4,974
In reality you'll have ~5.9TB of usable storage on your freeNAS build with the solution your proposing. If you search the forums and read the manual you'll see that you don't want to go over 80% capacity or performance will suffer. With 3TB of data you're starting out with your target at 50% capacity. I'd take another look at your storage plans and make sure that it's going to be a practical starting point for your build.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
@danb35: I'm pretty sold on the redundancy offered by RAIDZ2. Thanks for the idea though. I'll be sure to do a comparison when I build the box.

A four-disk stripped mirror will have the same capacity as a four-disk RAIDZ2 array, and each will tolerate the loss of two disks without loss of data. With the mirror, however, there's the additional caveat that the two disks would have to be the "right" two disks. Specifically, they'd have to be from different vdevs. A four disk striped mirror would have disk1 mirrored by disk2, and disk3 mirrored by disk4. If both disk1 and disk2 failed, for example, your data would be lost. If disk1 failed, either disk3 or disk4 could fail without losing any data.

With RAIDZ2, you could lose any two disks.
 

DKarnov

Dabbler
Joined
Nov 25, 2014
Messages
44
Your today's plan isn't a bad one, where you keep data on both the local workstation storage and the NAS, but in the scenario it sounds like you're worried about, where a non-trivial event nukes the whole NAS and severely impairs your business, you probably want to be looking at keeping a copy of at least your most vital 'unique data' offsite. Either in a colo data vault or cloud.
 

ninjarobert

Dabbler
Joined
Dec 13, 2014
Messages
30
In reality you'll have ~5.9TB of usable storage on your freeNAS build with the solution your proposing. If you search the forums and read the manual you'll see that you don't want to go over 80% capacity or performance will suffer. With 3TB of data you're starting out with your target at 50% capacity. I'd take another look at your storage plans and make sure that it's going to be a practical starting point for your build.
Ah, I forgot about that. Thanks for the reminder.

A four-disk stripped mirror will have the same capacity as a four-disk RAIDZ2 array, and each will tolerate the loss of two disks without loss of data. With the mirror, however, there's the additional caveat that the two disks would have to be the "right" two disks. Specifically, they'd have to be from different vdevs. A four disk striped mirror would have disk1 mirrored by disk2, and disk3 mirrored by disk4. If both disk1 and disk2 failed, for example, your data would be lost. If disk1 failed, either disk3 or disk4 could fail without losing any data.

With RAIDZ2, you could lose any two disks.
Thanks for the clear explanation!

Your today's plan isn't a bad one, where you keep data on both the local workstation storage and the NAS, but in the scenario it sounds like you're worried about, where a non-trivial event nukes the whole NAS and severely impairs your business, you probably want to be looking at keeping a copy of at least your most vital 'unique data' offsite. Either in a colo data vault or cloud.
Great point. We've thought about that, but it's a side business and don't bring enough revenue for those services. We do however have family nearby where we can store physical copies of the data.

***** Start Brainstorming *****
Four drives seems silly in RAIDZ2 (still like its redundancy). Going with 6 drives. 6x 4TB drives gives met he capacity that I will need for the next few years. HGST vs Red... same cost. HGST appears to be more reliable (DOA and failure rate). Kind of a bummer with NewEgg's current deal on 3TB reds :( I thought the hard drive would be the easiest component to decide upon, but it proved to be the most difficult one. lol

Thanks again for all the feedback!
***** End Brainstorming *****
 

ninjarobert

Dabbler
Joined
Dec 13, 2014
Messages
30

Jailer

Not strong, but bad
Joined
Sep 12, 2014
Messages
4,974
Looks good
 

ninjarobert

Dabbler
Joined
Dec 13, 2014
Messages
30
Pulled the trigger. Phew! Can't wait to start the build!
 

ninjarobert

Dabbler
Joined
Dec 13, 2014
Messages
30
FYI.... I mixed purchases between NewEgg and Amazon depending on price and shipping costs. I ordered the HGST drives from Amazon. Nice to see them arrive in retail boxes. Each had it's own Amazon box and was secured against rattling around by being affixed (with plastic) to cardboard within the box.
 

Z300M

Guru
Joined
Sep 9, 2011
Messages
882
Let us know how those HGST drives work out for you: they would be my preferred option right now, but my present drives still have space to spare and the budget won't accommodate more drives for a while.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
My Reds from Amazon didn't arrive in retail packaging, but they had individual boxes with said plastic dampers. Comparable to retail packaging. Same thing goes for a WD Green I bought shortly after the Reds.
 

eldo

Explorer
Joined
Dec 18, 2014
Messages
99
My Reds from Newegg arrived in the same way. Individual cardboard boxes with plastic holdsters/dampers.
 
Status
Not open for further replies.
Top