Planning my first NAS build. HDDs purchased, planning computer hardware.

[AND_YOU_ARE]

I'm getting ready for my first FreeNAS build, and my first NAS. I have picked up 8x 8TB WD Reds that I shucked from the WD Easystore enclosures on Black Friday and Cyber Monday sales. The NAS will be utilizing six of the 8TB Reds in a Raid-Z2 configuration. The other two reds will be used as a spare drive and a backup of essential data. After tax, the 6 Reds cost me $1040. The rest of the hardware is coming in at about $1100, and is near the upper limit of my budget. I could be convinced to spend a little more if it is justifiable. At the same time I am planning this NAS build, I am working on installing Ubiquiti wired and wireless network hardware in my house. So I cant spend all my money on the NAS.

I currently have about 6TB of storage that I would be transferring over to the NAS. I would use this to host files, backup data from laptops and desktops, stream media using Plex (lets say 2-3 simultaneous stream), and possibly record TV shows OTA. I am planning this hardware to last a minimum of 6 years before upgrade (hopefully more than 6 years). The only upgrade I would plan on doing is to add another 8TB drive or two depending on how fast I fill up the first six drives.

My two main build goals are longevity and energy efficiency. I went with the Xeon E3 because I had read it is more efficient at idle, as well as the 5400RMP Reds for their lower power consumption at startup and idle.

So far I have picked the following hardware after reading though the recommended hardware guides.

SUPERMICRO MBD-X11SSH-F-O Micro ATX Server Motherboard LGA 1151 Intel C236 ($ 209.99)
https://www.newegg.com/Product/Product.aspx?Item=N82E16813182995

Crucial CT16G4RFD4213 16GB 288-Pin DDR4 2133 Server Memory ($ 398.68)
https://www.newegg.com/Product/Product.aspx?Item=N82E16820148855

SanDisk SSD PLUS 2.5" 120GB SATA III MLC Internal Solid State Drive (SSD) SDSSDA-120G-G26 ($ 64.99)
https://www.newegg.com/Product/Product.aspx?Item=N82E16820173150

Intel Xeon E3-1230 V5 3.4 GHz LGA 1151 80W BX80662E31230V5 Server Processor ($ 259.99)https://www.newegg.com/Product/Product.aspx?Item=N82E16819117613
https://www.newegg.com/Product/Product.aspx?Item=N82E16819117613

Fractal Design Define R5 Black Silent ATX Midtower Computer Case ($ 79.99)
https://www.newegg.com/Product/Product.aspx?Item=N82E16811352048

Seasonic FOCUS Plus Series SSR-550FX 550W 80+ Gold Intel ATX 12V Full Modular 120mm FDB Fan Compact 140 mm Size Power Supply ($ 74.99)
https://www.newegg.com/Product/Product.aspx?Item=N82E16817151189

Fractal Design Silent Series R3 140mm Silence Optimized Rifle Bearing Black/White Computer Case Fan ($ 19.98)
https://www.newegg.com/Product/Product.aspx?Item=N82E16835352016

My questions so far are the following:
The rule of thumb for ram (1GB per 1TB of storage), is that for usable space or hard drive capacity? Due to cost I don't want to buy 64GB of ram at this time. Six 8TB drives in Z2 gives me 28TB of usable space (minus 20%). Planning for adding drives, 7x 8TB gives 33TB (minus 20%) and 8x 8TB gives 39TB (minus 20%). Would this 32GB of ram be enough to handle to 1-2 more 8TB drives in the future if I need it?

Right now the motherboard I picked out has 8 sata ports, I will be using 7 with the SSD for the OS. Should I spend the money now on a higher end board with more sata/sas ports, or buy a SAS card when I need to add the second additional 8TB drive?

I could drop back the CPU to a Xeon E3-1220 to save a few dollars, but I feel the E3-1230 is in a good price to performance spot. The other motherboard I am looking at is the Supermicro X11SSL-CF-O ($280.00). The main difference I see is the SAS capacity for an extra 6 drives.

The Ram I picked out wasn't on the tested ram list, but will it be fine with this board or another X11 board?

Thanks for any help!

 

[Johnnie Black]

The Ram I picked out wasn't on the tested ram list, but will it be fine with this board or another X11 board?

Nope, that's registered ECC, you need unbuffered ECC

 

[Chris Moore]

The rule of thumb for ram (1GB per 1TB of storage), is that for usable space or hard drive capacity? Due to cost I don't want to buy 64GB of ram at this time.

Above the 8GB minimum, RAM impacts performance more than anything because ZFS uses the memory for ARC. With the amount of memory you put in your buy list, you should be fine, but if you feel the need, you can always upgrade it later. Don't worry.

Right now the motherboard I picked out has 8 sata ports, I will be using 7 with the SSD for the OS. Should I spend the money now on a higher end board with more sata/sas ports, or buy a SAS card when I need to add the second additional 8TB drive?

If you need more drives later, you can add a SAS controller then, you don't have to get into that at this point.

I could drop back the CPU to a Xeon E3-1220 to save a few dollars, but I feel the E3-1230 is in a good price to performance spot.

If I could, I would go with the 1230 as it gives you more to work with if you want to create a VM or two. I think it is a good choice.

Overall, it looks like a good build, except for the RAM issue that @Johnnie Black already pointed out.

 

[AND_YOU_ARE]

Thanks for catching the ram. I am now looking at this set of ram:
Kingston ValueRAM 32GB (2 x 16GB) DDR4 2133 RAM (Server Memory) ECC DIMM (288-Pin) KVR21E15D8K2/32 ($393)
https://www.newegg.com/Product/Product.aspx?Item=20-242-334

Its my understanding that I would see a decent performance loss if I went with just 16GB of ram, correct?

 

[danb35]

The rule of thumb for ram (1GB per 1TB of storage), is that for usable space or hard drive capacity?

The rule is deliberately vague in that regard, and also quite loose. You'd likely be fine with 16 GB.

 

[droeders]

I would recommend the X11SSM-F motherboard instead. @Ericloewe has a great breakdown of the X11 series here:

https://forums.freenas.org/index.php?resources/so-you’ve-decided-to-buy-a-supermicro-x11-board.13/

 

[MikeSuede]

The only upgrade I would plan on doing is to add another 8TB drive or two depending on how fast I fill up the first six drives.

Unfortunately, you can't expand your volume by adding drives to current vdev, once it's created. To expand, you'll either have to increase the size of all your current drives or add another vdev of equal size (in your case: 6 more 8TB drives in Raid-Z2) to your volume. It's the way the ZFS file system works.

 

[Chris Moore]

There is talk of improving the functionality of that to allow adding drives to an existing vdev but that probably won't happen until sometime next year, if it happens that soon.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[danb35]

They hope to be able to demonstrate at the conference in about a year. I'd say two years, minimum, before it makes it to FreeNAS.

 

[MikeSuede]

Good to know. I will definitely keep a watch out for the news. :)

 

[AND_YOU_ARE]

Thanks MikeSuede, I guess that changes my plans a little. I might now make this a seven drive Raid-Z2 setup. Might as well use up all the sata port from the start.

Thanks doeders, the X11SSM-F board saves a few dollars. From what I was reading, if you use the m.2 slot for the OS disk, you lose a sata port. So since I wasn't planning on using the m.2 why pick out a board that has it.

Now its time to start picking up the hardware. I'm leaning towards having 7x 8TB drives in Raid-Z2 with 32GB of ram (for 25.8TB of usable storage).

 

[AND_YOU_ARE]

Ive been reading more about FreeNAS while I wait for the right price points on the hardware.
One thing I was reading about is that FreeNAS does better when running Raid-Z2 on even numbers of drives. Does it make more sense to go with mirrored USB flash drives for the OS, and install 8x 8TB drives in Raid-Z2?

 

[MikeSuede]

Ive been reading more about FreeNAS while I wait for the right price points on the hardware.
One thing I was reading about is that FreeNAS does better when running Raid-Z2 on even numbers of drives. Does it make more sense to go with mirrored USB flash drives for the OS, and install 8x 8TB drives in Raid-Z2?

That would be my move. You might as well maximize your capacity since you can. Just make sure that you use a reputable brand of USBs and it should be smooth sailing.


Sent from my iPhone using Tapatalk

 

[AND_YOU_ARE]

Here is an update on the NAS build. I decided to ditch the SSD drive I already owned and use all eight sata headers for the eight 8TB drives setup in Raid-Z2. My current plan is to use two USB drives mirrored for the OS. The following items have been purchased:

SUPERMICRO MBD-X11SSH-F-O Micro ATX Server Motherboard LGA 1151 Intel C236 ($ 209.99)
https://www.newegg.com/Product/Product.aspx?Item=N82E16813182995

NZXT H440 STEEL Mid Tower Case. Next Generation ($ 86.99)
https://www.newegg.com/Product/Product.aspx?Item=N82E16811146218

Seasonic FOCUS Plus Series SSR-550FX 550W 80+ Gold Intel ATX 12V Full Modular 120mm FDB Fan Compact 140 mm Size Power Supply ($ 74.99)
https://www.newegg.com/Product/Product.aspx?Item=N82E16817151189

8x 8TB WD Red Drives (shucked from the WD Easystore 8TB External USB 3.0 Hard Drive)
https://www.bestbuy.com/site/wd-eas...-3-0-hard-drive-black/5792401.p?skuId=5792401


I am eyeing the following major hardware to complete this build:
Kingston ValueRAM 16GB (1 x 16GB) DDR4 2133 RAM (Server Memory) ECC DIMM (288-Pin) KVR21E15D8/16I (Intel Validated)
(2x $225.15)
https://www.newegg.com/Product/Product.aspx?Item=9SIA98C5JA8296

Intel Xeon E3-1230 V5 3.4 GHz LGA 1151 80W BX80662E31230V5 Server Processor ($ 259.99)
https://www.newegg.com/Product/Product.aspx?Item=N82E16819117613

Ive been thinking more about the OS drive(s). I don't yet have a a pair of flash drives picked out yet. Is there a good brand or size that I should use that is reliable for this application? My other thought is why not use the SSD I already have (a 256GB Samsung 850 EVO) but use it on a usb header? Using something like this:
https://www.newegg.com/Product/Product.aspx?item=N82E16812400542
Is that possible with FreeNAS? Or at this point does it make more sense to pick up a used HBA card?

 

[Chris Moore]

Ive been thinking more about the OS drive(s). I don't yet have a a pair of flash drives picked out yet. Is there a good brand or size that I should use that is reliable for this application? My other thought is why not use the SSD I already have (a 256GB Samsung 850 EVO) but use it on a usb header? Using something like this:
https://www.newegg.com/Product/Product.aspx?item=N82E16812400542
Is that possible with FreeNAS? Or at this point does it make more sense to pick up a used HBA card?

If I recall correctly, that was tried and didn't work reliably because there is a driver required for the adapter.

Are you short on SATA connectivity or something? Why would you want to do USB?

 

[AND_YOU_ARE]

My plan was to use all eight sata headers on the motherboard for eight 8TB WD Red drives.
The more I think about it, I am considering picking up a HBA card and using the SSD I already have as the boot device. Any recommendations? Is the best place to pick up an HBA card going to be ebay if my concern is price?

 

[Chris Moore]

My plan was to use all eight sata headers on the motherboard for eight 8TB WD Red drives.
The more I think about it, I am considering picking up a HBA card and using the SSD I already have as the boot device. Any recommendations? Is the best place to pick up an HBA card going to be ebay if my concern is price?

This one is conveniently already flashed with the IT mode firmware:
https://www.ebay.com/itm/Dell-H310-...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601
and the cables you need look like this:
https://www.ebay.com/itm/Lot-of-2-M...-Forward-Breakout-Internal-Cable/371681252206
Just be sure they are long enough. The ones I linked are relatively short, so you might need longer ones.

 

[Ericloewe]

A decent SATA device on a decent SATA/USB bridge should be fine as a boot device. The problem is finding and acquiring such an adapter.

 

[AND_YOU_ARE]

Ah I see.
I'll pick up one of those Dell H310 HBA cards. Does it matter which drive is connected to the HBA card vs the motherboard's stat headers (OS drive vs the storage drives)?

I am ready to place my order for the processor. Checking over Newegg, I see the following two processors are the same price - $270. Any reason not to pick the V6 Kaby Lake chip? It looks like its faster and uses less power. Both are compatible with the Supermico board I already purchased.

Intel Xeon E3-1240 V6 Kaby Lake 3.7 GHz (4.1 GHz Turbo) LGA 1151 72W BX80677E31240V6 Server Processor
https://www.newegg.com/Product/Product.aspx?Item=N82E16819117787

Intel Xeon E3-1240 V5 3.5 GHz LGA 1151 80W BX80662E31240V5 Server Processor
https://www.newegg.com/Product/Product.aspx?Item=N82E16819117614

I should be settling on RAM later tonight after reviewing the Supermicro QVL.
Thanks for the help so far!

 

[Ericloewe]

Does it matter which drive is connected to the HBA card vs the motherboard's stat headers (OS drive vs the storage drives)?

Unless you're a masochist, avoid trying to boot from an LSI controller. So, connect your boot device to the motherboard SATA ports.