First NAS build, input welcome!

[unclejoe1116]

Alright folks, go gentle. First post, and first NAS build as well.

So first, a few details about what I need this for:
- I run a small photography and videography company. Currently generate 3-5TB of data a year, but I'm anticipating that increasing over the next few years
- I already have a home network with dedicated CAT6 runs everywhere in the house
- Network switch and patch panel installed in a 6U wallmount rack unit
- Future growth may include a VM and a home camera system, so something better than a Synology is preferred
- The system may also run a home media system (HDMI pushed through Ethernet to the wall-jacks near the TV)

Right now, my only major drives are two WD Black 4TB in RAID 0 in my main computer. Here's what I'm currently looking at for my first NAS: (I was told no PCPartPicker?)
CPU: Intel Xeon E3-1245 V6 3.7GHz Quad-Core
CPU Cooler: Noctua NH-L9x64 33.84 CFM CPU Cooler
Motherboard: Asus P10S-M WS Micro ATX LGA1151
Memory: 2x Kingston 8GB DDR4-2666 C19 ECC/Unbuffered (KSM26ES8/8ME)
Boot Drive: Intel 760p 128GB M.2-2280 NVMe SSD
HDD: 7x Seagate IronWolf NAS 6TB 3.5" 7200RPM (RAID 5, so ~36TB usable)
PSU: Corsair SF 450 W 80+ Platinum Fully Modular SFX
Case: Here's the fun part: I've struck out looking for a 3U short-depth (<12") rackmount that will hold the drives. It does theoretically fit, so I'm working on a custom case design. Everything will fit, and I'm throwing in quite a few Noctua case fans to make sure everything stays cool and quiet.

So here are my questions:
- Thoughts on the parts list?
- Anyone have a recommendation for a short 3U case, or should I finish designing my own?
- Recommendations on HDD mounts for NAS use? (I'm thinking some sort of vibration dampening?)
- Should I jump to 32GB? I know the 1GB/1TB of storage rule of thumb, but half the folks here seem to say you can ignore that above 16GB...

 

[joeinaz]

FIRST: RAID 0 on two 4TB disks in your main system is a very, very, very, risky environment. If you cannot easily change that, backup the system frequently (daily if you can).

1. Do you have to buy newer hardware? You can save quite a bit of money and get similar performance with older SuperMicro boards.
2. How deep is your network rack?
3. If you have you heart set on a short depth device have you considered the 3U Case Logic SC32380 or SC3380? (if you can find them)
4. Are you willing to look at NAS solutions the in a case outside of your rack?
5. What are you doing for a storage controller? My suggestion is to look at an IBM M1015 as a key part of your solution.

Other thoughts:

The big challenges with this solution may include:

A. Space for cabling
B. Cooling (including airflow and noise)
C. Weight (a NAS with a power supply and a quantity of disk may be a bit much for a wall mount network rack)
D. Serviceability (I tend to favor solutions with removable disks and easy to access ports)
E. "RAID" level. RAID-5 (single parity) may be risky with 6TB drives due to the long rebuild time of such a large disk

Over to you Uncle Joe....

 

[PhiloEpisteme]

Right now, my only major drives are two WD Black 4TB in RAID 0 in my main computer. Here's what I'm currently looking at for my first NAS: (I was told no PCPartPicker?)

FIRST: RAID 0 on two 4TB disks in your main system is a very, very, very, risky environment. If you cannot easily change that, backup the system frequently (daily if you can).

I'm going to nit here a little and encourage folks to try to stick with zfs terminology with respect to redundancy levels. ZFS's vdev parity system is not exactly the same as the typical RAID that folks think of; though there are analogous designs but with different names. The typical RAID0's analog in ZFS is a pool composed of two vdevs each containing a single disk. This distinction becomes important when you consider that RAID1's ZFS analog is a mirror vdev but the name is VERY similar to ZFS's RAIDZ1 which is more similar to the typical RAID5. The terminology primer is super helpful here just to avoid confusion.

Memory: 2x Kingston 8GB DDR4-2666 C19 ECC/Unbuffered (KSM26ES8/8ME)

I typically recommend that folks go with the largest modules the motherboard supports. The reason is that FreeNAS likes memory; and if you decide to grow the memory of the system in the future using the largest modules possible allows you to increase your total memory without tossing any old sticks.

Boot Drive: Intel 760p 128GB M.2-2280 NVMe SSD

You can certainly go this route, but it is a waste of the speed of NVMe. If you can swing it; I'd recommend pairs of 64GB SSDs connected via SATA. If you're low on SATA ports a cheap HBA will solve that problem for you. That M.2 slot is super fast and would be well-utilized as a SLOG device if ever you need one.

HDD: 7x Seagate IronWolf NAS 6TB 3.5" 7200RPM (RAID 5, so ~36TB usable)

My two thoughts here are; 1, consider going with 5200 or 5900 rpm drives; they produce a lot less heat and the increased IOPS of the 7200rpm drives aren't worth the difference. If IOPS is a priority you'd be better off with stripes of mirror vdevs anyway. And just to repeat myself annoyingly; I think you mean RAIDZ1 rather than RAID5, yes? Do keep in mind that using RAIDZ1 exposes you to higher risk in the event one drive fails. If a second drive in that vdev fails during resilvering you'll lose your entire pool. Many folks opt for RAIDZ2 for that reason.

PSU: Corsair SF 450 W 80+ Platinum Fully Modular SFX

This may be a but under powered depending on your total expected draw. Consider that if a lot of drives are spinning up at the same time you may get close-ish to this. A higher wattage PSU also won't mean more power draw from the wall and in fact can mean less if you stick to the platinum grade PSUs since their efficiency increases as the total draw as a percentage of max supported goes down.

- Recommendations on HDD mounts for NAS use? (I'm thinking some sort of vibration dampening?)

Many enterprise disk chassis just use caddies that slide right into the chassis. I'd guess that humidity and temperature will make the biggest difference for the longevity of your devices rather than vibration, especially if you're using NAS drives.

- Should I jump to 32GB? I know the 1GB/1TB of storage rule of thumb, but half the folks here seem to say you can ignore that above 16GB...

Ditty above about going with the largest modules your board supports. You can start with 16GB and if you're performance is slow and/or your ARC hit ratio is low you can easily bump it later.

1. Do you have to buy newer hardware? You can save quite a bit of money and get similar performance with older SuperMicro boards.

Very much agree with the suggestion to go used. Used boards, chassis, cpus, memory, and HBAs are a great way to save some money and get more bang for your buck. I would personally avoid used HDDs though.

5. What are you doing for a storage controller? My suggestion is to look at an IBM M1015 as a key part of your solution.

These are great cards. There also great cards to be found on ebay such as this LSI-9211.



I know I offered a lot of feedback/advice on this build @unclejoe1116 but I'd say you're certainly on the right track. I'm curious to know what you think of the advice we've offered so far and how it may have affected your prospective build. Looking forward to updates.

 

[unclejoe1116]

Wow! Y'all definitely gave me a lot to think about, so I'm going to try my best to reply to everything:
1) I appreciate the link to the terminology, I will definitely reference that going forward. For what I currently have, my main computer is Windows-based, and I am not currently running any kind of NAS or backup system, so the two 4TB drives are in RAID0. Definitely risky, hence why I'm building the backup :) (I also have two m.2 working drives, so there is some safety...)
2) Based on the size drives I'm using, I may jump from RAIDZ1 to RAIDZ2. I could definitely see fully loaded drives failing on a rebuild.
3) My wife has been complaining about a lack of computing upstairs near the TV, so that was part of the reasoning behind the more expensive CPU and motherboard. I have a lot of ideas for future uses, including a media player, security system control head, and possibly a website server as well.
4) Do I need to get a storage controller? The Asus board has 8x SATA ports, which are rated at 6Gb/s
5) My network rack is 13.75" deep. I was planning on making the case 12" deep, to provide cable space and adequate airflow behind the case. (This rules out the Logic Case options)
6) Based on y'alls recommendations, I may jump to a single stick of 16GB RAM, since that will allow for growth to the max of 64GB.
7) Current power estimates for the system is 313W. That gives a pretty healthy margin to the 450W the PSU provides. If I can find a larger one that doesn't grow dimensionally, I'll drop it in.
8) m.2 drive was chosen simply because it's the intersection of easy, cheap, and small, making it super easy to hide the boot drive.
9) To my knowledge, the Seagate IronWolf line is only offered in 7200rpm? Would you recommend switching to WD instead? (I believe the WD Red series runs at 5400rpm)
10) Total estimated weight for the system 26-27 lbs. The wall-mount rack I have is rated at 40 lbs., so it should be good. (Currently just holds a small Cisco switch and patch panel. Will probably add a small UPS to fill the last slot, getting right at 40 lbs.)


Now, since the case seems to be attracting a bit of attention, I've included a mock-up of what I'm currently thinking. A few notes:
- The sides will be louvered, to provide exit ports for warm air
- The back will have grills along the top, so air will exit out the back as well, flowing over the drives
- The drives will be mounted to some sort of tray that can be removed easily, while also providing a limited thermal barrier between the drives and the PSU/CPU
- I'm toying with the idea of a small (single fan) AIO water loop that would fill the red square in the image. This would remove the CPU heat from the case almost completely, thus lowering ambient temperature for the drives. (Currently a low-profile Noctua fan)
- The case also allows for the potential addition of a GPU, if the home computing needs a bit of a boost. (Or any other PCIe card, for that matter. 10Gb anyone?)
- The fans on the front are only 80mm, just because that's what I found first on GrabCAD. I'll most likely put 92mm, or 120mm if I can fit them. Almost definitely Noctua, for maximum airflow, and minimum noise.

I apologize for the length. Clearly I've got a lot to learn, and these forums are a wealth of information! Thanks in advance for the shared knowledge!

 

[PhiloEpisteme]

4) Do I need to get a storage controller? The Asus board has 8x SATA ports, which are rated at 6Gb/s

Only if you run out of SATA ports or use SAS drives.

8) m.2 drive was chosen simply because it's the intersection of easy, cheap, and small, making it super easy to hide the boot drive.

Certainly true; just a shame to see a high speed port taken up by a device which would be totally happy on even the first gen SATA plug, especially when there are uses for that high speed bus which take advantage of it in many cases.

9) To my knowledge, the Seagate IronWolf line is only offered in 7200rpm? Would you recommend switching to WD instead? (I believe the WD Red series runs at 5400rpm)

It depends on the disk size etc. I wouldn't say the 7200rpm drives are an absolutely nono, just something to consider. If you go with 7200rpm drives you'll want to make sure you cool the thing well.

10) Total estimated weight for the system 26-27 lbs. The wall-mount rack I have is rated at 40 lbs., so it should be good. (Currently just holds a small Cisco switch and patch panel. Will probably add a small UPS to fill the last slot, getting right at 40 lbs.)

TBH it sounds almost like the wall-mount AV rack might not be appropriate for this use case. Many many excellent cases won't fit in there and whatever build you go with now if you decide to grow in the future it'll likely outgrow this rack and leave you with parts that don't really have a good use/use.

Though, based on your post it seems like you're a bit of an enthusiast when it comes to putting together your own builds. If designing/building your own case is something you're interested in you could certainly make it work. The most important thing, of course, is to keep the HDDs cool. The water loop may help a bit in that regard, but ensuring airflow over those drives is the most important thing. Humidity is also important in drive life; so trying to keep them in a low-humidity environment will help.

I recently read an interesting article suggesting that humidity falls as you move to the back of a chassis in cases which have intake in the front and exhaust out the back. You could consider taking this to heart and putting your mobo and psu up front with all those intake fans then then a row of HDDs in the back; if possible have all of the exhaust go out the back with fans pushing the air around the drives.

- The fans on the front are only 80mm, just because that's what I found first on GrabCAD. I'll most likely put 92mm, or 120mm if I can fit them. Almost definitely Noctua, for maximum airflow, and minimum noise.

Depending on the case you'll want to use static pressure fans to force air around the drives. The downside is they can be a bit loud.

I apologize for the length. Clearly I've got a lot to learn, and these forums are a wealth of information! Thanks in advance for the shared knowledge!

No need for apologies; I appreciate how thorough you were. Looks like a fun project.