Build advice

gdarends

Explorer
Joined
Jan 20, 2015
Messages
72
unless you plan to build several pools/VDEVs, you are best to start out with the correct number of hard drives up front. You need to know the capacity you need for data storage for the next 5 years (most drives will last about 5 years), and then double that capacity if you are a storage hog. So you have capacity now. Next you need redundancy, how many drives can fail before you lose all your data? Many of us use RAIDZ2 and two drives can fail. Next, how many drives do you want to have in your system? Some folks are limited by case size or other factors. I'd recommend using a RAID-Z calculator (like one of my links below) to play around with the numbers. Maybe you only need 5TB and RAIDZ2, then you could see if it's 4 drives, 5 drives, or even 6 smaller drives. Maybe you want 4 drives so you can space them out and keep them cool (that is what I do now).

Pro's and Con's to think about: Smaller physical drives resilver faster. Many smaller drives crammed together get hotter. Larger drives are generally cheaper for capacity per dollar. To upgrade to a larger capacity you need to replace all the drives, it's not incremental, more drives could cost much more money. If you want to change the number of drives in your system, yo need to destroy your pool and rebuild it.

I really like TrueNAS (I started when is first came out as FreeNAS) and I have no complaints. You get what you put into it. It sounds like you are going down the right path, getting a good quality motherboard, putting in 32GB RAM (Question: is that a single stick or two sticks? I would go for two 16GB sticks so you run in Interleave mode, and if one should fail, you still have the other to run the system.) Some folks really love that 804 case but myself, it's too small. You are cramming a lot into it. If this is the path you go down, when you put it together just think Air Flow! Use zip ties to bundle the cables nicely, if you need to add more fans or better fans, do so (you may not know this until after it's together), and keep it in a cool place. Check those system and hard drive temps. If you are running near 50C, that is too hot in my opinion to continuously run. You can look in my description for my case and you will see a fair sized full case, lots of airflow, cool drives. And go figure, it's quiet too.

Mixing drives from different placed, manufacturers, lots, etc was a thing done more in the past. Drive manufacturers had a period of time that things were just not going well. These days most manufacturers are doing better, just stay away from Archive/SMR drives.

Best of luck to you.
Thanks for the feedback.

My current QNAP has about 5.5TB usable space and I'm using about 60% of that. I do purge a lot of non essential Plex media off of it regularly.
I did use a ZFS calculator and I came to 14.5 TB of usable storage but with a practical usable capacity of about 11.2TB. So I'm more than happy with that, and it should last me a while.

The 804 case has space for 10 HDD so I will be spreading it around a bit to reduce heat because the server won't be in a cool space and yes, I was planning on setting up everything and then see if I need to get more fans. The case supports lots of fans, so that was the idea. I'm already looking into fans that are good but silent. I was mainly looking into Noctua, Arctic and be quiet fans.

I thought SMR drives were bad and CMR were good?

Edit: And yes, I wen't with 2x16GB sticks.
 
Last edited by a moderator:

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,134
The 804 case has space for 10 HDD so I will be spreading it around a bit to reduce heat because the server won't be in a cool space and yes, I was planning on setting up everything and then see if I need to get more fans. The case supports lots of fans, so that was the idea. I'm already looking into fans that are good but silent. I was mainly looking into Noctua, Arctic and be quiet fans.
You can't go wrong with Noctua fans… but bear in mind that HDDs make noise, and that the mesh top of the Node 804, which is great for cooling, lets all the buzzing and humming noise out. Fan noise is not the major concern here, so you may as well keep the stock Fractal fans and/or go for quiet-for-cheaper-than-Noctua if you do add more fans (two input fans in the HDD chamber is a good idea; two input fans in the motherboard chamber is only useful is there are drives on the floor).
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,994
I thought SMR drives were bad and CMR were good?
Sorry, I did a typo, don't know what I was thinking. Of course SMR drives are bad for ZFS and CMR drives are good.
Edit: And yes, I wen't with 2x16GB sticks.
Good to hear.
The 804 case has space for 10 HDD so I will be spreading it around a bit to reduce heat because the server won't be in a cool space and yes, I was planning on setting up everything and then see if I need to get more fans.
Due to the case design, you might be able to just use the fans provided in order to push all the air through to cool the hard drives and keep the motherboard voltage regulators cool. I have found that using a smoke stick helps to visually verify air flow, but some people might think that is overkill, I guess it depends on your point of view. I want the minimum number of fans I need, spinning the slowest they can reasonably go. This is why I run my fans at 7VDC. None run at 12VDC except the CPU fan, and even then it's regulated by the motherboard so it's quiet 99.9% of the time, rebooting is the only time I actually hear it.

I've used many different fans, the Be Quiet Silent 2 fans for case fans and are nice, and I even have a few Noctua fans but those are for CPU's. When I evaluate a fan I check RPM, CFM, Thickness, Sound Level (normally at full RPM), and of course Voltage and Current. The thicker the fan typically means more CFM so I could run it at a lower voltage level and get the same CFM as a thinner fan with very little noise. The blade shape relates a lot to sound level as well. And if my case will support a larger fan (140mm, 120mm) over an 80mm fan, I definitely will go that route.

When you are done, I think you will have a very nice system that will last you a long time.

Last thing since I mentioned lasting you a long time, make sure you have a good UPS. You will want the system to automatically shutdown during a prolonged power failure.
 

gdarends

Explorer
Joined
Jan 20, 2015
Messages
72
Last thing since I mentioned lasting you a long time, make sure you have a good UPS. You will want the system to automatically shutdown during a prolonged power failure.

I've been browsing a bit for a UPS. I was interested in a lithium ion UPS but they are expensive. I was planning on setting up the system first and then see how much watt the system pulls and then go find one with the right capacity.

Update: I did a rough estimate and came out at a peak of ~223W.

Motherboard 80W
CPU 62W
CPU Cooler 1W
RAM 6W
Boot Drive 3W
Chassis/Case Fans 3W
Storage 68W
Total 223W
 
Last edited:

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,994
Update: I did a rough estimate and came out at a peak of ~223W.
That would be the maximum under full load. And what is the purpose of the NAS?

I tend to like a 1500VA UPS, it gives you lots of room to spare and they are small these days. That is all I buy now for my computers. I have one on my main computer right now, two monitors, high end graphics card, several hard drives. 48 minutes run time. If I were to lose power I would bank on 30 minutes tops. But lots of run time to shut the system down after 5 minutes if power doesn't return.
 

gdarends

Explorer
Joined
Jan 20, 2015
Messages
72
That would be the maximum under full load. And what is the purpose of the NAS?

I tend to like a 1500VA UPS, it gives you lots of room to spare and they are small these days. That is all I buy now for my computers. I have one on my main computer right now, two monitors, high end graphics card, several hard drives. 48 minutes run time. If I were to lose power I would bank on 30 minutes tops. But lots of run time to shut the system down after 5 minutes if power doesn't return.

Purpose is mainly file storage/backup with Plex and some other services. I think I mentioned all the services in the OP.

I was looking at 1000VA UPSes, but will be doing tests when I setup the server to be able to calculate running time better and then choose the right size.

I've placed the orders for all the parts. Hoping to get everything by next week.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,994
I've placed the orders for all the parts. Hoping to get everything by next week.
Make sure that you do proper Burn In testing to ensure your components are rock solid. These are stress tests for the Motherboard, CPU, and RAM, some folks like to toast the system for a good 24 hours or longer. They should be able to support the heat without failing. If they fail, send it back for a replacement while you can. Also there is the hard drive testing as well and that can take a while.

Good luck on your NAS, I hope it all works out flawlessly for you.
 

gdarends

Explorer
Joined
Jan 20, 2015
Messages
72
Make sure that you do proper Burn In testing to ensure your components are rock solid. These are stress tests for the Motherboard, CPU, and RAM, some folks like to toast the system for a good 24 hours or longer. They should be able to support the heat without failing. If they fail, send it back for a replacement while you can. Also there is the hard drive testing as well and that can take a while.

Good luck on your NAS, I hope it all works out flawlessly for you.
I have these on the list to go through...



Anything I missed?
 
Last edited:

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,994
Anything I missed?
Nope. Prime95 (the one I use) is very good, some folks will run it for an hour and call it good, some folks will run it for 24 hour or even longer to ensure the CPU and motherboard are heat saturated, and when the system MUST be verified stable. I run it for a few hours personally but have left it running overnight once just for the heck of it, and no issues. MemTest86 is also a very good one, I like to enable either it's called All Cores or All Threads (think it's cores) which will of course make the test run faster but it's in my opinion a better stress test. Make sure it runs at least 24 hours or I like at a minimum 3 full complete test runs, meaning not just three patterns but three complete sets of patterns. Typically I will run through at least 5 sets of patterns or 3 days, whichever comes first (if you have a huge amount of RAM, I'd run 5 sets even if it takes a long time because if you have that much RAM, you are probably working on a business project).

These two tests alone really test your CPU, RAM, Motherboard, and Power Supply (if all your hard drives are also connected pulling a current draw).

You can kick off a SMART LONG/EXTENDED Test on all your drives before you kickoff the MemTest86 as well. The drives only require uninterrupted power once the test starts, it's an internal drive test. Or just take it one test at a time. There is no rush.

Good luck
 

gdarends

Explorer
Joined
Jan 20, 2015
Messages
72
Hi all. Just wanted to give an update.

I finally received all parts friday, and I finished the build. I decided to install TrueNAS Scale.
I have yet to transfer the 2 HDD's from my previous NAS to the new one.

These are the changes from the OP:
2x Supermicro Certified MEM-DR416L-HL01-EU26 Hynix 16GB DDR4-2666 ECC UDIMM
ARCTIC Freezer 34 CO
4x HGST Ultrastar HUS724040ALE640 4TB 64MB 7200RPM SATA 6Gb/s

I did have some issues with the build that I overlooked.
The PSU only provides 2 x 3 SATA power connectors. It was just enough for the 6 HDD's that I will be installing, but has no room for upgrade later on. Also the case uses a SATA power connector which has a switch to control the speed to power the 3 fans, but the PSU has no SATA connectors left since they all are used for the HDD's. There is a "Perif" connector, but the cable is molex. I wonder if I could just buy a SATA cable and use that instead?
I opted to connect the case fans to the motherboard connectors instead, but the fans are not PWM. I think I will upgrade these to Noctua PWM fans.

I have been running stress test since friday midnight.
I ran memtest twice (you can only do 4 passes at a time with the free version). So I did 8 passes and it took about 10.5hrs.
I'm currently running prime95. It's been running for about 6hrs (I have it set to run 24hrs).
The highest the CPU temp got was around 72C. This was around midday, the hottest time of the day (the weather has been crazy hot this summer) and the machine is not in a cooled environment.

Here are some pictures of the build. Haven't done any cable management yet.

1.jpg

2.jpg

3.jpg

4.jpg

5.jpg
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,994
I wonder if I could just buy a SATA cable and use that instead?
Yes, there are Molex to SATA adapter cables, they work great. Once your system has been burned in, I would try to switch the case fans to low speed and monitor temperatures of the hard drives (since you have 7200 RPM drives) and any temperatures you can derive from the motherboard. The key is a constant flow of air across the components. Burning in is harsh and odds are your system will never feel those extremes again.

It looks like a very nice build, I hope it all works out when you are done and it's exactly what you desire. It's too bad the left front panel looks like it got scratched up.
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
Consider adding a fan or two blowing to the HDD "rack".
Besides that it looks really good.
 

gdarends

Explorer
Joined
Jan 20, 2015
Messages
72
Yes, there are Molex to SATA adapter cables, they work great.
I was thinking of getting a new cable, but an adapter is simpler. Will do that!

Once your system has been burned in, I would try to switch the case fans to low speed and monitor temperatures of the hard drives (since you have 7200 RPM drives) and any temperatures you can derive from the motherboard. The key is a constant flow of air across the components.
At the default speed, the CPU fan would fall to around 300rpm and I would get a bunch of alerts about the fan. That's why I wanna get PWM fans.
When I install all the HDDs, I was planning on running tests again. Also haven't done HDD tests yet. That will be run when I get all the drives installed.

1,Information,2022/08/27 12:01:02,Fan(FAN1),Lower Non-recoverable - going low - Deassertion
2,Information,2022/08/27 12:01:02,Fan(FAN1),Lower Critical - going low - Deassertion

It's too bad the left front panel looks like it got scratched up.
That's just some lint. There are no scratches. :smile:

Consider adding a fan or two blowing to the HDD "rack".
Besides that it looks really good.
There is one fan blowing directly on the HDD rack. I might add an extra, depending on how it performs later.
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
At the default speed, the CPU fan would fall to around 300rpm and I would get a bunch of alerts about the fan. That's why I wanna get PWM fans.
This might help.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,994
There is one fan blowing directly on the HDD rack. I might add an extra, depending on how it performs later.
Make sure you ensure airflow goes across all the drives. Even slow properly directed airflow will remove heat better than a fast chaotic airflow.

Glad the case is in good condition.

At the default speed, the CPU fan would fall to around 300rpm
I'm surprised there isn't a setting in the BIOS for that. I understand that you may replace the fan with a PWM fan, but understand that you should compare the CFM and RPM ratings of your current fan to the new fan. You do not want to install a lower rated fan if you can help it.
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
It's a supermicro board.
In mine there is nothing fan-related in BIOS (I couldn't even find and option to switch from PWM to DC), and the fan profile can be choosen by IPMI.
 

gdarends

Explorer
Joined
Jan 20, 2015
Messages
72
Prime95 ran for 24hrs without a hitch.
Had a bit of a hick up with porting over the drives from the old QNAP system. Couldn't find the keys for the hot-swap drive bays :eek:
I tried the numerous ways to pick the lock, but wasn't successful. So I used pliers to bend the metal frame that was responsible for locking the drives :grin:. The old system is now probably unusable, oh well, it had a nice 14 years of service.
Then I couldn't create the vdev, because of the 2 drives that were transfered from the the old system. I had to use "wipefs -af" to get them in a usable state. It is now configured as RAIDZ-2, with 14.5TB of usable space.

I will do HDD stress test next.

I also didn't install the hot spare. I wanted to mount it at the floor of the case in the motherboard compartment. There are 2 spots there for 3.5 or 2.5 drives, but it was touching some jumper pins on the motherboard (in both locations), so now the drive will be a cold spare.


This might help.
Thanks!

Make sure you ensure airflow goes across all the drives. Even slow properly directed airflow will remove heat better than a fast chaotic airflow.
The fan points right at the HDDs.

PXL_20220826_221856496.jpg
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,994
The fan looks to be off center to the hard drives, but that is just how the photo looks, could be very different in reality. Just make sure all those 7200 RPM drives get some much needed airflow.

It looks like the fan there could be replaced by a larger one, is that true? If so, I'd install a larger fan. Also, I'd cut out the entire grill structure and install a nice wire fan grill in order to promote better airflow. I dislike the small holes in the sheet metal grill, it offers too much resistance. but this would require cutting the case, not a fun task after spending this kind of money, andy you may need to remove everything before hand. I'd like to know how large of a fan this case would handle in the hard drive bay.

Cheers,
-Joe
 

gdarends

Explorer
Joined
Jan 20, 2015
Messages
72
The fan looks to be off center to the hard drives, but that is just how the photo looks, could be very different in reality. Just make sure all those 7200 RPM drives get some much needed airflow.

It looks like the fan there could be replaced by a larger one, is that true? If so, I'd install a larger fan. Also, I'd cut out the entire grill structure and install a nice wire fan grill in order to promote better airflow. I dislike the small holes in the sheet metal grill, it offers too much resistance. but this would require cutting the case, not a fun task after spending this kind of money, andy you may need to remove everything before hand. I'd like to know how large of a fan this case would handle in the hard drive bay.

Cheers,
-Joe
Yes, that particular position can be upgraded up to 140mm fan. I can still add more fans, but will analyze and then add if needed.

Screen Shot 2022-08-29 at 16.46.29.png

2022-08-29-165123_003.jpeg

2022-08-29-165123_004.jpeg
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
Top