BUILD Building a FreeNAS box

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
You finally switched from the red team to the blue team?!
I did but I have my AMD CPU and Asus MB sitting right in front of me and I'm trying to figure out what to do with it. I may just box it up and keep it in case my ESXi box breaks.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
So, as I understand it, RAIDZ2 will remove two drives.
Basically, plus some overhead, and you need to keep a minimum of 10% free space or ZFS goes into slow motion (it's a feature).

'm also under the impression that it is a better idea to set up your RAID with blank drives first rather than go with less drives, then add to the array, and wait for quite.... a.... while... as the RAID is rebuilt.
I'm not sure I follow what you are saying but you can't build a pool and have blank drives in it, the data is striped across all the drives. You also cannot just add a drive or two to the pool to expand it either without possible serious consequences. You can add "spare" drives but those do not really work like I would think a spare drive would. They basically just sit there and are a waste of money for a home system, IMO. So don't think that you can simply add a drive to expand your storage, you will hate yourself if you do that.

So, why did I pick 8x6TB drives in the first place? It gives me some sort of idea on the cost and power consumption.
Not a bad idea actually since it should give a ballpark on where you need to be. Startup current for the hard drives will be one of the key items to take into consideration. But you bought a power supply and hopefully it will perform as you expect.

I had thought I was going to also need a SSD for write caching, but based upon my learning from my first experimental box, this isn't necessary.
Nope, these extra components are great for corporate level systems but it's rare when you see someone need this in a home system. More RAM is the preferred first step and I know you are planning for 64GB RAM, even though for your planned needs 16GB is fine.

Question: Have you already purchased the motherboard and RAM? If not, hold off on buying all that RAM, start with 16GB first, but you must install it into the DIMMB2 slot (blue slot furthest away from the CPU), or maybe two 16GB RAM modules (fill both blue slots). This would save you over $160 and you could always purchase more RAM if you feel you need it. This helps soften the blow when purchasing the drives since they are not cheap. Just a thought.
 

Mark Holtz

Contributor
Joined
Feb 3, 2015
Messages
116
So don't think that you can simply add a drive to expand your storage, you will hate yourself if you do that.
Just what I thought. Don't add additional drives to an existing pool.
Question: Have you already purchased the motherboard and RAM? If not, hold off on buying all that RAM, start with 16GB first, but you must install it into the DIMMB2 slot (blue slot furthest away from the CPU), or maybe two 16GB RAM modules (fill both blue slots). This would save you over $160 and you could always purchase more RAM if you feel you need it. This helps soften the blow when purchasing the drives since they are not cheap. Just a thought.
Nope. All I have at this point is the case, and the power supply is on order. I'm slating the purchase of the motherboard, CPU, and RAM for mid-July. I have almost everything nailed down from previous research, and needed verification and validation of my homework from those who have previous experience.

Maybe I'll go with 32GB (2x16GB) initially, then bump it up to 64GB (4x16GB) later. The biggest sticker shock is the NAS drives, but I am prepared for it.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
Don't add additional drives to an existing pool.
It's more complicated than that. You can absolutely add additional drives to an existing pool, and do it safely, but they need to be added in the form of a suitably-redundant vdev. So if I have six 3 TB drives in RAIDZ2, I can add six more drives in another RAIDZ2 vdev to the pool. My pool will then consist of twelve drives in two, six-disk vdevs. But you can't add a single disk to turn a four-disk RAIDZ2 array into a five-disk RAIDZ2 array.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
It's more complicated than that. You can absolutely add additional drives to an existing pool, and do it safely, but they need to be added in the form of a suitably-redundant vdev. So if I have six 3 TB drives in RAIDZ2, I can add six more drives in another RAIDZ2 vdev to the pool. My pool will then consist of twelve drives in two, six-disk vdevs. But you can't add a single disk to turn a four-disk RAIDZ2 array into a five-disk RAIDZ2 array.
What he said.
 

maglin

Patron
Joined
Jun 20, 2015
Messages
299
SATA DOM uses little power and takes up virtually no room. They are $50 each for the SuperMicro ones.


Sent from my iPhone using Tapatalk
 

Mark Holtz

Contributor
Joined
Feb 3, 2015
Messages
116
The parts arrived over the weekend, with the critical Motherboard and Memory arriving yesterday at work. After a Toastmasters meeting this morning, I have assembled the box, and am currently running the Memtest86. So far, over an hour in and passing. Hopefully, the tests will complete when I get home from work tomorrow morning.

I also have the m.2 drive installed, but it is sitting there.... idle.... pending install of FreeNas

I'm still putting in the EVO. I don't like the design of Intel's cooler.

The case is an excellent case. As noted in several reviews, there is a switch in the back that would allow you to hook up the three fans and control the fan speed. As the motherboard has a CPU fan header plus four fan headers, I removed the cables to that switch as it is not needed. The standoffs for the motherboard are a slight problem to screw in, however, Fractal Designs included the tool to make screwing those standoffs easier.

I don't think this motherboard comes with IPMI.
 
Last edited:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Which is which, and how do I tell the difference?
By reading the manual. Or tracing the connections running from each socket. The larger chip is the i210.

The i210 is "better", but you almost certainly wouldn't notice a difference.
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Hopefully, the tests will complete when I get home from work
One of the reasons to run a memory test is to stress the system. Completing a pass without errors is not the only goal. Leaving it running for several days is frequently recommended in these forums.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
One of the reasons to run a memory test is to stress the system. Completing a pass without errors is not the only goal. Leaving it running for several days is frequently recommended in these forums.
It is quite usual for intermittent errors to be missed in a single pass, too.
 

Mark Holtz

Contributor
Joined
Feb 3, 2015
Messages
116
One of the reasons to run a memory test is to stress the system. Completing a pass without errors is not the only goal. Leaving it running for several days is frequently recommended in these forums.
I understand what you are saying here. However, this is only phase one of the build. I will be temporarily running with a single 2TB drive and still playing around with FreeNas for about a month or three until I can purchase the necessary eight NAS drives.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
I understand what you are saying here. However, this is only phase one of the build. I will be temporarily running with a single 2TB drive and still playing around with FreeNas for about a month or three until I can purchase the necessary eight NAS drives.
Just having the system operate is not the same as running the MemTest86 on it. Lets say you get one full pass and then build up your system, well I'd still go ahead and run MemTest86 for a few more days, and also you should run a CPU stress test for about 30 minutes or so. Everyone has an opinion on how long a CPU stress test should run, mine is 30 minutes. If your system craps out during this test then post those results.

Also, larger CPU coolers place more stress on the motherboard and mounting assembly, especially when you set the computer down a bit hard or just transport it from one place to another. You will do what you desire, that's fine, I just wanted to let you know that the boxed CPU cooler is nice and light and does the job very well.
 

Mark Holtz

Contributor
Joined
Feb 3, 2015
Messages
116
Just having the system operate is not the same as running the MemTest86 on it. Lets say you get one full pass and then build up your system, well I'd still go ahead and run MemTest86 for a few more days, and also you should run a CPU stress test for about 30 minutes or so. Everyone has an opinion on how long a CPU stress test should run, mine is 30 minutes. If your system craps out during this test then post those results.

I had Memtest86 go through three passes in a 24 hour period without any issues. I believe that Memtest86 v6.3 also tested across all eight threads/four cores of the Xeon Processor. Based upon my testing for my own personal use, I am satisfied with the results. Would it benefit from further testing? Yes, it would. I'm backed into a corner though because I believe in backups for my systems.

Previously, I was testing out FreeNAS on a old Dell OptiPlex 755 with a Intel Core 2 Duo E6850 on a experimental basis. It was a old work box with an old single 1TB hard drive and 8GB of non-ECC memory (out of 8GB) available. Anyone who is somewhat experienced with FreeNAS builds can name a dozen things wrong with this configuration. However, it was "good enough" for me to put automated weekly whole drive backups on from Acronis True Image. It was "good enough" to actually learn how to use FreeNAS and get some experience so that I know what to expect. It was, however, not "good enough" to store non-replaceable data because of the known limitations.

I had planned on doing this build at the end of the year, not now. It got accelerated because the "experimental" box had a fan failure in late April. Not of the hard drive, not of the memory, but because of a stupid fan that was mounted in the front that would have cost more in effort that it was worth to disassemble and repair. From late-April until now, I have not had regular backups of my main computer, my personal laptop, my multimedia computer, or my mother's laptop. Any backups (haha) involved hooking up a portable hard drive. It is much easier to automate the backups to a NAS drive, then copy the data to a external drive every month. Because of that "major issue" (for me),

However, even though the memory, processor, case, and m.2 boot drive is in a finalized state, I am running with a old single 2TB drive from 2009 because I am unable to afford the drives that I want. There is no RAID redundency. Because of that single point of failure, I still classify my FreeNAS setup as "experimental". But, it's a personal NAS, and it is "good enough" for what I need to do right now.

Also, larger CPU coolers place more stress on the motherboard and mounting assembly, especially when you set the computer down a bit hard or just transport it from one place to another. You will do what you desire, that's fine, I just wanted to let you know that the boxed CPU cooler is nice and light and does the job very well.
With all due respect.... once the final assembly takes place, the FreeNAS server will be sitting in the corner collecting dust. I prefer the larger heatsink because if the fan fails, I still have an allowance for passive cooling.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
With all due respect.... once the final assembly takes place, the FreeNAS server will be sitting in the corner collecting dust. I prefer the larger heatsink because if the fan fails, I still have an allowance for passive cooling.
You do realize that the system shuts down if the CPU overheats?
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
Actually it will depend on how your system is setup. When the CPU starts to overheat it will throttle it's speed and that is built into the CPU itself so that it doesn't self-destruct because of a fan problem. You can have the motherboard force the system to shutdown but I wouldn't recommend that if you are storing critical data on it, but if you set the temp setting up to it's maximum threshold and use it as a fail-safe then it's better to have your system come back alive than completely dead. If you have FreeNAS setup to shutdown based on a high CPU temp, you can have it shutdown gracefully. I can't speak from experience on the CPU shutdown setup needed for FreeNAS to shutdown but I'm certain it can be done provided the CPU temp can be sensed by FreeNAS (I'm thinking a script).
 
Top