BUILD Building a FreeNAS box

Mark Holtz

Contributor
Joined
Feb 3, 2015
Messages
116
So, after a year of playing around with FreeNAS on a old computer (a Dell OptiPlex 755 with a Intel Core 2 Duo E6850), I'm about ready to build a real FreeNAS box for personal use. The usage will be:
  • Centralized storage for system backups from several home systems using True Image
  • File storage
  • Media server for DLNA
  • Transmission download
Plus, of course, a few others...

  • I'm going to have to build this in two phases... one to meeting the immediate need with a temporary 2TB drive, and then get the additional drives later.

Already purchased
Phase I Assembly
Phase II Assembly
Changes from original posting
  • Power supply changed from ETL450AWT-M. Mo' power.
  • Power supply purchased 5/29. It was on sale, plus a rebate. I can dig that.
  • Processor changed from 3.6GHz to 3.5GHz.
  • Added M.2 drive as boot drive.
  • Motherboard, processor, memory, and M.2 drive purchased.
  • Hard drives changed from eight 6GB drives.
 
Last edited:

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
Hello
Typically there is no benefit to get a higher clocked model within a processor family to achieve the argument you provide.
When a E3-1230v5 won't cut it, you're not likely to have a problem on your hands that E3-1275v5 would solve immensely better.
However, differences between for example a G4400, i3-6100 and E3-1230v5 would be substantial to say the least.
Your call. I'm just putting forward this for you to at least make a more informed decision about your overkill - since it probably won't provide you with the benefits you hope for.
Eight 6GB NAS Drives
I am calculating about 300W actual usage, 450W should be more than plenty.
You should probably recalculate that following the stickies in the hardware section on power calculations.


Cheers /
 
Last edited:

religiouslyconfused

Contributor
Joined
Dec 14, 2015
Messages
184
So, after a year of playing around with FreeNAS on a old computer, I'm about ready to build a real FreeNAS box for personal use. The usage will be:
  • Centralized storage for system backups from several home systems using True Image
  • File storage
  • Media server for DLNA
  • Transmission download
Plus, of course, a few others...

I'm going to have to build this in two phases... one to meeting the immediate need with a temporary 2TB drive, and then get the additional drives later.

Already purchased
Phase I Assembly
Phase II Assembly
  • Eight 6GB NAS Drives - This should more than enough storage.
  • m.2 PCI Express drive - To be used as a boot drive, and thus avoid a USB drive. Any recommendations?

FreeNAS 9.10 supports USB on Skylake so no need for NVMe. I would second the power supply calculator, but if you really need a good CPU, then check out the Xeon E5 platform. More expensive and will likely need a bigger power supply, but you can support much more RAM and use RDIMM. I would probably stay on E3 but use something like a 1240 or something like that as the internal GPU is not supported in FreeNAS.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,457
It's overpowered as you say, but it also includes a built-in GPU that won't do anything for you. If it costs the same or less than a non-GPU chip, it won't hurt anything, but it'd be a waste to spend extra on it.
If I understand correctly, for each TB of hard drive, 1 GB of memory required
That's a general, and vague, rule of thumb, and it's left deliberately vague whether it refers to raw HDD capacity, or net usable storage on your pool. Also depends greatly on your use case and a number of other factors. The more RAM the better, but odds are that 16-32 GB would be fine for what you're doing.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
Since I have plans for 8x6GB drives, that means I will need at least 48GB of memory.
No. For the things you described as what you want to do with FreeNAS, 16GB RAM would be just fine however if you are building a new system, I'd shoot for 32GB RAM .

Overkill and has the GPU as previously mentioned. I use an E3-1230 V5 and it is overkill but my system is more than just FreeNAS. The E3-1230 V5 is a good option but you could purchase something cheaper as well, again only taking into account the applications you are looking to use.

Use the stock box cooler, it's very quiet and works well. There is no need for a huge a$$ heatsink these days for a system like this.

I used the following RAM which is on the recommended list (double-check me) and could be cheaper: http://www.superbiiz.com/detail.php?name=D416GE21S
 

Mark Holtz

Contributor
Joined
Feb 3, 2015
Messages
116
Typically there is no benefit to get a higher clocked model within a processor family to achieve the argument you provide.
When a E3-1230v5 won't cut it, you're not likely to have a problem on your hands that E3-1275v5 would solve immensely better.
However, differences between for example a G4400, i3-6100 and E3-1230v5 would be substantial to say the least.
Your call. I'm just putting forward this for you to at least make a more informed decision about your overkill - since it probably won't provide you with the benefits you hope for.
In my building philosophy, the motherboard and CPU are essentially the heart of the computer. As has been emphasized numerous times in this forum, "you are building a server, not a gaming box, damnit". I selected the motherboard because it has eight SATA ports and one CPU socket, then selected the CPU from there. 64GB of memory is a bonus. And, as I understand it, if I want to use ECC (which is highly recommended), I need to use a Xeon CPU.

Also, I tend to go overkill with new system build for myself because I want the system to last at least five years (give or take). Considering that I am building what can be essentially considered a "big bleeping hard drive with redundency", it's safe to say that I am building something with a even longer lifetime (I'm shooting for ten years), stick it in a closet with a UPS, and let it do it's thing.. The experimental FreeNAS box was at least six years ago, and the only gripe was that the web interface for Transmission took a while to bring up. My guess is that it was a combination of a old Core 2 Duo processor, 8 GB of memory, and 1TB hard drive.

You should probably recalculate that following the stickies in the hardware section on power calculations.
Shrug. The price difference between the 450W and 650W model of the same power supply is $10-$20.

FreeNAS 9.10 supports USB on Skylake so no need for NVMe.
That's one thing that has me scratching my head. How much is the USB accessed while FreeNAS is in use?

It's overpowered as you say, but it also includes a built-in GPU that won't do anything for you. If it costs the same or less than a non-GPU chip, it won't hurt anything, but it'd be a waste to spend extra on it.
The price difference between the non-GPU CPU and the GPU CPU is.... $5. I might as well spend the extra $5 for the five minutes that I will be interfacing directly with the CLI. :(

That's a general, and vague, rule of thumb, and it's left deliberately vague whether it refers to raw HDD capacity, or net usable storage on your pool. Also depends greatly on your use case and a number of other factors. The more RAM the better, but odds are that 16-32 GB would be fine for what you're doing.
That is something that has me scratching my head. From my desktop point of view, sure, more memory is better. However, after a point, it becomes a point of diminishing returns. You will see an great improvement when going from 4GB to 8GB, some improvement when going from 8GB to 16GB, and maybe minor improvement wheil going from 16GB to 32GB unless you are doing virtualization or video editing.

FreeNAS is a server, and for me, a bit of a different critter. Thus, I ask what I hope are intelligent questions, and adjust what I do with the answers provided.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,457
if I want to use ECC (which is highly recommended), I need to use a Xeon CPU.
ECC is highly recommended, but you do not understand correctly that a Xeon is required in order to use it. Many Pentiums (Pentia?) and i3s support ECC as well.

The RAM usage is what it is because of ZFS, which will use whatever RAM it can get its hands on for caching. My server, as you see in my .sig, has 128 GB of RAM (which is gross overkill for my needs, but it came with the system), and ZFS will use over 100 GB for the read cache if it can (which it can most of the time). The requirement for 8 GB is a pretty hard minimum--sure, the system will (usually) boot and run with less than that, but there have been plenty of cases of system instability, and even lost data, with < 8 GB. Above 8 GB, 1 GB / TB is a rough rule of thumb, but for most home users, 16-32 GB is fine for any reasonable amount of storage.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
That's one thing that has me scratching my head. How much is the USB accessed while FreeNAS is in use?
Not much. Updates are brutal by USB flash drive standards, though. Good ones will cope and possibly be slow as molasses, crappy ones will die.

I might as well spend the extra $5 for the five minutes that I will be interfacing directly with the CLI. :(
Not even then, the motherboard's graphics are courtesy of the ASpeed 2400 BMC. Might help resale value or reusability, though.

That is something that has me scratching my head. From my desktop point of view, sure, more memory is better. However, after a point, it becomes a point of diminishing returns. You will see an great improvement when going from 4GB to 8GB, some improvement when going from 8GB to 16GB, and maybe minor improvement wheil going from 16GB to 32GB unless you are doing virtualization or video editing.
You're absolutely right. 16GB is the sweet spot for most home systems, but 32GB is awfully cheap these days. I generally recommend starting with 16GB and then adding more RAM if and as needed, since it's a trivial upgrade (as long as you stick to 16GB DIMMs).

ECC is highly recommended, but you do not understand correctly that a Xeon is required in order to use it. Many Pentiums (Pentia?) and i3s support ECC as well.
The Pentiums are appropriate for most workloads at home. As are the Pentiums, but I'm partial to the extra L3 cache that the i3-6300 has.

Many Pentiums (Pentia?)
I must point out that it's all about the Pentiums.
 

Mark Holtz

Contributor
Joined
Feb 3, 2015
Messages
116
So, do I call you danb35 or Kosh? At least you don't speak like a Vorlon.

ECC is highly recommended, but you do not understand correctly that a Xeon is required in order to use it. Many Pentiums (Pentia?) and i3s support ECC as well.
You are right. i3 supports ECC, i5 and i7 do not.

The RAM usage is what it is because of ZFS, which will use whatever RAM it can get its hands on for caching. The requirement for 8 GB is a pretty hard minimum--sure, the system will (usually) boot and run with less than that, but there have been plenty of cases of system instability, and even lost data, with < 8 GB. Above 8 GB, 1 GB / TB is a rough rule of thumb, but for most home users, 16-32 GB is fine for any reasonable amount of storage.
The big gotcha that I have seen is that running a Plex server requires 12GB. Not sure about the Emby alternative.
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
The big gotcha that I have seen is that running a Plex server requires 12GB. Not sure about the Emby alternative.
Plex by itself does not require 12GB. I think the recommendation you have seen is that if you are only running a Plex server then you need at least 8GB for OS and 4GB for Plex, which totals 12. I have never seen any requirement that says Plex needs 12GB.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,457
Impudent!

Plex doesn't require 12 GB on its own, but a system running Plex should have at least that much. Better would be 16 GB or more.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
m.2 PCI Express drive - To be used as a boot drive, and thus avoid a USB drive. Any recommendations?
You need to be smart if you plan to use this as a boot device. This is not an M.2 SATA and you need to purchase a true M.2 PCI card interface, pay attention to the interface if you buy this device. Also, I don't know if anyone here has experience with this as a boot device. I would expect it to work fine but without personally testing it, who knows.

As for USB flash drives as a boot device, buy a good quality USB 2.0 device. Many people like to buy those tiny USB 3.0 devices but many have a flaw. They are not designed for continuous use and run hot, thus they fail early. If you must have a USB 3.0 device, get one that has a metal shell and is at least 1.5" or longer (longer is better) so it can dissipate the heat.
 

Mark Holtz

Contributor
Joined
Feb 3, 2015
Messages
116
NewEgg had a sale on the 650W power supply. I pulled the trigger. Yes, I realize that it is Bronze power supply instead of the recommended Gold power supply.

So, lets see here.... here is where I got some of my info.... from Hardware recommendations (read this first) (which needs some updating for the X11 and the E3V5 Xeons....
It might have all sorts of great features that are useful for desktops but we aren't building a desktop. We're building a server and we're looking for reliability and high uptime.
On the i3 CPUs...
Per this thread the 2nd Generation and 3rd generation i3s do not have ECC support (there are a few exceptions, but they are so expensive that you'd be better off buying a Xeon at the prices the i3s are selling at). The entries on the Intel ARK were in error and I know that many of you bought i3s expecting ECC support. Sorry, but you'll have to get a new CPU if you want to use ECC as it appears you probably never were. The *only* i3s that support ECC and are reasonably priced are Haswell i3 CPUs.
On memory...
ZFS loves to use memory. The more you have the faster it will be. If performance starts slacking that's your queue to add more RAM. 8GB is the minimum for FreeNAS and do not go below that. You aren't as special as your mommy told you and you risk your data if you think you are.

For most home users 16GB is a very good sweet spot. If you plan to run lots of jails like Plex or Minecraft you should consider going with 32GB of RAM. It is better to have too much than too little.
On USB as boot device...
USB devices are slowly being phased out because of all of the problems they can have with FreeNAS 9.3.0 and newer. USB works, bootup times are about the same as your typical fast SSD, but their reliability is lower.
If you want to do some forward thinking consider a SATA DOM or a really small SSD (even 30GB of disk space is seriously oversized).
And on Plex...
Try to use less than 8GB of RAM (If you plan to use Plex you should have at least 12GB of RAM)
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
Also, I tend to go overkill with new system build for myself because I want the system to last at least five years (give or take).
I've followed a very similar logic to yours for a long time too. I agree it's fun, and provides you with that extra 'coziness' of having real neat hardware.
However, not until quite recently I was pulled out of that bubble of adding the extra cost up front to the CPU (within a CPU family). Now I regard CPU options as one "per family". G4400, i3-6100, E3-1230v5 - for 1151 socket.

Problem is, at some point false dreams should be shattered.
Spending an additional 25-30% CPU cash, on top of a E3-1230v5 is ...just not going to give you the additional years of performance you are looking for.

I suggest you reconsider again. Here are two options that I see fit:
-E3-1230v5 <for all the processing power you'd "experience" for the buck
-Put the cash into more RAM, perhaps an additional 16GB stick.
-Save that extra ~100USD (my local price difference) and spend them in X years - into your next machine.

If those $100 are worth spending for the <mere feeling of having a killer machine> then so be it. Just don't confuse those bucks with 'giving additional year(s) of performance. Because they don't.

On the i3 CPUs...
"Per this thread the 2nd Generation and 3rd generation i3s do not have ECC support"

The current relevant i3-6100 is from the 6th generation. Which has not shown any doubts in regards to ECC. capability. See here.
The hardware recommendation thread have not yet included Skylake (1151).

Cheers /
 
Last edited:

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
NewEgg had a sale on the 650W power supply. I pulled the trigger. Yes, I realize that it is Bronze power supply instead of the recommended Gold power supply.
Bronze vs. Gold, no big deal for the system you plan to build. If you were looking to really make a power miser of a system then you would have obtained all the power specs for each component and done some serious calculations and then find the appropriate sized power supply (supplying proper amps of power for each voltage level) for the task. I personally think 650 watts is a bit high, assuming it was a true quality 650 watt power supply, but if the sale is right... Some power supplies are rated a bit high and have unclean power when running at the upper limits.

So, lets see here.... here is where I got some of my info.... from Hardware recommendations (read this first) (which needs some updating for the X11 and the E3V5 Xeons....
Very true and also remember that this is only a guideline, not gospel.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
As for using a USB device as a boot device, I personally am against it but you may be in a situation where either you use USB, the M.2 port (making sure you buy a proper card that will also boot your system), or reduce your total number of hard drives so you have an available SATA port, or you could buy a different motherboard with more SATA/SAS ports, or an HBA card. If I were in your shoes, I'd go with reducing the total number of hard drives and leave an SATA port open for a SSD boot device, but my storage requirements are not as high as you feel yours might be.

Lets do some planning because the way you made a statement in your first post makes me thing you might not understand how much storage capacity you will have, and if you do understand then I apologize for this lengthy posting.

Right now you are looking at ~32TB of storage using a RAIDZ2 (the recommended setup) and eight 6TB drives (all my calculations are rough). Subtract 10% of that and you end up with ~29TB of storage to keep your pool healthy. Using seven 6TB drives gives you ~23TB of storage (10% already subtracted).

Do you have any idea on how much storage you will need for the first 3 years? I say 3 years because you should only plan on your hard drives lasting 3 years before you need to replace them, or go by your warranty on the drives you do end up purchasing. This is the worse case scenario (yea, we hope drives last longer and many do but you cannot count on it) and how it should be planned and the hard drives are the single most expensive item when building any NAS unit so this really does matter.

If you feel you will need over 30TB of storage, you could purchase seven 8TB drives and have ~36TB (~32TB after removing that 10%) of storage. That is a lot of storage!

Figure out what you need for storage for the period of the warranty of the drives (plan 3 years if you don't know up front). Hopefully after you figure out your real storage values, you will be able to purchase seven smaller drives and save a few bucks. Once you exceed the warranty period and a drive fails, you can replace it with a larger one if that makes sense to you and keep replacing those drives as they fail until all the drives are the new size and BAM, your pool is larger.

For myself, I never see expanding my capacity in my NAS because I use well under 50% of the capacity and media actually takes up very little space compared to all my system backups I have, which are more important to me. I will likely reduce my drive count down to four 4TB drives in a RADIZ2 or maybe three 8TB drives as a mirror. All my current drives have exceeded their warranty period by over half a year, not a single failure, and none are showing signs of failure so this is good news for me as a cost savings. Maybe my drives will last a few more years, who knows but I am prepared to purchase new drives if needed.

I would like to hear what you feel your storage requirements are.
 

Mark Holtz

Contributor
Joined
Feb 3, 2015
Messages
116
I've followed a very similar logic to yours for a long time too. I agree it's fun, and provides you with that extra 'coziness' of having real neat hardware.
However, not until quite recently I was pulled out of that bubble of adding the extra cost up front to the CPU (within a CPU family). Now I regard CPU options as one "per family". G4400, i3-6100, E3-1230v5 - for 1151 socket.

Problem is, at some point false dreams should be shattered.
Spending an additional 25-30% CPU cash, on top of a E3-1230v5 is ...just not going to give you the additional years of performance you are looking for.

I suggest you reconsider again. Here are two options that I see fit:
-E3-1230v5 <for all the processing power you'd "experience" for the buck
-Put the cash into more RAM, perhaps an additional 16GB stick.
-Save that extra ~100USD (my local price difference) and spend them in X years - into your next machine.
Not sure where I would put in an additional 16GB stick where I have already budgeted to put in 64GB on the board. I'm not operating under the assumption of "If I use a 3GHz processor, I'm getting 9 years of usage, but if I use a 3.6GHz, I'm getting 11 years". Sorry, I'm looking at 10 years no matter what the CPU is finally selected.

Looking at NewEgg, the price of Skylake processors is (in US Dollars) is $210 for a 3.0GHz, $225 for a 3.3GHz, $265 for a 3.4 GHz, $285-$300 for a 3.5 GHz, and $350-$355 for a 3.6GHz model. Now, since FreeNAS is a file storage system, the limiting factor isn't the CPU, it's the speed of the hard drives. So, my question is... has anyone done a comparative study of the different speed of CPUs as it affects the performance of FreeNAS?
Bronze vs. Gold, no big deal for the system you plan to build. If you were looking to really make a power miser of a system then you would have obtained all the power specs for each component and done some serious calculations and then find the appropriate sized power supply (supplying proper amps of power for each voltage level) for the task. I personally think 650 watts is a bit high, assuming it was a true quality 650 watt power supply, but if the sale is right... Some power supplies are rated a bit high and have unclean power when running at the upper limits.
I did a calculation of all of the components, and came up with the magic number of 306 Watts. Thus, I thought that 450 Watts was the correct number. However, the feedback that I got from this is that I should get a higher-power supply. Thanks to a sale and rebate, 650 Watt was actually prices LOWER than a 450 Watt. And, it's not a Rosewill.
As for using a USB device as a boot device, I personally am against it but you may be in a situation where either you use USB, the M.2 port (making sure you buy a proper card that will also boot your system), or reduce your total number of hard drives so you have an available SATA port, or you could buy a different motherboard with more SATA/SAS ports, or an HBA card. If I were in your shoes, I'd go with reducing the total number of hard drives and leave an SATA port open for a SSD boot device, but my storage requirements are not as high as you feel yours might be.
The plan is to use USB as a boot device while I play around with a single 2TB drive (sort-of experimental box #2), and get my backups running again. Then, when the price is right, reformat, use the M.2 port as a boot device, and go with the purchase of eight drives.

Lets do some planning because the way you made a statement in your first post makes me thing you might not understand how much storage capacity you will have, and if you do understand then I apologize for this lengthy posting.

Right now you are looking at ~32TB of storage using a RAIDZ2 (the recommended setup) and eight 6TB drives (all my calculations are rough). Subtract 10% of that and you end up with ~29TB of storage to keep your pool healthy. Using seven 6TB drives gives you ~23TB of storage (10% already subtracted).

Do you have any idea on how much storage you will need for the first 3 years? I say 3 years because you should only plan on your hard drives lasting 3 years before you need to replace them, or go by your warranty on the drives you do end up purchasing. This is the worse case scenario (yea, we hope drives last longer and many do but you cannot count on it) and how it should be planned and the hard drives are the single most expensive item when building any NAS unit so this really does matter.
I have some clue, but not enough to make a definitive decision. My knowledge of RAID-based systems is a decade-and-a-half old, and only went up to RAID5. So, as I understand it, RAIDZ2 will remove two drives. I'm also under the impression that it is a better idea to set up your RAID with blank drives first rather than go with less drives, then add to the array, and wait for quite.... a.... while... as the RAID is rebuilt.

I also haven't finalized if I'm going to go with 6TB drives, or go with the smaller 4TB drives or even 3TB drives. All that I can tell you definitely is that I have four 2TB external drives with data (six if you consider that two of the 2TB drives are mirror copies) that is offline. My most immediate need is that my four computers (personal computer, home media PC, personal laptop, mother's laptop) are not backing up at the moment unless I hook up an external drive. A single 2 TB consumer drive will have to suffice until I both finalize my decision, and have the money to purchase the drives. Fortunately, this is a personal computing environment, not a corporate computing environment.

So, why did I pick 8x6TB drives in the first place? It gives me some sort of idea on the cost and power consumption. For me, it is easier to shoot for a high point, then scale downward to match with reality. It is also a personal learning project with real world goals. I had thought I was going to also need a SSD for write caching, but based upon my learning from my first experimental box, this isn't necessary.

And, before anyone asks, yes, I have a UPS as well. It's a CyberPower CP1500AVRLCD.
 
Last edited:

diedrichg

Wizard
Joined
Dec 4, 2012
Messages
1,319
I use an E3-1230 V5 and it is overkill but my system is more than just FreeNAS.
You finally switched from the red team to the blue team?!
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
Bronze vs. Gold, no big deal for the system you plan to build. If you were looking to really make a power miser of a system then you would have obtained all the power specs for each component and done some serious calculations and then find the appropriate sized power supply (supplying proper amps of power for each voltage level) for the task.

Just throwing in another argument beyond 'efficiency': the idea of quality. I'd believe comparing a new gold/platinum PSU and bronze rated psu would give good indications on build quality which might be ever so interesting in long term builds.


Cheers,
 
Last edited:
Top