BUILD Home Server Build - Requesting Feedback

[STREBLO]

I'm building a server for personal use. I plan use it as a home server mainly for backups and media. I'm also going to host an owncloud server, use plex, and possibly run a mail server as well. I want to have the ability to use it for other things I might not initially plan for so I don't want to under build it.

Here is my current build setup, i'm not sure if i need another fan or if the case fans are sufficient. I have only bought the WD Red's so far so everything else is changeable. I'm looking for some feedback as to whether there's a better choice out there, I've made a mistake or if i'm missing anything.

• 3x - Western Digital 6TB Red Hard Drives
• Fractal Design Node 804 Chassis
• Supermicro X10SL7-F Motherboard
• Intel Xeon E3-1231 v3 CPU
• 4x 8GB ECC RAM Samsung DDR3 1.35v-1600
  
• 16GB Supermicro SATA DOM (SuperDOM) Boot Device
• SeaSonic G-550 Gold Rated Power Supply

Not sure if I should have L2ARC and ZIL cache SSD's.

 

[Fuganater]

With only 3x drives you will only be able to have RAIDZ1. If your data is important to you get 5 drives and do RAIDZ2.

 

[gpsguy]

With 3 drives, the OP could also do a 3-way mirror. Sure, it's a lot of overhead. But, it does provide additional redundancy.

 

[Fuganater]

With 3 drives, the OP could also do a 3-way mirror. Sure, it's a lot of overhead. But, it does provide additional redundancy.

That seems like a horrible waste of space.

 

[danb35]

Not sure if I should have L2ARC and ZIL cache SSD's.

If you're not sure, you don't need them. Nothing in the use case you've suggested indicates a need for them, and you don't have enough RAM to really make use of L2ARC anyway.

Everything looks good with your component choices, though the 32 GB of RAM is probably more than needed. It certainly won't hurt, but you could always start with 16 GB and add more if needed. As @Fuganater suggested, RAIDZ1 isn't recommended with 6 TB disks if the data is important to you, due to the time needed to rebuild after a disk failure and the possibility that data will be corrupted on one of the remaining disks. RAIDZ2 is safer, but requires a minimum of four disks.

SATA DOMs are kind of expensive. You might want to consider a small (30 GB or so) SSD for the OS instead, as it will probably cost less, and performance will still be entirely adequate. Or a USB stick still works just fine, too, it just isn't quite as fast.

 

[STREBLO]

With only 3x drives you will only be able to have RAIDZ1. If your data is important to you get 5 drives and do RAIDZ2.

Yea, I really wasn't thinking about how expensive the drives would end up being when I decided to go with the 6tb size... I guess a 4 drive RAIDZ2 would be suboptimal too. Guess i'll have to fork out another grand.

If i'm running say a plex server, the 32GB of ram with no L2ARC should be sufficient right?

Oh, do I need a HBA too?

 

[jgreco]

That seems like a horrible waste of space.

You want a horrible waste of space? I've got a shelf of 24 2TB 2.5" drives. In order to meet the requirement that no single drive loss results in a loss of redundancy, it is configured as seven sets of three-way mirrors, plus three warm spare drives. That means 48TB of disks provides 14TB of pool space. Of that, since this is being used for VM service, you really can't fill more than 50%, so there's actually 7TB of usable space with good performance AND redundancy. From 48TB of raw space.

 

[jgreco]

Yea, I really wasn't thinking about how expensive the drives would end up being when I decided to go with the 6tb size... I guess a 4 drive RAIDZ2 would be suboptimal too. Guess i'll have to fork out another grand.

If i'm running say a plex server, the 32GB of ram with no L2ARC should be sufficient right?

Oh, do I need a HBA too?

4 drives in RAIDZ2 works fine. Not rocket ship fast but fine. One of the filers here is 4 x 6TB in RAIDZ2. General purpose office fileserver. Works fine unless someone's moving huge files to it, then everything gets a little slowish.

32GB of RAM should be plenty to run a Plex server. You could safely try it on 16GB first, just buy that as two sticks of 8GB, not four of 4GB.

You don't need a HBA unless you need more ports than your mainboard provides, or if your mainboard provides crippled SATA ports like the Marvell ports.

 

[STREBLO]

So I might be better off returning the drives I have and getting 5 smaller drives? Ultimately it would be best to run at least 4 in Z2, best would be I buy 5 in some size.

 

[Fuganater]

You want a horrible waste of space? I've got a shelf of 24 2TB 2.5" drives. In order to meet the requirement that no single drive loss results in a loss of redundancy, it is configured as seven sets of three-way mirrors, plus three warm spare drives. That means 48TB of disks provides 14TB of pool space. Of that, since this is being used for VM service, you really can't fill more than 50%, so there's actually 7TB of usable space with good performance AND redundancy. From 48TB of raw space.

You still do not convice me because your setup seems like a huge waste of money. Granted you know way more about FreeNAS than I ever will but from how we do things at work compareded to you... it does not make sense.

Also this thread is for a home server. Unless you are sh**ing out benjimians or data is THAT critical, a 3 way 6TB mirror seems like a waste. You should have at least 2 backups of your data running several times a day so if you lose 1 drive or all of them, there should be Zero data lost.

 

[gpsguy]

Depending on your storage needs, I'd look at 6 x 4TB disks. In RAIDz2, you should see ~14TiB of space.

Six disks is one of the sweet spots. 5x6TB drives, doesn't offer much more in usable space, but will cost about $350 (USD) more than 6x4TB drives.

So I might be better off returning the drives I have and getting 5 smaller drives? Ultimately it would be best to run at least 4 in Z2, best would be I buy 5 in some size.

 

[STREBLO]

Wait, I was just looking at some other threads and the ZFS docs are saying 4 drives is better for RAIDZ2 and 5 for RAIDZ1, so if I plan to go with RAIDZ2 I would be better off with 4 disks?

 

[Bidule0hm]

Don't worry too much about this rule, since compression is enabled by default you'll probably use it and if it's the case then the rule doesn't apply (for speed, but it's still valid for space overhead).

 

[jgreco]

You still do not convice me because your setup seems like a huge waste of money. Granted you know way more about FreeNAS than I ever will but from how we do things at work compareded to you... it does not make sense.

When you're replacing several dozen machines with virtualization, it makes perfect sense. Many of the machines already had redundant disks. Downsizing from sixty or seventy 3.5" hard drives that were each being underutilized, to one large storage array with two dozen low power drives and a lot of ARC, you can justify doing WHATEVER it takes, including going to SSD, in order to replace those.

MY analysis was that our workload was actually light enough that the SSD solution would have been a "huge waste of money." The 24 2.5" hard drives are fast enough.

The other part of that analysis was that the VM inventory has slowly grown from that several dozen machines up to almost 200. A storage array failure causing loss of the pool would be a major problem, so one of the rules for a replacement was that the loss of a single disk could not result in the loss of redundancy. This meant that simple two-way mirrors were unacceptable, but three-way was acceptable. That also happened to fit the read-heavy workload model here.

Also this thread is for a home server. Unless you are sh**ing out benjimians or data is THAT critical, a 3 way 6TB mirror seems like a waste. You should have at least 2 backups of your data running several times a day so if you lose 1 drive or all of them, there should be Zero data lost.

Depends how much data you're backing up and what losing access to it for awhile means. For a home user, that might be just fine. Or it might not. For me, since I travel for business and am sometimes away for a month, I prefer to have more redundancy and to have spare resources in place where possible.

 

[jgreco]

Wait, I was just looking at some other threads and the ZFS docs are saying 4 drives is better for RAIDZ2 and 5 for RAIDZ1, so if I plan to go with RAIDZ2 I would be better off with 4 disks?

Possibly. Don't get too worked up about it, but given two options, one of which is optimal and one of which isn't, the optimal one is still preferable. RAIDZ2 with 4, 6, or 10 disks is an optimal configuration. That's why the 6 x 4TB HDD option listed above by @gpsguy is a good one.

 

[STREBLO]

Alright guess i'll go with either 4x/6x 6tb RAIDZ2. Thanks for the help.

I had oversized my PSU by a bit incase I decided to add more drives later. That's perfectly safe right?

 

[jgreco]

I had oversized my PSU by a bit incase I decided to add more drives later. That's perfectly safe right?

As long as you haven't gone stupid-big, yes. The primary concern is that your average PSU can get wonky if it doesn't have enough load placed on it. For a quality PSU like the ones we suggest, that probably means below 10% load, so if you get a 550W PSU, and you're drawing less than 55W, that could be a problem and you might want to contact the vendor to see if there's actually a low end. There's a nice PSU sizing sticky that also shows how you might estimate your peak and idle loads.

 

[TXAG26]

Go with 5TB drives. They are the sweet spot right now in $$$/capacity.

 

[jgreco]

Go with 5TB drives. They are the sweet spot right now in $$$/capacity.

That's probably good advice. A typical mistake is to just look at the cost of the drives and pick the lowest cost-per-TB drive. Don't do that. Consider instead the cost per usable TB, which means the cost after RAID parity. I also like to add in the cost of the server as a whole, because you did have to pay for the whole thing in order to get that capacity.

So if you have an $800 server, and four 4TB disks at $150/ea in RAIDZ2, your total cost is $1400 for 8TB of usable space, or $175/TB. If you bump that to four 6TB disks at $250/ea , that's $1800 for 12TB of space, or $150/TB.