Home server build

[achianese]

I'm about to build a home FreeNAS server and I'd appreciate any suggestions on my plans. The server will need to store about 3 TB of data at first, and will probably grow slowly. I'm planning to run the Plex plugin, a few other lighter plugins, and one potentially interesting challenge is to run a MythTV backend (in an Ubuntu Server VM using the phpVirtualbox plugin). I "borrowed" my non-ECC workstation (Core i7, 16GB) for the weekend to confirm that MythTV does in fact work (very well!) as a backend in a VM configured like this.

Here are my hardware plans:

Fractal Design Arc Midi R2 - plenty of ventilation and space for drives

Supermicro X11SSL-O - Cheapest X11.. I know it doesn't have IPMI and I'm ok with that.

Xeon E3-1220 v5 - This is admittedly overpowered for a fileserver, but I expect the plex transcoding and mythtv operations will be fairly processor intensive and I want to be safe here

Seasonic S12G-550 power supply

1 x 16 GB Samsung D416GE21S - on the compatibility list and only $93. Yes, ECC. I feel like 16 GB is enough, but not sure. I think the VM will get 6-8 GB. Can always buy another 16 later if needed.

Cyberpower CP1000AVRLCD - does anyone know if this one works easily with upsd? I suspect it will.

3 x SanDisk Cruzer Fit 16GB for boot devices (three-way ZFS mirror). This one I'm not sure about. I see a lot of people using small SSDs as boot devices recently, but to me it seems better to have redundancy than speed in the boot device. Am I missing something here?

And for hard drives, I'm planning kind of a weird strategy. I have two Samsung 2TB drives that are a few years old but check out fine in smart. I think I want to buy 3 x 3TB WD Blue drives, and make a 5-disk RAIDZ2 out of them. I know that the 3TB drives will only be used as 2TB, giving me a total usable space of about 4.8 TB up front. If I decide to upgrade those 2TB drives to 3TB (or if they fail) then I will have 7.2 TB usable. It would only save me $60 to buy 3 x 2TB instead, and to me it's worth the money for the future potential storage.

This build will cost about $1070.

Does anyone have thoughts on this build? Has anyone been running a MythTV backend in a VM for longer than a weekend? Is Skylake ready for primetime, assuming I'll install 9.10? Thanks for any help.

p.s. the box in my signature is a server I built for file storage at my workplace. I love it to pieces.

 

[tvsjr]

I would suggest IPMI - it's a big advantage.
You can't go wrong with more RAM, especially if you're going to run VMs. I'd start with 32.
Boot devices - three-way USB mirror? Meh. USB sticks are slow and often unreliable. I'd rather run a single small SSD than a pair of USB sticks. Honestly, my boot devices are 2 40GB Intel drives I bought off eBay for $20/ea. They work great.
WD Blue isn't rated for NAS use. You'd want WD Red. HGST NAS drives are another popular option.

Personally, I like boxes being single-purpose... let FreeNAS be FreeNAS, build a separate system as a VM host for your Myth, etc. But that's a religious argument :)

 

[Jailer]

So if I read all this correctly you are going to build a new server that will be around 90% full when you start and then reduced to around 50% full when you change drives to expand your pool. I think that is a very poor strategy and you will wish that you would have started your build with more/larger drives. You're nice shiny new server is going to be hamstrung by a lack of storage space.

 

[achianese]

I would suggest IPMI - it's a big advantage.
You can't go wrong with more RAM, especially if you're going to run VMs. I'd start with 32.
Boot devices - three-way USB mirror? Meh. USB sticks are slow and often unreliable. I'd rather run a single small SSD than a pair of USB sticks. Honestly, my boot devices are 2 40GB Intel drives I bought off eBay for $20/ea. They work great.
WD Blue isn't rated for NAS use. You'd want WD Red. HGST NAS drives are another popular option.

Personally, I like boxes being single-purpose... let FreeNAS be FreeNAS, build a separate system as a VM host for your Myth, etc. But that's a religious argument :)

Thanks. The IMPI is actually only $5 to add, so I may as well do it. I was waffling on the RAM anyway.

Regarding the boot drives, the advice last time I built something was that they were mostly read at boot time, and once the system is running it runs completely from RAM. This has been my experience with my current setup (see sig). I once had a USB drive fail and it was a breeze to replace in the GUI. If I had only one sata SSD I'd have to restore from a saved configuration (and wait for the new drive to arrive before rebooting). If I do mirrored SSDs, now I've used two of my 6 SATA ports, when I want a 5-drive RAIDZ2. Then I'd need to get the X11SSM instead, which is only $20 more. I could I guess I'm wondering what I missed in the last two years that is causing more people to use SSDs as boot drives now. (Jails?)

Reading the forums, I think the choice of WD Blue (formerly green) vs Red is more a matter of philosophy. Assuming I stop the heads from parking with camcontrol or wdidle, I would expect similar performance and durability. But I'd appreciate more input on what Reds would offer me.

As for the "religious" issue, I'm partial to building one machine to take care of my various server needs because:
1) My home server needs are not that strenuous, even with the MythTV VM
2) Power consumption is better with one more powerful box than several single-use boxes
3) Total cost to build is better with one box
4) I like having my VMs stored on ZFS.

Thanks again for the advice. I hope I don't come off as confrontational.

 

[achianese]

So if I read all this correctly you are going to build a new server that will be around 90% full when you start and then reduced to around 50% full when you change drives to expand your pool. I think that is a very poor strategy and you will wish that you would have started your build with more/larger drives. You're nice shiny new server is going to be hamstrung by a lack of storage space.

I think you are reading that part wrong, but I did explain it in a convoluted way.

Storage need: 3TB now, may grow to 4TB in ~3 years

Current hardware: 2x2TB Seagate drives
Planned purchase: 3x3TB WD Blue drives

Initial setup: 5-disk RAIDZ2 out of these drives. 10 TB total, ~4.8 TB usable at 80% capacity
This means the drives will start out using around 60% of the practically usable capacity. (note this is 50% of the actual free space)

Then if I replace the 2TB drives with 3TB in the future, I will have 15TB total or 7.2 TB usable at 80% capacity.

To me, this seems like a good way to cover my present needs with a good amount of wiggle room, but leave the possibility of a cheap upgrade in a few years if needed.

 

[tvsjr]

Regarding the boot drives, the advice last time I built something was that they were mostly read at boot time, and once the system is running it runs completely from RAM. This has been my experience with my current setup (see sig). I once had a USB drive fail and it was a breeze to replace in the GUI. If I had only one sata SSD I'd have to restore from a saved configuration (and wait for the new drive to arrive before rebooting). If I do mirrored SSDs, now I've used two of my 6 SATA ports, when I want a 5-drive RAIDZ2. Then I'd need to get the X11SSM instead, which is only $20 more. I could I guess I'm wondering what I missed in the last two years that is causing more people to use SSDs as boot drives now. (Jails?)

Even if you aren't running jails, updates will run much faster, etc. And logs do get written to the boot device.

Reading the forums, I think the choice of WD Blue (formerly green) vs Red is more a matter of philosophy. Assuming I stop the heads from parking with camcontrol or wdidle, I would expect similar performance and durability. But I'd appreciate more input on what Reds would offer me.

It's a matter of warranty and fitness for purpose. WD specifically builds the Reds for NAS service, and the Blues/Greens for desktop use. If you ever need to RMA a drive, they're likely to not honor the warranty if you aren't using the drives for their intended purpose.

If I had a pile of Greens/Blues laying around free? Sure, I'd use them. Buying new drives anyway? Buy the right stuff.

As for the "religious" issue, I'm partial to building one machine to take care of my various server needs because:
1) My home server needs are not that strenuous, even with the MythTV VM
2) Power consumption is better with one more powerful box than several single-use boxes
3) Total cost to build is better with one box
4) I like having my VMs stored on ZFS.

Like I said, it's a religious issue. You can have VMs stored on ZFS - you use NFS or iSCSI. Myth has a habit of being pretty hard on the CPU, especially if you're running multiple streams and doing lots of transcoding. I'd overbuild a tad and keep a close eye on things.

 

[achianese]

Thanks again. I'm sold on the Reds, 32 GB ram, and the SSDs, which means I'm getting an SSM rather than an SSL board. How can you sleep at night knowing you've added $200 to my build cost? :)

Regarding MythTV, I'm currently running my backend and frontend together on an old Core2 Duo Ubuntu box with 4GB of RAM, along with Plex. It basically works but it's certainly taxed when I watch TV and stream at the same time.

 

[tvsjr]

How can you sleep at night knowing you've added $200 to my build cost? :)

Pimpin' ain't easy!

 

[Robert Trevellyan]

I'm sold on the Reds

Good, because they support TLER. Without that, a failing drive can make the whole box unresponsive, which makes drive replacement much more challenging.

 

[danb35]

And logs do get written to the boot device.

Logs are only written to the boot device if you put your .system dataset there. On the data pool is usually a better choice (and definitely a better choice if you're using a USB boot device).

It's a matter of warranty and fitness for purpose. WD specifically builds the Reds for NAS service, and the Blues/Greens for desktop use. If you ever need to RMA a drive, they're likely to not honor the warranty if you aren't using the drives for their intended purpose.

I'm trying hard not to say "bullsh*t" here, but it's difficult. Is there any documented case of WD denying warranty coverage of a blue/green drive because it was used in a NAS? Without lots of such cases, saying that WD is "likely to not honor the warranty" is, to be more polite, FUD.

Reds do have advantages over greens/blues, and the longer warranty (3 years vs. 2) is IMO one of the bigger ones. TLER, I guess, is a benefit. If you put any faith in MTBF numbers, the Reds are longer. And yes, the cost delta is usually pretty small.