Advice on New Build

[eye3]

Hello. I'm new to the forum and new to FreeNAS. I've been reading some of the stickies, forum posts and the ZFS Powerpoint presentation for the past couple weeks. I'm now ready to start building my system and I would like some of your more experienced members to let me know if there are some issues with my hardware configuration.

My use case for FreeNAS is a Time Machine backup server, plex, sabnzbd, couch potato and plex. I'll also use FreeNAS as an iSCSI target for an ESXi machine that I have. I'm an Systems Engineer so I use the ESXi machine in a limited fashion to mock up integration projects. I'm not so concerned with IOPS performance since my VMs will not be used for production purposes.

I currently have two Macs that I'll use Time Machine for and I would plan to have an additional two machines in the future. I also plan use AFP shares for additional centralized storage for my macs (ISO's, etc..).

As far as plex and the other home theater components I plan to store 1080p movie files. I'll probably move to 4k in the next couple years as the media becomes more readily available.

Here is my proposed build.

Case: Fractal Node 804
CPU: Intel Xeon E3-1245 v5 SkyLake 3.5 GHz
Motherboard: Supermicro X11SSM-F-O
RAM: 2X Crucial CT16G4RFD4213 16GB DDR4 2133 (32 GB Total)
Power Supply: Sea Sonic G550
Disks: 6X WD Red 4TB (RAID-Z2)
UPS: ???
Bootdisks: 2 SanDisk Cruiser low profile 32gb USB sticks.

USB Expansion Card: This will be used for the two USB stick boot disks since FreeNAS does support onboard USB Port boot disks with Skylake. If anyone has some suggestions on a specific compatible USB 2.0 PCIe expansion card I would appreciate some input.

Network Infrastructure: 24 port gigabit switch

My case has room for a total of 8 3.5 inch disks. So I'm contemplating adding the other two disks now. I actually thought I could just add the additional disks in the future but after reading the ZFS Powerpoint posted on the forum, I found out that your unable to add additional disks to a video I've been really surprised how much I've learned about ZFS by reading this forum. This is coming from someone who was a consultant for Sun and has been using S10 and ZFS since it was released. There is really some great technical information on here. So back to the disk configuration that's one question I'm still pondering on now. I'm not sure if 24TB RAW / 14 TB usable is going to overkill or should I just max out the case now because of the way the vdev is configured.

If I also could get some advice on a UPS I would appreciate it. I would plan to run this box and a SUX X4150 (single power supply) off it. If you see anything else that I should change just let me know. I chose the more powerful CPU and motherboard with 4K transcoding (1 stream) in mind for the future. I chose that mother board so that I could add more RAM for additional jails or ZFS in the future. I plan on starting at 32GB now and upgrading as I need more.

Thanks for your help and it's nice to meet all of you.

 

[Jailer]

I would get all 8 drives now, no sense in trying to add them later. Also if you plan on storing a lot of 1080 and/or 4K video I would skip the 4TB and go with 6TB drives. You'll be surprised how fast you can fill your server up when you start saving a lot of large video files.

 

[nightshade00013]

Yep, 8 drives now cause later on you can't just add two more to the pool and by that point you are going to be begging the mule to come kick you where it counts.

1080p files can be dealt with since as an mp4 they will be in the 2 to 3 gigabyte range most of the time. However 4k video is going to eat your space up since those files will probably be in the 10 to 25 gigabyte range. Also things may look good on paper up until Plex tries to render the file down to 1080p for your chromecast and then puke on it. Your CPU should be able to handle one stream but won't be able to do much other than chew on that 4k stream. If you expect to need to transcode a file you may be better off having two copies of that file, one at 4k and another at a lower quality that is supported by the playing device. The Passmark on the cpu you have is around 9300 and a passmark in the 8000 range will likely be minimum for 4k transcoding.

 

[eye3]

I can't believe I missed this but after looking up information the Node 804 case it looks like it does not support the ATX form factor. It's a Micro ATX case and that mother board is ATX. I'm now looking into a Skylake based board. I'll edit my original post with the new hardware configuration.

 

[brando56894]

Depending on what RAID configuration you're using you MAY be able to add disks afterwards, RAIDZ was too restrictive with the upgradability so I decided to go with a mirrored pool. If you can live with sacrificing half of your total space it's worth it because you can add mirrored pairs until your heart's content, it also allows you to upgrade your pools space more quickly since you only have to upgrade two at a time, rather than the whole pool in order to see the increased space. Think long and hard on your pool layout because it's a pain in the ass to re-do (already done it like 3x in a year, not fun).

Sent from my Nexus 6 using Tapatalk

 

[eye3]

Depending on what RAID configuration you're using you MAY be able to add disks afterwards, RAIDZ was too restrictive with the upgradability so I decided to go with a mirrored pool. If you can live with sacrificing half of your total space it's worth it because you can add mirrored pairs until your heart's content, it also allows you to upgrade your pools space more quickly since you only have to upgrade two at a time, rather than the whole pool in order to see the increased space. Think long and hard on your pool layout because it's a pain in the ass to re-do (already done it like 3x in a year, not fun).

Sent from my Nexus 6 using Tapatalk

Yes I hear you. I've been thinking about that a lot. I understand from an IOPS perspective this would be ideal since IOPS is determined by the number of vdevs and as you said this makes expanding the pool easier down the road. I'm pretty sure I'm going to go down the road of getting 8 disks now and if I want to expand the pool in the future I will move to a larger case and create a new pool from the additional disks.

 

[Bhoot]

Depending on what RAID configuration you're using you MAY be able to add disks afterwards, RAIDZ was too restrictive with the upgradability so I decided to go with a mirrored pool. If you can live with sacrificing half of your total space it's worth it because you can add mirrored pairs until your heart's content, it also allows you to upgrade your pools space more quickly since you only have to upgrade two at a time, rather than the whole pool in order to see the increased space. Think long and hard on your pool layout because it's a pain in the ass to re-do (already done it like 3x in a year, not fun).

Sent from my Nexus 6 using Tapatalk

I'll just give you a heads up for that. A raidZ2 tolerates upto 2 disk failure in your vdev without any loss of data whereas a mirror can not tolerate 2 disk failure. It is essentially why everyone asks you not to go for a raidz and go raiz2. Let's just say you have an error in your disk and you decide to resilver off the mirror. Any guarantee that the disk which you are resilvering off won't fail in the process? Just give it a thought. Cheers..

 

[gpsguy]

One can also create 3-way (and I believe 4-way) mirrors, if you need additional redundancy.

One of our users has a server with 8 striped sets of 3-way mirrors. Yes, there's a lot of overhead, but he needed high performance and data protection.




Sent from my phone

 

[Bhoot]

One can also create 3-way (and I believe 4-way) mirrors, if you need additional redundancy.

One of our users has a server with 8 striped sets of 3-way mirrors. Yes, there's a lot of overhead, but he needed high performance and data protection.




Sent from my phone

I would call it excess paranoia.. Would rather put stuff in the cloud and save all the money.. :p
edit
Still won't stop you from losing data in a triple hard disk failure from the same vdev... just saying.. when the luck goes bad the L in Luck changes to something totally different..

 

[gpsguy]

Neither will RAIDz2.

Mirrors do have their place and resilvering a mirrored vdev is much faster than a RAIDz2 vdev.

At the end of the day, neither RAIDzX, nor mirrors are a replacement for good and tested backups.

Still won't stop you from losing data in a triple hard disk failure from the same vdev.

 

[Bhoot]

Neither will RAIDz2.

Mirrors do have their place and resilvering a mirrored vdev is much faster than a RAIDz2 vdev.

At the end of the day, neither RAIDzX, nor mirrors are a replacement for good and tested backups.

yes sorry. that's what i meant. RaidZX with remote physical backups instead of mirroring disk in the same system.

 

[headcrunch]

Here is my proposed build.

Case: Fractal Node 804
CPU: Intel Xeon E3-1245 v5 SkyLake 3.5 GHz
Motherboard: Supermicro X11SSM-F-O
RAM: 2X Crucial CT16G4RFD4213 16GB DDR4 2133 (32 GB Total)
Power Supply: Sea Sonic G550
Disks: 6X WD Red 4TB (RAID-Z2)
UPS: ???
Bootdisks: 2 SanDisk Cruiser low profile 32gb USB sticks.

The crucial RAM wont work on that mainboard.

 

[brando56894]

You do realize that this thread is 10 months old right? Please don't revive an old thread, especially when it's a time-sensitive thread like a build thread.

 

[Ericloewe]

You do realize that this thread is 10 months old right? Please don't revive an old thread, especially when it's a time-sensitive thread like a build thread.

It's probably a good thing to mention, lest someone make the same mistake.