BUILD New Server Build (or learning from past mistakes) - How am I doing?

[avggeek]

So about 3 years ago, I built a server to act as my file/backup and media server. Since I had only built PC's till then, I realize now I made some mistakes when it came to general reliability and ugpradeability - poor airflow in the case, not enough drive slots etc. After lurking around the forum for a month or so, here's the build I'm planning to put together:

• Case : Fractal Design Define R5
• Motherboard : Supermicro X10SL7-F
• Memory : 16 GB ECC Crucial RAM (2*8GB)
• Power Supply : Corsair HX750i
  
• CPU : i3-4170 (Preferred due to budget) or Xeon E3-1231v3
• Boot Drive : Samsung EVO 120 GB (OS Partition - 60 GB / ZIL - 10 GB (?))
• RAIDZ1 Volume 1 : HGST DeskStar 3 * 3 TB
• RAIDZ1 Volume 2 : HGST DeskStar 3 * 4 TB

The server is primarily used as a File/Backup server and a media server (for 1-2 Plex clients). I'm planning to add another 3 * 4 TB drives later on so in another RAIDZ1 volume which I will add to the second pool.

The biggest confusion I now have is about flashing the LSI 2308 controller onboard the Motherboard into IT mode. From what I had understood, once I flashed the controller into IT mode it would pass through any drives connected to the SAS ports directly to the OS. However, when I was trying to order the motherboard locally the distributor here told me I could either use the 6 SATA ports or the SAS ports and not combine the two.

I'd appreciate if somebody can confirm that once I flash the controller into IT mode, can I have hard drives connected to the regular SATA ports as well as to the SAS ports?

I'm also trying to make up my mind about the CPU. My current server has an i3 and it's worked fine but I'm wondering if I should move up to the Xeon to give myself more headroom over the next few years.

Any other feedback or advice on the build before I pull the trigger on my purchases are most welcome!

 

[Robert Trevellyan]

Samsung EVO 120 GB (OS Partition - 60 GB / ZIL - 10 GB (?))

I'm really no expert, but I think using one SSD for these two completely unrelated purposes is probably a bad idea. What makes you think you need a dedicated ZIL device anyway?

RAIDZ1 Volume 1 ... RAIDZ1 Volume 2

Are you aware that RAIDZ1 is generally not recommended when using large drives? Do you only plan to store easily replaceable data on this system?

I'm planning to add another 3 * 4 TB drives later on so in another RAIDZ1 volume which I will add to the second pool.

I suggest taking your existing six drives and making a single-vdev RAIDZ2 volume. Then when you acquire your 3 additional 4TB drives, replace the 3TB drives in the original pool one by one. This will automatically expand the pool.

 

[Robert Trevellyan]

another RAIDZ1 volume which I will add to the second pool

One more thing. You can grow a pool by adding a vdev, not a volume. Volumes in FreeNAS are equivalent to pools in ZFS. However, I can see how the terminology in the FreeNAS Volume Manager GUI could cause confusion, since it seems to use the term Volume for vdevs too.

 

[avggeek]

I'm really no expert, but I think using one SSD for these two completely unrelated purposes is probably a bad idea. What makes you think you need a dedicated ZIL device anyway?

I don't think I need a ZIL device on Day 1. In fact it's quite possible I won't need one at all. However, since FreeNAS will lock the install drive I wanted to leave the possibility of being able to provision a ZIL drive by limited the install partition to less than total capacity. My understanding is that once FreeNAS is loaded there is very little activity on the system drive so I felt there would not be much contention if the ZIL drive was on the same device.

Are you aware that RAIDZ1 is generally not recommended when using large drives? Do you only plan to store easily replaceable data on this system?

Yes I have spent time reading up on the several threads discussing RAIDZ1 and RAIDZ2. The first RAIDZ1 volume is where the backup data from other machines on the network will live. Having said that, my current server is configured to backup a RAID1 volume to an external USB drive with critical folders backed up to the cloud from the client machines directly. I'm reasonably confident I can write a shell script to backup the data living in the RAIDZ1 volume to the external USB drive under FreeNAS as well. This then gives me two additional backup points apart from the RAIDZ1 volume which I believe is sufficient to counteract any risk of drive failure - something I've tested when my existing RAID1 setup failed due to bad hard drive choices (WD Green's in a RAID1 do not work!) and I had to restore the data from the USB drive.

I suggest taking your existing six drives and making a single-vdev RAIDZ2 volume. Then when you acquire your 3 additional 4TB drives, replace the 3TB drives in the original pool one by one. This will automatically expand the pool.

One more thing. You can grow a pool by adding a vdev, not a volume. Volumes in FreeNAS are equivalent to pools in ZFS. However, I can see how the terminology in the FreeNA.S Volume Manager GUI could cause confusion, since it seems to use the term Volume for vdevs too.

Apologies for the poor wording on my side. In fact it is for the reason of wanting to be able to add vdev's to a pool when I need to expand capacity for the 2nd RAIDZ1 volume that I want to go with the two RAIDZ1 volumes. The 2nd set of 3x4 TB drives gives me extra 7.45 TB capacity when I need it and I also have the flexibility to grow this pool by replacing drives in it. I expect this pool will run out of capacity much more than the other one for two reasons 1) This is going to be storing media 2) Backups tend to have a fairly consistent size (due to older backups being automatically aged out)

 

[Robert Trevellyan]

I don't think I need a ZIL device on Day 1. In fact it's quite possible I won't need one at all.

I would go with highly unlikely. A dedicated ZIL only helps when you have a workload of sync writes, and you haven't indicated that you'll have that.

Yes I have spent time reading up on the several threads discussing RAIDZ1 and RAIDZ2.

OK, good.

 

[avggeek]

I would go with highly unlikely. A dedicated ZIL only helps when you have a workload of sync writes, and you haven't indicated that you'll have that.

OK, good.

Any advice/input on the question I have about the motherboard SAS ports / CPU decision?

 

[Robert Trevellyan]

Any advice/input on the question I have about the motherboard SAS ports / CPU decision?

Sorry, I can't help with the ports question. The CPU decision is all about the likely workload. You said "media server" but I didn't notice anything about transcoding. If all you're doing is streaming to a couple of clients I doubt you would need a Xeon.

 

[Jailer]

Any advice/input on the question I have about the motherboard SAS ports / CPU decision?

Yes you can use the SAS ports in conjunction with the SATA ports. Once in IT mode the drives are passed directly to the OS just as they are on the SATA ports. As long as FreeNAS has direct access to the drives it doesn't care what they are connected to.

Only you can decide on the CPU. It's entirely workload dependent and how you intend to use your server will dictate what CPU to choose. I always tend to overdo everything so I went with a Xeon in my build.

 

[avggeek]

Yes you can use the SAS ports in conjunction with the SATA ports. Once in IT mode the drives are passed directly to the OS just as they are on the SATA ports. As long as FreeNAS has direct access to the drives it doesn't care what they are connected to.

Thank you - that's a big potential roadblock removed from the build for me.

Sorry, I can't help with the ports question. The CPU decision is all about the likely workload. You said "media server" but I didn't notice anything about transcoding. If all you're doing is streaming to a couple of clients I doubt you would need a Xeon.

Only you can decide on the CPU. It's entirely workload dependent and how you intend to use your server will dictate what CPU to choose. I always tend to overdo everything so I went with a Xeon in my build.

For media serving, almost all the clients are running native Plex apps which use direct play rather than transcoding. The remaining Plex clients and Samba/AFP clients don't typically hit the server at the same time. I'll play with the budget to see if I can afford the extra headroom of the Xeon.

 

[Scharbag]

System looks powerful.

I would axe the SSD as a boot drive and use 2 8GB USB sticks in a mirrored arrangement. ZIL will not be needed. L2ARC for your use is not needed. It would be better to use the money that you save on using USB boot volumes and buy 32 GB of ECC RAM.

Unless you are going to use the CPU for something, go with the i3. I have a XEON, I virtualize a ton of stuff and it barely gets warmed up. Just make sure you use the ECC ram which you say you are going to!!

As for the hard drives, I would recommend that you buy 6@4TB HDDs and use a single Z2 array. This offers better fault tolerance. Partition it up as you see fit with datasets and script your backups with Rsync.

Cheers,

 

[avggeek]

It would be better to use the money that you save on using USB boot volumes and buy 32 GB of ECC RAM.

Would I need 32 GB RAM right-away? I was planning to monitor the memory utilization and then look into upgrading.

Unless you are going to use the CPU for something, go with the i3. I have a XEON, I virtualize a ton of stuff and it barely gets warmed up. Just make sure you use the ECC ram which you say you are going to!!

The ECC RAM is already on it's way from Newegg - picked up the Crucial 2x8GB Kit that is listed in the stickie as compatible with the X10SL7. I'm still waffling on the Xeon vs i3 since the differential is not that big the way I'm paying for this purchase.

As for the hard drives, I would recommend that you buy 6@4TB HDDs and use a single Z2 array. This offers better fault tolerance. Partition it up as you see fit with datasets and script your backups with Rsync.

I'm reasonably comfortable with the RAIDZ1 approach for both pools based on the history I've had with the equivalent setup in my current server. Although I may wind-up with an oddball 4x3TB RAIDZ1 for one pool as I have some 3 TB drives from my current server that will be extra after the upgrade.