BUILD First Build

[rfvgyhn]

I'm looking at creating a nas for storage only (no plex transcoding or anything) with a raidz2 vdev. I plan to have it store all of my media (movies, music) and in the near future, VMs that will be run on a separate machine.

PCPartPicker part list / Price breakdown by merchant

Storage x4: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($179.99 @ Micro Center)
Case: Fractal Design Node 304 Mini ITX Tower Case ($59.99 @ Newegg)
Power Supply: Corsair CSM 450W 80+ Gold Certified Semi-Modular ATX Power Supply ($29.99 @ Newegg)
Other x2: Crucial 8GB Single DDR3 1600 MT/s (PC3-12800) CL11 Unbuffered ECC UDIMM 240-Pin Server Memory CT102472BA160B ($90.00)
Other: ASRock C2550D4I Mini ITX Server Motherboard DDR3 1600/1333/1066 ($279.99)
Total: $1269.93
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2014-11-26 14:25 EST-0500

My questions are mainly in regards to when I start using the nas for vm storage.

1. First, do these parts all look like they would work well together? Any suggestions?
2. I assume the quad core version cpu will be good enough. Does that sound like a correct assumption or should I get the octa core version (or a separate mobo/cpu entirely)?
3. I have read that having an ssd for a zil and l2arc can help vm performance (though not as much as maxing system ram). Is that a "plug and play" sort of feature in that I could add it after zpools have been setup and running for a while?

 

[Fraoch]

First, do these parts all look like they would work well together? Any suggestions?

Everything looks great, but those 7200 RPM drives will run hot. There isn't much you can do to improve ventilation in the case apart from replacing the front fans for ones with higher airflow or run the fans at full speed. That will come at the expense of noise.

I assume the quad core version cpu will be good enough. Does that sound like a correct assumption or should I get the octa core version (or a separate mobo/cpu entirely)?

It should be fine, especially if you're not transcoding.

I have read that having an ssd for a zil and l2arc can help vm performance (though not as much as maxing system ram). Is that a "plug and play" sort of feature in that I could add it after zpools have been setup and running for a while?

I believe so, but I'm not 100% on this - hopefully someone else can comment.

 

[Tywin]

I would consider maxing out the case with 6× HDDs for a better upgrade path. You could do 6×2 TB for the same storage you have now, or 6×3 TB for an extra 50% usable space.

 

[marbus90]

First off, for virtualisation you want raid10 (striped mirrors), raidz is just painfully slow. Also for SLOG (ZIL on separate disks) you want mirrored SSDs - everything written to the NAS will be going to the SLOG first. When the single SLOG SSD dies, all cached data is lost. L2ARC is nice to have, but probably SLOG would help more. Also, with that board you're still limited to 6 ports - the Marvel controllers are known to have issues.

Recommendation for SLOG would be Intel S3500 for light and S3700 for heavy use, since they seem to be the only ones with a reasonably sized capacitor.

 

[Ericloewe]

I'm looking at creating a nas for storage only (no plex transcoding or anything) with a raidz2 vdev. I plan to have it store all of my media (movies, music) and in the near future, VMs that will be run on a separate machine.

PCPartPicker part list / Price breakdown by merchant

Storage x4: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($179.99 @ Micro Center)
Case: Fractal Design Node 304 Mini ITX Tower Case ($59.99 @ Newegg)
Power Supply: Corsair CSM 450W 80+ Gold Certified Semi-Modular ATX Power Supply ($29.99 @ Newegg)
Other x2: Crucial 8GB Single DDR3 1600 MT/s (PC3-12800) CL11 Unbuffered ECC UDIMM 240-Pin Server Memory CT102472BA160B ($90.00)
Other: ASRock C2550D4I Mini ITX Server Motherboard DDR3 1600/1333/1066 ($279.99)
Total: $1269.93
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2014-11-26 14:25 EST-0500

My questions are mainly in regards to when I start using the nas for vm storage.

1. First, do these parts all look like they would work well together? Any suggestions?
2. I assume the quad core version cpu will be good enough. Does that sound like a correct assumption or should I get the octa core version (or a separate mobo/cpu entirely)?
3. I have read that having an ssd for a zil and l2arc can help vm performance (though not as much as maxing system ram). Is that a "plug and play" sort of feature in that I could add it after zpools have been setup and running for a while?

Don't even think about using an L2ARC before you have 64GB of RAM (meaning never, on your hardware).

As for SLOG, it varies.

I recommend you read Cyberjock's guide for details.

 

[rfvgyhn]

First off, for virtualisation you want raid10 (striped mirrors), raidz is just painfully slow.

So do you think it's a bad idea to share my media, document storage with vm storage? Performance isn't a big concern of mine since my vm stuff will be more of a testing grounds than anything else.

Also for SLOG (ZIL on separate disks) you want mirrored SSDs - everything written to the NAS will be going to the SLOG first. When the single SLOG SSD dies, all cached data is lost. L2ARC is nice to have, but probably SLOG would help more. Also, with that board you're still limited to 6 ports - the Marvel controllers are known to have issues.

Recommendation for SLOG would be Intel S3500 for light and S3700 for heavy use, since they seem to be the only ones with a reasonably sized capacitor.

Don't even think about using an L2ARC before you have 64GB of RAM (meaning never, on your hardware).

As for SLOG, it varies.

I recommend you read Cyberjock's guide for details.

I'm still uncertain about one thing regarding a SLOG. Can that be bolted on at a later time or would that need to be setup at the same time as the zpool/vdev?

 

[marbus90]

It's not neccessarily bad to share the same vdev for everything, but the VM won't like it much when they need IOPS and other stuff is hogging them away. That being said, I'm fine with 7200rpm drives in a raid1 for everything.

SLOG can be added afterwards.

Ratio for L2ARC is roughly 20GB L2ARC vs 1GB RAM, but it only helps if you access the same files over and over again. IMHO not so much in a home enviroment.

 

[DKarnov]

Considering for SLOG it's recommended to have two mirrored SSD's, it seems to make more sense to just put two mirrored SSDs in the box you're running the VMs on to begin with. No matter how much you soup up the NAS end, you're still going to be running into gigabit Ethernet limits, and defeating those is $$$. SSD prices are dropping like stones, so assuming your VMs sans data aren't going to be very big you can probably do that for little more than a hundred bucks. I can't tell you the specifics of periodically backing the VMs up to the NAS if you want snapshots, but I doubt it's strenuous.

The orthodoxy will also say, given today's very large drive sizes, that RAID5 / RAIDZ1 is obsolete. You might be using those four drives as two mirrored pairs, but if you were planning RAIDZ1, you might want to rethink, and it's easier to go RAIDZ2 from the start than it is to juggle your data to move to it later.

I have a similar setup with 6x of the HGST NAS 3TBs. I doubt the 4TBs are much different thermally. Keeping in mind that my ambient temps are stuck at or over 25c, with the stock fans I saw a high on one drive during stress testing of 44C. I swapped the front fans with Noctuas, left everything under mobo speed control and didn't notice any increase in noise (I could've also used the 7VDC adapters for similar effect) and re-ran the stress tests, and the highest drive only hit 40. When not getting slammed the drives are in the mid 30s. I don't consider that unreasonable, and if I did there are further things I could do to increase cooling without changing noise, but those who aren't slaves to fashion could definitely get better numbers with a bigger, better flowing chassis. The Noctuas (and appropriate SATA/power cables so it's not a rat's nest in there and air can flow) did add ~60-70 bucks to build cost, so be sure to factor that into the price of packing tens of terabytes into a breadbox.

 

[joeschmuck]

I'm looking at creating a nas for storage only (no plex transcoding or anything) with a raidz2 vdev. I plan to have it store all of my media (movies, music) and in the near future, VMs that will be run on a separate machine.

PCPartPicker part list / Price breakdown by merchant

Storage x4: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($179.99 @ Micro Center)
Case: Fractal Design Node 304 Mini ITX Tower Case ($59.99 @ Newegg)
Power Supply: Corsair CSM 450W 80+ Gold Certified Semi-Modular ATX Power Supply ($29.99 @ Newegg)
Other x2: Crucial 8GB Single DDR3 1600 MT/s (PC3-12800) CL11 Unbuffered ECC UDIMM 240-Pin Server Memory CT102472BA160B ($90.00)
Other: ASRock C2550D4I Mini ITX Server Motherboard DDR3 1600/1333/1066 ($279.99)
Total: $1269.93
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2014-11-26 14:25 EST-0500

My questions are mainly in regards to when I start using the nas for vm storage.

1. First, do these parts all look like they would work well together? Any suggestions?
2. I assume the quad core version cpu will be good enough. Does that sound like a correct assumption or should I get the octa core version (or a separate mobo/cpu entirely)?
3. I have read that having an ssd for a zil and l2arc can help vm performance (though not as much as maxing system ram). Is that a "plug and play" sort of feature in that I could add it after zpools have been setup and running for a while?

Taking your posting at face value here is my two cents...

1) For what you want to do I would recommend a different hard drive, slower rotational speed which means less heat and less noise, but cost and warranty are also key factors to consider. Take a look at the WD Green here for $129, or you could purchase 6 of the 2TB drives at $69 each. All you need to do is run WDIDLE on the WD Green drives to change the head parking timer to 300 seconds vice 8 seconds and it will be a great drive and you will save a lot of money. Not sure if you know this but your total storage would be ~7.2 TB, that is a lot of storage, exactly what I have been running for a few years now. If you were to switch over to the 2TB drives there is one major gain from that which is it will take less time to resilver (replace) a drive failure. The down side is you are looking at slightly more money and slightly more overall power consumption. I believe performance wise the 6 drive is better however your system isn't a high grade server so you may never even notice it. There are pros and cons to just about every choice building a system.

2) Your other choices sound fine, including the four core for you're stated purpose.

3) You have no use for a ZIL or L2Arc and they will be of no benefit but more likely will harm your throughput slightly. Don't do it. You want to share your data with a VM, well that is no different than sharing it with any other computer. The part you misunderstood was these items (zil/l2arc) were for running a VM on your FreeNAS system, huge difference from your intentions. Your choice of computer is not up for that challenge if you were to attempt that, but again that is not what you originally asked. If you want a nice and simple NAS, you have the right parts.

So the real question for you is, what else are you planning to use your FreeNAS machine for? Your selection might be able to handle some transcoding as well, but maybe not 2 streams at a time.

 

[rfvgyhn]

Considering for SLOG it's recommended to have two mirrored SSD's, it seems to make more sense to just put two mirrored SSDs in the box you're running the VMs on to begin with. No matter how much you soup up the NAS end, you're still going to be running into gigabit Ethernet limits, and defeating those is $$$. SSD prices are dropping like stones, so assuming your VMs sans data aren't going to be very big you can probably do that for little more than a hundred bucks. I can't tell you the specifics of periodically backing the VMs up to the NAS if you want snapshots, but I doubt it's strenuous.

3) You have no use for a ZIL or L2Arc and they will be of no benefit but more likely will harm your throughput slightly. Don't do it. You want to share your data with a VM, well that is no different than sharing it with any other computer. The part you misunderstood was these items (zil/l2arc) were for running a VM on your FreeNAS system, huge difference from your intentions. Your choice of computer is not up for that challenge if you were to attempt that, but again that is not what you originally asked. If you want a nice and simple NAS, you have the right parts.

Running the VMs on their own SSD(s) makes much more sense. Since they can be easily rebuilt via vagrant/chef, I can just store their data on the NAS.

1) For what you want to do I would recommend a different hard drive, slower rotational speed which means less heat and less noise, but cost and warranty are also key factors to consider. Take a look at the WD Green here for $129, or you could purchase 6 of the 2TB drives at $69 each. All you need to do is run WDIDLE on the WD Green drives to change the head parking timer to 300 seconds vice 8 seconds and it will be a great drive and you will save a lot of money. Not sure if you know this but your total storage would be ~7.2 TB, that is a lot of storage, exactly what I have been running for a few years now. If you were to switch over to the 2TB drives there is one major gain from that which is it will take less time to resilver (replace) a drive failure. The down side is you are looking at slightly more money and slightly more overall power consumption. I believe performance wise the 6 drive is better however your system isn't a high grade server so you may never even notice it. There are pros and cons to just about every choice building a system.

That looks very wallet friendly. Less heat sounds nice too.

I think I'll switch to the green drives and not even consider running the VMs off the NAS. I appreciate the help from all of you. An active and friendly community sure makes it much easier for newbies.