BUILD New FreeNAS Build, Feedback/Suggestions?

[eliteassassin]

Ok, so I have decided that I am going to build a FreeNAS box and wanted to post my SPECS and what I plan to use it for to get some feedback.

FreeNAS Server Hardware:
- This is a Dell PowerEdge T100
- Intel Xeon E3110 Dual Core 3.00Ghz
- 6Gb of ECC RAM (going to upgrade this to 8GB before FreeNAS Install)
- Onboard Intel NIC + Dual Port Intel PCIe Card

Storage Drives/Configuration:
- 2 x 2TB Western Digital RED Drives, configured as a stripe 1 drive per VDEV (no redundancy)
- 1 x 300GB Western Digital Velociraptor Drive
- 1 x 120GB Intel SSD

I do recognize that I am taking a risk by not using any redundancy and I am aware that if one disk fails in my VDEV that I will lose the entire ZPool, however redundancy is not needed in any way for this setup (home use) however a complete and reliable backup is, see below.

Backup Solution/Server:
- Plan on using CrashPlan to backup the data on the FreeNAS Server to a second server and to the CrashPlan Cloud (this server will not run FreeNAS). This server will consist of a single 3TB Western Digital RED Drive with the ability to add a 2nd 3TB drive in the future if needed. While once again I do acknowledge that I am assuming a level of risk by not using redundancy I do feel that my backup solution is reliable and sufficient should a COMPLETE failure of the FreeNAS server occur.

FreeNAS Usage:
- The 2 x 2TB Western Digital RED Drives will be used for general file storage and to house my media collection (music, videos, photos, movies, etc)
- The 120GB Intel SSD will be used to host a couple iSCSI drives for some virtual servers that I will be running on a separate box within a VMWare ESXI environment
- The 300GB Western Digital Velociraptor Drive will be used to host a couple of iSCSI drives for some virtual servers that will be running on a separate box within a VMWare ESXI environment, that demands less performance then the ones running on the Intel SSD Drive
- Other then the items above the FreeNAS Server will not do anything else except run the CrashPlan application for backup purposes. Please note that no other computers\servers will backup to the FreeNAS Server this will be done on a separate box not running FreeNAS.
- While I do acknowledge that there can be some high hardware requirements related to using iSCSI since this is only going to be used in a home environment the throughput/IOPS will be very low

Thank you in advanced!!!

 

[cyberjock]

Do you plan to have redundancy for any pool? The reason I ask is that its *very* illconceived to store the system dataset on a pool that doesn't have redundancy. If the pool that the system dataset resides on has problems, your whole server will go down. Not just that pool.

The CTO of iXsystems said something to someone that did a nonredundant pool that was like "if you are stupid enough to use ZFS with no redundancy, you deserve the consequences of that poor choice" and I pretty much agree.

If you are wanting to use ZFS, you should, as a natural extension of that, *want* redundancy. If you don't plan to use redundancy you're probably better off sticking with what you are used to.. linux/windows/whatever. ZFS does NOT forgive you when things like metadata get corrupted. You'll go from a pool that is just fine to a pool that is inaccessible and unrecoverable data the next second.

This just seems like a bad idea and you need a little nudge in the right direction.. so hopefully you'll do the right thing and add redundancy.

 

[eliteassassin]

I hear you...

So just so I am clear, if I was storing the "system dataset" on the above config and this dataset became corrupted/destroyed I would loose the data on every single hard drive in the FreeNAS Server, correct?

Possible solution to this would be to add a 3rd 2TB Western Digital RED drive which would allow the one pool to use a RAIDz1 configuration, which would allow for a single disk failure. (and yes I do understand that you run the risk of another disk failure when re-building the 1st failed drive). Now (warning stupid question) with RAIDz1 that would = 6TB total storage with only 4TB being usable correct?

Playing devils advocate... Worse case scenario even with no redundancy is that you lose the entire FreeNAS server and everything on it which would then simply require rebuilding all the storage pools and restoring the files via Crashplan from the local backup. However at the same time I understand where you are coming from.... Without any level of redundancy should data become corrupted for any reason a "scrub" would not be able to fix said data, correct? Would the "scrub" still be able to warn me that the data has been corrupted?

 

[eliteassassin]

Cyber,

Could I install a 2nd SSD Drive and set that up as RAID 1 and put that "system dataset" along with the iSCSI's that I want to host on there? Would this solve the issue described above? Would this increase performance of the NAS?

 

[cyberjock]

Cyber,

Could I install a 2nd SSD Drive and set that up as RAID 1 and put that "system dataset" along with the iSCSI's that I want to host on there? Would this solve the issue described above? Would this increase performance of the NAS?

It wouldn't increase the performance of the NAS, but it would resolve the concerns of .system causing "problems" for you.

 

[pschatz100]

What I don't understand is why you would not invest in another disk. Sure RaidZ1 is not the best choice, but it is viable along with a good backup.

Is your time worth so little that you would do all this work, and leave it at risk for the sake of one disk? Makes little sense to me.

 

[cyberjock]

Ultimately I think the OP is going to find that even though the VMs are stored on SSDs, their performance is gonna be dog slow because with just 8GB of RAM it's gonna SUCK. If the system can't keep up with the I/O required, then you start losing writes. Next thing you know your VMs are corrupted and you're asking yourself "but, I'm on ZFS and a scrub says everything is fine".

The ARC is gonna be turning over so incredibly fast it's probably gonna be a fail. :/ 8GB of RAM only give you about 4GB of ARC, and that is NOT a lot of space for ZFS to work with.

 

[eliteassassin]

What I don't understand is why you would not invest in another disk. Sure RaidZ1 is not the best choice, but it is viable along with a good backup.

Is your time worth so little that you would do all this work, and leave it at risk for the sake of one disk? Makes little sense to me.

At what point did I say I was not willing to invest in another disk?

 

[eliteassassin]

Ultimately I think the OP is going to find that even though the VMs are stored on SSDs, their performance is gonna be dog slow because with just 8GB of RAM it's gonna SUCK. If the system can't keep up with the I/O required, then you start losing writes. Next thing you know your VMs are corrupted and you're asking yourself "but, I'm on ZFS and a scrub says everything is fine".

The ARC is gonna be turning over so incredibly fast it's probably gonna be a fail. :/ 8GB of RAM only give you about 4GB of ARC, and that is NOT a lot of space for ZFS to work with.

If you don't think there is enough RAM why not say so, and then make a suggestion for how much I should use?

 

[cyberjock]

I'd go to 32GB of RAM minimum. Yes, this now means you are buying new hardware, and it also means you'll have to buy E5 hardware, which is also expensive.

That's the problem, and I figured you didn't want to hear it because so often people show up and expect a minimum requirements system to run VMs ideally. They won't. Not by a longshot.

 

[eliteassassin]

...Shouldn't make assumptions just because of a few bad eggs..

I am thinking at this point that the better option would be to move my SSD and Western Digital Velociraptor drive to the actual virtual host. By doing this it would eliminate the need for the extra hardware required and would eliminate the possibility of the local network becoming an issue. I could then implement the FreeNAS box using the specs/configuration above to store my actual files on.

 

[cyberjock]

...Shouldn't make assumptions just because of a few bad eggs..

I am thinking at this point that the better option would be to move my SSD and Western Digital Velociraptor drive to the actual virtual host. By doing this it would eliminate the need for the extra hardware required and would eliminate the possibility of the local network becoming an issue. I could then implement the FreeNAS box using the specs/configuration above to store my actual files on.

That is exactly what I would do unless you are expecting to spend lots of cash. ;)

 

[eliteassassin]

What do you think of my backup strategy with CrashPlan?

 

[cyberjock]

I don't use Crashplan (I do my own backups locally and to a friends house). So I can't really judge it since I've never used it.

 

[pschatz100]

At what point did I say I was not willing to invest in another disk?

From most of your posts, it looks like you are using existing hardware to somehow piece together a system. The discussion about where to put the .system dataset is, to me, an example of how to make something that could be simple... more complicated than it needs to be. Sure, there are lots of ways to configure and tune a FreeNAS - but just because you CAN do something, doesn't make it the best decision.

If I were in your shoes, I would consider a FreeNAS with three or four 2Tb drives (the two you have plus one or two more.) Since you plan for a solid backup strategy, I personally think three 2Tb drives in RAIDZ1 would be OK (but there are others who might not agree.) The point is to give yourself a solid and reliable foundation. After that, you can add "extra services" as desired, such as your VM's. Keep in mind Cyberjock's comments about memory requirements for running VM's - you might want to do some searching in the forum about this.

I run two backups: one local and one to an external device that is stored off-site. My data is divided into two groups: critical and non-critical (media and other things that don't change very often.) My critical data is backed up every night. Non-critical data is backed up once a month. I have been thinking about using Crashplan as another layer of redundancy for critical data, but have not used it. In the event of a major catastrophe, restoring a huge quantity of data from Crashplan would be a very slow process unless you pay for one of their expedited services.

Don't forget to budget for a UPS. Your system will thank you for reliable power.

 

[eliteassassin]

From most of your posts, it looks like you are using existing hardware to somehow piece together a system. The discussion about where to put the .system dataset is, to me, an example of how to make something that could be simple... more complicated than it needs to be. Sure, there are lots of ways to configure and tune a FreeNAS - but just because you CAN do something, doesn't make it the best decision.

If I were in your shoes, I would consider a FreeNAS with three or four 2Tb drives (the two you have plus one or two more.) Since you plan for a solid backup strategy, I personally think three 2Tb drives in RAIDZ1 would be OK (but there are others who might not agree.) The point is to give yourself a solid and reliable foundation. After that, you can add "extra services" as desired, such as your VM's. Keep in mind Cyberjock's comments about memory requirements for running VM's - you might want to do some searching in the forum about this.

I run two backups: one local and one to an external device that is stored off-site. My data is divided into two groups: critical and non-critical (media and other things that don't change very often.) My critical data is backed up every night. Non-critical data is backed up once a month. I have been thinking about using Crashplan as another layer of redundancy for critical data, but have not used it. In the event of a major catastrophe, restoring a huge quantity of data from Crashplan would be a very slow process unless you pay for one of their expedited services.

Don't forget to budget for a UPS. Your system will thank you for reliable power.

Thanks for the input...

Its not that I am piecing a system together, my existing server the Dell T100 is not powerful enough CPU wise (not enough cores) for some of my media streaming needs so I am replacing it with a new server and thought that I could just re-purpose this one to be a FreeNAS box. While I feel very confident that a Dual Core 3.00Ghz Xeon processor is more then enough for FreeNAS my area of concern is the RAM as it is limited to a max of 8GB.

The largest reason for not wanting to use redundancy is the lack of ability to expand the storage space.... If I were to install 3 x 2TB Hard Drives using RAIDz1 that would give me a total of 6TB of storage with only 4TB of usable space if in the future I needed to expand my storage capacity I am forced at that point into adding 3 more hard drives... Where as if I go the route of Raid 0 I can just add drives as needed with of course the concern that there is no redundancy... However all of this being said I think that you guys have convinced me at this point that I need to have some level of redundancy on the primary storage, the files stored on the SSD and Velociraptor Drive are of no importance if they are lost. I am debating now on which RAID method I should use, I have been tossing the idea of RAID 5 and RAID 10 around? Thoughts/suggestions? I don't want to kill the performance of my NAS with RAID as this will be severing uncompressed blu ray rips.

Oh and as far as CrashPlan goes I currently use this and have recovered files (whole systems) with out issues. I back up all of my current files to a local server and to the cloud using CrashPlan. The plan at this stage will be to mount the FreeNAS storage as a network drive on Ubuntu, install CrashPlan and back up the NAS via this method. While Crashplan can be directly installed on the FreeNAS in a jail this is not something that is supported by CrashPlan, whereas the above method is and I would prefer to use a backup method that is fully supported.

 

[eliteassassin]

So here is what I am thinking, thoughts are welcome...

I am going to configure the FreeNAS server with 4 x 2TB WD RED Drives giving me a total of 8TB of storage space with 6TB of usable space and then on my backup server I will install 2 x 3TB WD RED Drives.

What is the related performance impact of using RAIDz?