UFS or ZFS

[robertw]

hi all,

made a freeNAS server its up and running but i want to know what format to use for my HD's UFS or ZFS

i have heard UFS is rock solid and ZFS is fast also includes added features snapshot, replication

i have 5x1TB drives so i can use RAID 5, im using it to backup importnat data, im using it for all the time machine to back up all the macs

any views thoughts what i should do?

rob

 

[cyberjock]

UFS is dead for FreeNAS. All future versions will not support UFS. So your options seem pretty clear. ;)

 

[Knowltey]

made a freeNAS server its up and running but i want to know what format to use for my HD's UFS or ZFS

i have heard UFS is rock solid and ZFS is fast also includes added features snapshot, replication

First off, how much RAM are you running in this build, and is it ECC? If it's not at least 8GB then you should avoid ZFS until you can meet those requirements because if you have less than 8GB you are below the minimum system requirements for ZFS.

If the RAM you are running isn't ECC compatible then you should not run a fileserver until you can get hardware that runs ECC RAM. If you're running a fileserver then you should be running ECC RAM otherwise you are defeating the purpose of a fileserver which is file storage and data integrity.

For FreeNAS you should pretty much almost always be running ZFS except in very select few instances where you don't need to store much data or meet the 8GB ECC RAM minimum system requirements of the ZFS filesystem. ZFS is simply a better filesystem for the purpose of a fileserver. And in those aforementioned select few instances FreeNAS isn't the optimal tool for your purpose anyways. It may do stuff, but anything that you would be doing using FreeNAS in it's intended manner you should be running ZFS, not UFS. That and as cyberjock said UFS is going to be deprecated in future release versions of FreeNAS so you might as well just forget about running it on FreeNAS.

i have 5x1TB drives so i can use RAID 5, im using it to backup importnat data, im using it for all the time machine to back up all the macs

Two things,

Do **NOT** use hardware raid if you are using the ZFS filesystem. It handles it's disks on it's own via software and it CANNOT do pretty much any of the reasons you use ZFS for is you're running it on a hardware RAID as it is designed with direct access to the member disks in mind. Pretty much all of the important protection features of ZFS are broken if you are running it through the abstraction of a hardware RAID. If you are running ZFS the equivalent to RAID5 is RAIDZ1, but moving on to the next point now:

Also, do **NOT** run RAID5/RAIDZ1. Just don't. It is highly unsafe for disks of the size you are looking at. If our member disks are close to a terabyte in size don't even consider 5 or Z1. Hell, I wouldn't even consider it for disks if they are anywhere over 500GB in size, just not worth the risk. They can only handle 1 disk failure, and due to the nature of how the rebuild process works on RAID5/Z1 you have a large likelihood of a second disk failing during the rebuild process. I believe it is an 8% chance of second disk failure during rebuild. It simply isn't worth the risk.

If you can't afford to get one more disk to run RAIDZ2 (ZFS version of RAID6) then I would advise to run a RAID 1+0 instead which can handle certain two disk failure as long as they aren't on the same stripe. Don't even consider RAID5/Z1, just forget that it even exists, it is not longer considered safe for data protection with disks as large as they are now.

**Do not use RAID5/Z1 for any data that you value.**

 

[9C1 Newbee]

If you have the minimum hardware requirements such as 8 or more gigs of ECC ram and stuff like that, I'd roll with ZFS.

I would also get an additional TB drive and run raidz2.

And I REALLY think you should read cyberjock's newb guides BEFORE you do anything.

EDIT: I thought I was gonna be the first post. You guys are no joke! LOL

 

[robertw]

hardware specs are 8GB DDR2 ECC RAM and 5 x 1TB HD

how come dont use/run RAID5/Z1, as all the other servers we use RAID 5 but there not freeNAS machines tho and we havnt had any trouble with them for good old few couple of months

will i get the same performance running RAID 1+0 than RAID5?

 

[Knowltey]

It's not a FreeNAS thing. RAID5 (Z1) is simply not as safe for large capacity disks as it is for smaller capacity disks. As I stated earlier, if you get a disk failure, you have an 8% chance of a second disk failing during rebuild which will kill the pool and you will lose all data. That's on 1TB disks, it only increases vastly with larger disks.

As long as you are aware of and fine with that possibility then feel free to use 5/Z1. I'd personally recommend just getting to Z2 though since you are so close to it already.

http://www.zdnet.com/blog/storage/raidfail-dont-use-raid-5-on-small-arrays/483 is a popular post on the matter.

Performance-wise RAID10 should actually be faster than RAIDZ1 (I'm not calling it RAID5 here because as I stated before RAID5 would mean that you are running it through a hardware RAID which is not recommended with ZFS. ZFS relies on direct access to the disks.)

 

[DrKK]

It's not a FreeNAS thing. RAID5 (Z1) is simply not safe for large capacity disks. RAID5 is widely held in the industry as a "worst practices" only ever so slightly less stupid than RAID0.

With all due respect, Knowltey, I wish you wouldn't promulgate this kind of horseshit.

RAID0 isn't "ever so slightly less stupid than RAID5", it's ****MONUMENTALLY MORE STUPID**** than RAID5. It's not even close. When you say things like this, less sophisticated lurking users will read it and will be misinformed.

And for the record, RAID5/RAIDZ1 does have some theoretical problems these days with high capacity drives. But the reality and mathematics is MUCH MUCH MUCH MUCH MUCH MUCH more nuanced, in real practice, than "RAIDZ1 is dead" or "RAID5 is dead", when you're talking about real practice.

Yes, if you're in Enterprise IT, and you're making several 12-drive vdev's with RAIDZ1, then you're a jackass that needs to be fired. But, if you're Suzy Creamcheese, and you're putting 4 2TB drives together for your movies, RAIDZ1 is MORE than ample, with several "nines" of resultant reliability.

The truth is, most people with smaller pools containing smaller-sized drives are going to be amply served by RAIDZ1 without a hell of a lot of risk to their data. For these users, forcing them to go to RAIDZ2 is a considerable (as a percentage) increase in the price of their pool and/or other hardware, and they should not be misinformed. The situation is FAR FAR more nuanced than "RAID-Z1 is dead", or "RAID5 is considered only slightly less stupid than RAID0"; the latter statement is blatantly false and misleading. Just for the record.

Let's not sensationalize or oversimplify what is a fairly nuanced situation with RAIDZ1. The truth is, many, many users are well-served, and quite safe, with RAIDZ1. It depends on more factors.

 

[danb35]

i have heard UFS is rock solid and ZFS is fast also includes added features snapshot, replication

ZFS is a lot of things, mainly focused on data integrity. Performance isn't really one of its design goals, though. Its performance is likely to be entirely adequate for most applications, just be aware that it wasn't designed to be a screaming fast filesystem.

 

[Knowltey]

With all due respect, Knowltey, I wish you wouldn't promulgate this kind of horseshit.

RAID0 isn't "ever so slightly less stupid than RAID5", it's ****MONUMENTALLY MORE STUPID**** than RAID5. It's not even close. When you say things like this, less sophisticated lurking users will read it and will be misinformed.

And for the record, RAID5/RAIDZ1 does have some theoretical problems these days with high capacity drives. But the reality and mathematics is MUCH MUCH MUCH MUCH MUCH MUCH more nuanced, in real practice, than "RAIDZ1 is dead" or "RAID5 is dead", when you're talking about real practice.

Yes, if you're in Enterprise IT, and you're making several 12-drive vdev's with RAIDZ1, then you're a jackass that needs to be fired. But, if you're Suzy Creamcheese, and you're putting 4 2TB drives together for your movies, RAIDZ1 is MORE than ample, with several "nines" of resultant reliability.

The truth is, most people with smaller pools containing smaller-sized drives are going to be amply served by RAIDZ1 without a hell of a lot of risk to their data. For these users, forcing them to go to RAIDZ2 is a considerable (as a percentage) increase in the price of their pool and/or other hardware, and they should not be misinformed. The situation is FAR FAR more nuanced than "RAID-Z1 is dead", or "RAID5 is considered only slightly less stupid than RAID0"; the latter statement is blatantly false and misleading. Just for the record.

Let's not sensationalize or oversimplify what is a fairly nuanced situation with RAIDZ1. The truth is, many, many users are well-served, and quite safe, with RAIDZ1. It depends on more factors.

Fine, I'll admit to scaremongering a mite there. Hell, that line was a quick edit in I did on lunch break without proofreading it, should've left the original intact. And yeah RAID0 is a slap across the face and "WTF are you thinking?" and RAID5 is more of an "eh, there's better options."

That said I still would highly recommend against it for any data considered important data as the OP said they would be storing on the pool unless you have a backup solution that you can depend on in case of the failure of a second disk or using smaller capacity member disks. RAID isn't backup so you should be having an adequate backup solution in place no matter if you're using JBOD or some absurdly redundant RAID level like 7+1 (that'd probably be a bit overkill).

Since he has 5 drives though, I would recommend him either get a 6th drive and run Z2 or use 4 of the disks for 1+0 and the 5th disk perhaps for 2nd copies of the really extra important data.

 

[robertw]

thanks guys for all your help much appreciated

 

[9C1 Newbee]

He said his data was "important". Based on that, I would not recommend RAIDz1 in this case. But hell, that's just me.

 

[DrKK]

There is a risk with consumer grade RAIDZ1, I'm not saying there's not. But if you run the mathematics on it, it's really not that bad, for Joe Blow homeuser with 3 or 4 or 5 drives. The notion that "RAIDZ1 is dead" is really quite a misnomer. If you've got 3 or 4 or 5 WD "red" 2TB's in a RAIDZ, and you're properly curating the system (scrubs, good thermals, SMART tests), your risk of a RAIDZ failure is exceptionally low.

Yes, RAIDZ is dead for a 12x3TB pool, for damn sure. But it's alive and well for a 4x2TB pool.

 

[Knowltey]

There is a risk with consumer grade RAIDZ1, I'm not saying there's not. But if you run the mathematics on it, it's really not that bad, for Joe Blow homeuser with 3 or 4 or 5 drives. The notion that "RAIDZ1 is dead" is really quite a misnomer. If you've got 3 or 4 or 5 WD "red" 2TB's in a RAIDZ, and you're properly curating the system (scrubs, good thermals, SMART tests), your risk of a RAIDZ failure is exceptionally low.

Yes, RAIDZ is dead for a 12x3TB pool, for damn sure. But it's alive and well for a 4x2TB pool.

Well as you said earlier

"does have some theoretical problems these days with high capacity drives" and "people with smaller pools containing smaller-sized drives are going to be amply served by RAIDZ1"

The thing is OP is using high-capacity drives and not "smaller-sized" drives. So the risk is still there. 1T is where it is generally considered to begin having unproportional risk to other RAID levels. I personally wouldn't run RAID5/Z1 for anything over 800G.

If he had 500G disks, then there wouldn't be any real issue, but for the disks he is using I would recommend just spending an extra 60 bucks to get another 1T disk and run RAIDZ2. The higher the capacity of the disk the higher the likelihood of experiencing an URE in the space, and since an URE on a second disk rill kill the pool. There is actually a formula for calculating the likelihood of secondary disk failure, I don't have it at the moment though, does anybody else have it? Being able to have the exact number for his actual array may help him make his decision. I'm not sure of the exact number for a 5 disk RAIDZ1 at 1TB each disk. THe chance may be 5%, 1%, or maybe even less than 1%. It's really all just about probabilities here and what you consider to be an acceptable probability for your data.

 

[TXAG26]

If build cost is a concern, I've seen more issues crop up with consumer motherboards, consumer SATA controllers, and non-ECC ram.

If dropping down to RaidZ1 will allow you to spend a bit more on hardware (CPU/MB/ECC RAM), then by all means do it and use it to get server-grade components.

RaidZ2 is worthless if run on a cheap motherboad with dodgy SATA controllers using non-ECC ram. The whole purpose for running ZFS is the constant check-summing of data as its manipulated, read/write, and during scrubs. Non-ECC ram and consumer-grade SATA controllers increase the likelihood of errors being introduced into that process. With ZFS, once a component starts throwing uncorrectable errors, your done, along with your data.

I don't have any hard numbers, but it seems like consumer-grade motherboards, dodgy components, and hardware raid controllers introducing errors have been responsible for more data loss than actual HDD "crashes". Just my $0.02...