Interesting ZFS article

Status
Not open for further replies.

Whattteva

Wizard
Joined
Mar 5, 2013
Messages
1,824
I don't know how many of you here visit DistroWatch, but I happen to be a frequenter there as they run pretty nice and informative weekly articles about Linux and BSD's and open source stuff in general.

This week's issue was a particularly special one, however, since it features this head-turning headline:
"Myths and Misunderstandings: ZFS"

Give it a read here and let me know what you guys think.

PS: As always, please keep discussions civil :D

EDIT: Updated with a hopefully more reliable link.
 
Last edited:

Starpulkka

Contributor
Joined
Apr 9, 2013
Messages
179
Well.. you tecnically can set ZFS filesystem to do all those things but not in sametime , that is the beaty in ZFS.
(ofcourse all setups cant be inplace sametime, as an example deduplication 8GB ram machine versus 1TB ram machine, it is not really happening on other setup)


But if you heavily cut ZFS in the end you will loose all those good things, what ZFS is all about.
As for zfs on linux i know a company here in Finland who has successfully worked, i think it comes 4 years now when linux zfs storage business started. But it is not free, but on other side everything works, mails, leds,speakers,encryption, etc.

Of course who write that text on distropage he talks free linux stuff and i think that freely avaiable linux stuff is shit at best.

Edit:
Whoa now.. Thats a pretty darn huge statement that's simply false.
Last time i tryed some OCR linux software it didnt reconize ä,ö,å correctly. And all media players i tryed vlc, media player, and some other players i dont member, videos just didnt play corretly. And local tv videos didnt show, as an example www.katsomo.fi or www.ruutu.fi where you can watch shows and news.

But you clearly are a correct guy for a linux setup job.
I am willing to try that debian distro. Could you /would you recommend some free good stuff for OCR and pdf maker, and how i could watch silverlight videos? I believe you can PM to me, i would lovely be wrong.

Edit: thought so. In real world you need more than a stable linux kernel.
 
Last edited:

DataKeeper

Patron
Joined
Feb 19, 2015
Messages
223
Of course who write that text on distropage he talks free linux stuff and i think that freely avaiable linux stuff is shit at best.

Whoa now.. Thats a pretty darn huge statement that's simply false. While some freely available Linux "stuff" is crap at best most freely available Linux stuff can be great. Just one example would be Debian Linux which is 100% free stuff and is likely one of the most rock stable distributions available. It's stable tree is just that.. Rock stable.. Something many other software companies, including FreeNAS, should mimic.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
I don't know how many of you here visit DistroWatch, but I happen to be a frequenter there as they run pretty nice and informative weekly articles about Linux and BSD's and open source stuff in general.

This week's issue was a particularly special one, however, since it features this head-turning headline:
"Myths and Misunderstandings: ZFS"

Give it a read here and let me know what you guys think.

PS: As always, please keep discussions civil :D

I haven't opened it, but just based on the title, I'm willing to bet he took a few liberties and doesn't actually understand stuff and instead quotes other people to make his discussion valid. I'll read it later and comment.

From my experience, most (but not all) people that try to discuss inaccurate information in ZFS add their own inaccurate information because they don't even know themselves.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
I haven't opened it, but just based on the title, I'm willing to bet he took a few liberties and doesn't actually understand stuff and instead quotes other people to make his discussion valid. I'll read it later and comment.

From my experience, most (but not all) people that try to discuss inaccurate information in ZFS add their own inaccurate information because they don't even know themselves.
Let's say he's just not part of the "Every bit is sacred" crowd. Of course, we know that if a bit is wasted, God gets quite irate.

 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
It appears he has removed his article. I click on it and it has no body.

Just reading the comments made, he seems to have stirred the hornet's nest with some comments about ECC. Well deserved too IMO and the person that quoted him and responded is totally correct.

It also appears that he argued back at the person with ECC, with comments that are laughable as he clearly has NO understand of what ZFS does or how it works...
>> "No. It doesn't *only* apply. It's strongly recommended as business data = money. However, my family photos from years gone by are priceless to me and I want to take precautions to keep them safe forever. Yes, I back them up, but a flipped bit won't show up until years down the line when my grandkids want to look at the photos."

I do not think storing family photos will be affected by the use of ECC RAM, for two reasons. First, a few bit flips are unlikely to be noticed while viewing the photos. Your photos are likely to be affected by bitrot less than a physical phtoto would fade over time. Second, unless you are looking at (and editing) the photos on a regular basis, the photos will spend almost zero time in RAM. Photos will almost certainly spend 99.9% of their lives on a hard drive or other storage media. Having ECC RAM would only affect your photos if you were opening them, editing the photo and then saving the result on a regular basis, which seems unlikely. ECC RAM will not protect you against bitrot on the storage media.

Financially you (and the photos) would be much better off investing in an extra external hard drive or optical media so you can store multiple backups of your photos. ECC may have uses outside the enterprise, but protecting family photos isn't likely to be one of them.

Yeah, he has ZERO concept of ZFS if he's giving advice like "better off investing in an extra exteranl hard drive". WRONG. The whole point of ZFS is so you can prove, for certainty, that all the bits are actually there. You cannot do that, even if you have multiple copies, unless you intend to cross-compare from two sources (and you'd be laughed at if you went to that kind of extent to validate you had 1 copy that was "safe").

Family photos isn't likely to be one of them!? Has he even see the FreeBSD, ZFS on Linux, or FreeNAS community!? We're *all* part of that group. Has he even looked at the users we've seen around here that lost their entire pools because of bad RAM?

And a few bitflips aren't likely to be noticed!? Really! There's a website somewhere that took some pictures and changed a single bit. The end result was a broken image that wasn't worth saving. So yeah.. a single flipped bit can be a big deal for you if your pictures are valuable. (In fact, the fact that some of my picture have been corrupted over the last 10+ years is *why*, after I read about ZFS, I went all-in and went to FreeNAS.)

@45: >> "Typically, you'd schedule scrubs once or twice a month. During this time, the file system checks your disk for any errors and automatically corrects them if corruptions are found. ZFS, however, cannot detect bit errors in RAM and so, this could result in the file system actually "correcting" your file that is perfectly fine, which inadvertently corrupts it instead."

You are half right. When ZFS performs a scrub it does check the data on disk against a checksum to make sure the data's integrity has not been comprimised. However, ZFS only attempts to correct the bad checksum if it is used in a RAID or mirrored disk scenario. What this means is if the checksum of one copy of the file is bad and another copy of the file is good, then the bad copy is replaced with the good copy.

For a ZFS scrub to over-write a file, it needs to have another good copy of the data to use. The corrupted file is overwritten by a known good copy. Even if ZFS mistakenly thinks a good file is really corrupted, it will only replace that file with a verified good copy of the data. This is discussed in the zpool manual page under the "scrub" sub topic.

Laughable. It won't fix a file without another good copy!? No, scrubs work at the BLOCK level. If a BLOCK is corrupted, it's replaced with data from either a mirror or parity data (that is verified good). There is no differentiation between a block of file data and a block of metadata. It is all checksummed and all replaced with either parity data or the mirror, after validating that the alternate source is good.

Clearly he does not understand the tech he is writing about.

"Armchair ZFS expert" is what I'd call him. Kind of like an Armchair lawyer, but just as clueless.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
It appears he has removed his article. I click on it and it has no body.

Just reading the comments made, he seems to have stirred the hornet's nest with some comments about ECC. Well deserved too IMO and the person that quoted him and responded is totally correct.

It also appears that he argued back at the person with ECC, with comments that are laughable as he clearly has NO understand of what ZFS does or how it works...


Yeah, he has ZERO concept of ZFS if he's giving advice like "better off investing in an extra exteranl hard drive". WRONG. The whole point of ZFS is so you can prove, for certainty, that all the bits are actually there. You cannot do that, even if you have multiple copies, unless you intend to cross-compare from two sources (and you'd be laughed at if you went to that kind of extent to validate you had 1 copy that was "safe").

Family photos isn't likely to be one of them!? Has he even see the FreeBSD, ZFS on Linux, or FreeNAS community!? We're *all* part of that group. Has he even looked at the users we've seen around here that lost their entire pools because of bad RAM?

And a few bitflips aren't likely to be noticed!? Really! There's a website somewhere that took some pictures and changed a single bit. The end result was a broken image that wasn't worth saving. So yeah.. a single flipped bit can be a big deal for you if your pictures are valuable. (In fact, the fact that some of my picture have been corrupted over the last 10+ years is *why*, after I read about ZFS, I went all-in and went to FreeNAS.)



Laughable. It won't fix a file without another good copy!? No, scrubs work at the BLOCK level. If a BLOCK is corrupted, it's replaced with data from either a mirror or parity data (that is verified good). There is no differentiation between a block of file data and a block of metadata. It is all checksummed and all replaced with either parity data or the mirror, after validating that the alternate source is good.

Clearly he does not understand the tech he is writing about.

"Armchair ZFS expert" is what I'd call him. Kind of like an Armchair lawyer, but just as clueless.

Huh, should've waited to get home to read the insanity.

Yes, I back them up, but a flipped bit won't show up until years down the line when my grandkids want to look at the photos.

Uhmm, yes. Otherwise, why have family photos at all? Ars took a random JPEG, flipped a single bit and ruined half the image.

"Meh, the next generation will take care of it." is not a valid engineering principle.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
"Meh, the next generation will take care of it." is not a valid engineering principle.

I used to work in nuclear power. Say that to the government regulators. I bet that's one of the fastest ways to end up unemployed and banned from the industry for life.
 

Whattteva

Wizard
Joined
Mar 5, 2013
Messages
1,824
It appears he has removed his article. I click on it and it has no body.
Strange, I can still see the article in the link, anyone else have problems with the link?

EDIT: I tried the link again and I see what you're talking about the article not loading and only the comments loading. I think this link might work better. http://distrowatch.com/weekly.php?issue=20150420#myth

Laughable. It won't fix a file without another good copy!? No, scrubs work at the BLOCK level. If a BLOCK is corrupted, it's replaced with data from either a mirror or parity data (that is verified good). There is no differentiation between a block of file data and a block of metadata. It is all checksummed and all replaced with either parity data or the mirror, after validating that the alternate source is good.

Clearly he does not understand the tech he is writing about.

"Armchair ZFS expert" is what I'd call him. Kind of like an Armchair lawyer, but just as clueless.
Post #45 about ZFS scrub was actually me trying to somewhat explain my opinion of why I use ECC (though obviously I don't claim to be the expert on ZFS). Obviously, as you can see in post #49, he replies to me and basically says that I'm horribly misinformed and heard too many things repeatedly that it became common knowledge.
 
Last edited:
Status
Not open for further replies.
Top