Can you provide me with a reference that the block-based compression has any noticeable performance impact?
Yep.. my e1230v2 saw a decrease that was significantly enough to lower my NFS writes by almost 50%. So when all of these people at iX and elsewhere say "you won't notice" I have to laugh.
When using compression, ZFS should utilize dynamic block sizing to fit in smaller, compressed blocks. Nonetheless I assume it starts its compression efforts based on the default block size (i.e the incoming chunk of data to be compressed). I'm not sure about the default block size in FreeNAS (I think I remember that it was rather small compared to other OS defaults), but when using for example a default size of 1024 kb, this shouldn't induce too much additional effort to check once in 1 MB for compressibility.
ZFS uses a block size that is variable from 512-bytes to 128kB depending on the load. You cannot force a given larger size, you can only set a maximum allowed block size. 1MB would be nice, but that's currently only implemented in zpools that are v32 which is Oracle's implementation only.
I couldn't find any more specifics in the sources I found:
-
Illumos wiki article about lz4: "The extremely high per-core throughput of LZ4 [..]" (indicating actual benefits from multi-threading) and "LZ4's performance on incompressible data is very high".
-
LZ4 google code page: "typically reaching RAM speed limits on multi-core systems"
-
FreeBSD 10 commit: "
delivers very high compression and decompression performance compared to lzjb (>50% faster on compression, >80% faster on decompression and around 3x faster on compression of incompressible
data)"
-
Wikipedia article on ZFS (I know, unreliable source): "ZFS uses variable-sized blocks of up to 1024 kibibyte"
Yeah.. there's very very little actual data provided to end users. To me this is a disappointment, but it's pretty typical for ZFS.
For example:
- When people spew shit like "around 3x faster on compression of incompressible data" you've gotta talk about *what* it's 3x faster then. 3x faster than gzip9 compression, even on the most expensive CPU money can buy is still unacceptable speeds.
- Your google code link mentions single thread performance of 422MB/sec compression on an i5-3340M. I can't find *solid* evidence that LZ4 is actually multi-threaded in the ZFS implementation. But, 422MB/sec is pretty crappy for an i5-3340M in the big picture. Sure, home users won't likely have a problem. But 422MB/sec is 1/2 of a single 10Gb link. And many users with G3220s and the new Intel Atoms are using CPUs that are slower than an i5-3340M, so that's not necessarily a "good thing" anyway. But it may be multi-threaded. It may not. It also may be multi-threaded only in certain circumstances.
In short, trying to sell me on a compression routine where there's very little transparency is pretty disappointing. This is pretty par-for-the-course for ZFS though. It's not a FreeNAS problem in my opinion, except that it's the default.
Also, I firmly believe that compressions should
NEVER be enabled by default. And even more so, if it is going to be enabled by default, it should be 100%
*blatantly* obvious to the user. For example, during pool creation there's a dropdown to choose compression with the default being lz4. This sneaky enable it and not say a word about it is extremely dishonest. Just look at the
9.2.1 announcement. No comment at *all* about the change in default compression. Conspiracy theorists can now rejoice.
And once a company has shown where it's transparency is or isn't, it's not far fetched to wonder what other things they may be doing behind the scene that you may not be too appreciative of. Some forum users have accused iX of taking a stable product and going to bleeding edge with these changes to the risks not being made known at all unless you look VERY closely and are an advanced user. For example, 9.2.1 brought about the feature flags enable_txg, hole_birth, extensible_dataset, and bookmarks. Those are in the FreeBSD CURRENT branch only according to
http://open-zfs.org/wiki/Features. I don't know how much you keep up with the different branches, but CURRENT is for developers only because it may contain serious bugs. Is that the kind of thing *you* want storing your precious data? I'm still on v28 because I'm not interested in these games. I'm also on 9.2.0 because I have to wonder what other BS is being added I don't really want to be involved in.
My trust in this project has rapidly diminished in recent months with many things that have become "standard operating procedures". It's making me wonder when that bug is going to come out that's gonna eat pools everywhere when they release some poorly developed ZFS feature without adequate testing. Talk about the community losing faith in a project. That'll be one for the record books. I don't think it's a matter of if, I think it's a matter of when.
Just look at what has happened in the last 5 months with development of FreeNAS.
- 9.2.0-RCs had some major bugs. The expectation across the IT industry has always been that RCs should good enough to be a RELEASE version and they're just making sure that there's nothing too nasty before making it official. Instead, 9.2.0 had two RCs, and there was almost a 3rd. Why? Because 9.2.0-RCs had major problems. A lot of people put it on their home servers and had major problems with it that made their server unusable unless they rolled back to 9.1.1.
- Then, since the community had lost confidence, when 9.2.1-RC came out only a small subset of users were even willing to try it. So 9.2.1 went to RELEASE version, where everyone felt safe that the problems had been worked out. Fast forward through 9.2.1.1, 9.2.1.2, 9.2.1.3 and now 9.2.1.4-beta and the problems are finally almost fixed. I say almost because the 9.2.1.4-beta has more fixes to go before it hits release. Now keep in mind that 9.2.1 came out February 7th. It's April 17th and we're still struggling to get a 9.2.1.x that's keeping people happy. I bet I've posted in at least 20 threads in the last 2 months to go back to 9.2.0 and not try to make 9.2.1 work because the average user can't solve the problems on their own. But tons of users everywhere are stuck with broken servers that are behaving erratically, some aren't even functional, etc for various reasons and don't have a choice to go back to 9.2.0(or earlier).
Mind you, iX guys I've chatted with said they tested 9.2.1 extensively in the lab and never had the problems we've seen in the real world. Some of the guys have said that they are shocked at how messed up 9.2.1.x has been for the end-users. I think they are being honest when they say that(despite what conspiracy theorists might want you to think). So I have to assume that either the QA testing isn't really reproducing good real-world scenarios, the QA testing isn't effective at finding problems, or both. Of course, having to release a 9.2.1.1, 9.2.1.2, 9.2.1.3, and eventually a 9.2.1.4 tends to make me think nothing has changed, no lessons have been learned about whatever processes are being used to vet code, and that makes me even more fearful of what future versions will have in store for some people.
So, based on all of this, here's my prediction for 9.2.2. It's gonna get to RC and everyone's going to be just as fearful of it because faith has been lost from all the other BS users have had to deal with for the last 6 months. 9.2.2 is going to roll out to RELEASE and we're going to see a repeat of 9.2.1's problems. I have no doubt that we'll be telling a lot of people then to go back to 9.2.0 if they can still. Of course, from iXsystem's perspective, "9.2.0 is too old. Die in a fire." Yet plenty of us are still using 9.2.0 because the "upgrades" aren't exactly functional for many people.
One thing is for sure though. Using the community as the QA process is failing miserably. Companies that use the public or community for their QA process that aren't fully transparent, honest, and responsive to user feedback have major problems.
I don't know about the rest of the world, but I don't upgrade my server because its new and shiny and therefore
must be better. I upgrade because it *is* better. And so far, 9.2.1.x has been a big fail for many reasons. I thought 9.2.1.4-beta was going to finally be the 9.2.1 release we didn't have. But, apparently the 9.2.1.4-beta still doesn't have all of the big ticket fixes. :(
