Developer musings about the future of FreeNAS

[jpaetzel]

I don't stay as involved with the FreeNAS community as I would like. There are a couple of (very good) reasons for that, but it's regrettable nonetheless.

I would like to take a few minutes to ramble while FreeNAS 8.2.0-RELEASE is uploading to Sourceforge.

We haven't unleashed a FreeNAS with ZFS v28 on the world yet, but it's coming really really soon. (FreeNAS 8.2 is going to be really short lived, like 8.3.0-BETA in a week or so.)
There are some caveats with ZFS v28 and FreeBSD 8.3 that I thought I would bring up.

1) It's slower than v15. I guess you'd expect that, more features at the cost of performance. Generally it seems about 10-15%, but of course that's simply a generalization, there are cases where it's faster.

2) zpool upgrade is a one way street. You can run a v15 pool on a system with a newer ZFS version, but once you run zpool upgrade the pool is unimportable on older systems.

3) dedup is next to worthless. It takes a TON of RAM, and when it runs out of RAM the performance craters, and I mean craters. I've seen snapshot deletes take hours. To add insult to injury, you think the box has locked up because it's gone catatonic, so you reboot it, and then the system panics on reboot because it runs out of RAM trying to import the pool, and swap won't help you here, you need RAM, and a lot of it. We've seen cases where the import takes 5GB of RAM per TB of deduped data to import the pool. Use compression, in many cases it offers comparable space savings to dedup with far less performance hit. (In some cases you trade CPU time for disk I/O and it's a performance win)

4) The mps driver in FreeBSD 8.3 (6 Gbps LSI SAS HBAs) is *really* twitchy about firmware versions on the controller. You absolutely *need* to be running phase 11 or later firmware on the controller or bad things will happen to your pool.

Of course, there are good things about ZFS v28 as well. It's much better at importing damaged pool, it can survive ZIL failure or detachment, it has RAIDZ3.

FreeNAS really needs zfsd, and we're getting there, but it won't be a feature of FreeNAS 8.3.x. It's too new, too beta, and relies on kernel features that are making their way into FreeBSD. (zfsd won't be in the upcoming FreeBSD 9.1-RELEASE either)

That's about it for now, thanks for using FreeNAS, I hope everyone has as much fun with plugins and FreeNAS 8.2.0 as we have had testing and developing them.

 

[sunflashx]

1) It's slower than v15. I guess you'd expect that, more features at the cost of performance. Generally it seems about 10-15%, but of course that's simply a generalization, there are cases where it's faster.

Is this performance loss related to what version of ZFS the OS supports, or the version the pool is on?

If it's the pool version, is it possible to select which version the pool is built with when creating a new pool?

 

[Daisuke]

Thanks for the info and great work, J.

Floren

 

[paleoN]

3) dedup is next to worthless. It takes a TON of RAM, and when it runs out of RAM the performance craters, and I mean craters. I've seen snapshot deletes take hours. To add insult to injury, you think the box has locked up because it's gone catatonic, so you reboot it, and then the system panics on reboot because it runs out of RAM trying to import the pool, and swap won't help you here, you need RAM, and a lot of it. We've seen cases where the import takes 5GB of RAM per TB of deduped data to import the pool. Use compression, in many cases it offers comparable space savings to dedup with far less performance hit. (In some cases you trade CPU time for disk I/O and it's a performance win)

Perhaps this will be a future Performance post?

I realize you already touched on the major points but a more complete post might help the thick-headed or slow-witted. ;) I've already read enough about it to know I won't be using it as:

• In the first place my data dedups poorly, so why would I dedup?
• I don't come close to the hardware, RAM, requirements.

 

[jgreco]

There are environments where dedupe is incredibly worth it, but sometimes there are alternate ways to accomplish similar things.

My guess is that dedupe will remain primarily of use in virtualization and backup environments, and at least on the virtualization side, there is the possibility to design your storage systems to make the most out of a smaller dedupe datastore (think: maybe a TB of incredibly fast SSD) while using a traditional ZFS pool for additional storage.

One size fits all storage systems are easier to use, yes, but making the most of your hardware is still a bit of an art. Sounds like our jobs are safe for a while ;-)

 

[dziring]

I show my ignorance: what is zfsd?

 

[Gnome]

He is probably talking about a ZFS daemon.

 

[jgreco]

zfs runs in kernel space, but as it tries to be a comprehensive storage solution, it is sufficiently complicated that there are really some things that cannot be done properly from kernel space, and it looks to me like zfsd is the intended method to address these deficiencies. As it stands, for example, it is a bit difficult for certain device status updates, normally handled via devd, to be integrated cleanly, and handling errors, replacing disks, and other higher level management functions one might normally expect a RAID controller to automagically do are not readily possible in the kernel framework. zfsd is expected to address a lot of that.

Some developer is welcome to correct me if that's wrong.

 

[deludi]

Freenas 8.3

Hi,

Do you know when Freenas 8.3 stable is released?

Best regards,

Dirk Adamsky

I don't stay as involved with the FreeNAS community as I would like. There are a couple of (very good) reasons for that, but it's regrettable nonetheless.

I would like to take a few minutes to ramble while FreeNAS 8.2.0-RELEASE is uploading to Sourceforge.

We haven't unleashed a FreeNAS with ZFS v28 on the world yet, but it's coming really really soon. (FreeNAS 8.2 is going to be really short lived, like 8.3.0-BETA in a week or so.)
There are some caveats with ZFS v28 and FreeBSD 8.3 that I thought I would bring up.

1) It's slower than v15. I guess you'd expect that, more features at the cost of performance. Generally it seems about 10-15%, but of course that's simply a generalization, there are cases where it's faster.

2) zpool upgrade is a one way street. You can run a v15 pool on a system with a newer ZFS version, but once you run zpool upgrade the pool is unimportable on older systems.

3) dedup is next to worthless. It takes a TON of RAM, and when it runs out of RAM the performance craters, and I mean craters. I've seen snapshot deletes take hours. To add insult to injury, you think the box has locked up because it's gone catatonic, so you reboot it, and then the system panics on reboot because it runs out of RAM trying to import the pool, and swap won't help you here, you need RAM, and a lot of it. We've seen cases where the import takes 5GB of RAM per TB of deduped data to import the pool. Use compression, in many cases it offers comparable space savings to dedup with far less performance hit. (In some cases you trade CPU time for disk I/O and it's a performance win)

4) The mps driver in FreeBSD 8.3 (6 Gbps LSI SAS HBAs) is *really* twitchy about firmware versions on the controller. You absolutely *need* to be running phase 11 or later firmware on the controller or bad things will happen to your pool.

Of course, there are good things about ZFS v28 as well. It's much better at importing damaged pool, it can survive ZIL failure or detachment, it has RAIDZ3.

FreeNAS really needs zfsd, and we're getting there, but it won't be a feature of FreeNAS 8.3.x. It's too new, too beta, and relies on kernel features that are making their way into FreeBSD. (zfsd won't be in the upcoming FreeBSD 9.1-RELEASE either)

That's about it for now, thanks for using FreeNAS, I hope everyone has as much fun with plugins and FreeNAS 8.2.0 as we have had testing and developing them.

 

[Gnome]

Hi,

Do you know when Freenas 8.3 stable is released?

Best regards,

Dirk Adamsky

I'm not a FreeNAS developer but this would probably be a good way to judge the progress:
http://support.freenas.org/roadmap

I'd say another week or so.