Is anyone using an alpha/prerelease FreeNAS with ZoL/ZoF as the storage subsystem? What are your impressions?

[Stilez]

I'm very interested in the eventual FreeNAS 12.0 release with ZoL/ZoF (which do we call it?) as the new build for ZFS. There are features in it that really matter for my pool and would help a lot in a practical sense.

I'm therefore interested in trying to get a feel for how close to production the new ZoL/ZoF subsystem is (as opposed to 12.0 wider development generally), and, if anyone is actively using a build of 11.x-REL or prerelease 12.0 which has the usual ZFS subsystem swapped out for ZoL.

I don't much care about the GUI/middleware/API integration of ZoL/ZoF if that's an issue, since I tend to use CLI for ZFS management and bypass the GUI. I use the GUI to manage everything else - jails, Samba, tunables, general build quality, etc, but not to directly manage my pools.

Tl,dr - I am curious if ZoL/ZoF based builds are regularly made for testers (or indeed in house if any ixSystems staff want to reply). If so, what are people's experiences and current impressions are regarding data stability, system stability, completeness, corruption risk, 11.3 compatibility, anything else interesting.

Any insights really appreciated, thanks!

 

[Yorick]

The nightlies use that code, and have the feature flags you’d expect.

11.3 compatibility: Well if you upgrade feature flags then you are stuck, no going back.

System stability: Much better now than a month ago, and, still, it’s a nightly, things break. Don’t expect any particular service to be available or crash free. SMB, for example, has been great the past few weeks, but for all I know, more SMB work is on the horizon and things may break again.

Jails have been working well for me; bhyve currently crashes. I think someone reported iSCSI crashes that are being looked at.

None of this is shocking: it’s a nightly, you expect things to be unstable and break.

Taking this into production now would be foolhardy, is my 2 cents. As for taking an alpha2 or beta1 into production - maybe, but what is the plan if you run into bugs and some services you rely on fail? You can always go back to 11.3 if you didn’t upgrade feature flags but it sounds like you want new ZFS features (which?).

 

[Stilez]

The nightlies use that code, and have the feature flags you’d expect.

11.3 compatibility: Well if you upgrade feature flags then you are stuck, no going back.

System stability: Much better now than a month ago, and, still, it’s a nightly, things break. Don’t expect any particular service to be available or crash free. SMB, for example, has been great the past few weeks, but for all I know, more SMB work is on the horizon and things may break again.

Jails have been working well for me; bhyve currently crashes. I think someone reported iSCSI crashes that are being looked at.

None of this is shocking: it’s a nightly, you expect things to be unstable and break.

Taking this into production now would be foolhardy, is my 2 cents. As for taking an alpha2 or beta1 into production - maybe, but what is the plan if you run into bugs and some services you rely on fail? You can always go back to 11.3 if you didn’t upgrade feature flags but it sounds like you want new ZFS features (which?).

At this point I'm doing future planning. I want to test out on an early build, to what extent it looks like ZoL as it already is, might mitigate issues I have with 11.3 (eg device allocation to offload metadata/DDT to very fast SSDs). That influences what hardware I might endure for a year or upgrade now because I'll need to anyway.

I have enough disks to throw together a copy pool for this. I wouldn't make any 11.3-breaking changes on my actual pool. But there will come a point in developing 12 where ZoL is good, and the few services I need are pretty good, and the areas being worked on aren't things I'm bothered by. I'm wondering how we are making progress towards that point......

 

[Yorick]

I'm wondering how we are making progress towards that point......

That'll depend heavily on your use case. In my limited testing:
- SMB works and is solid
- jails work and are solid
- bhyve works for Ubuntu; crashes for Win10

Testing I've read about on the forums:
- Fusion pools work well
- iSCSI was crashing at some point, not sure whether it's stable yet
- NFS?

I expect my Win10 test VM should be stable around Alpha 2 time frame; I know it'll fail in Alpha 1.

 

[diskdiddler]

May as well ask here - does anyone know if any new compression options come with TrueNAS 12 ? for the zvols? I see I can use lz4 and it's fast (As it has been for years) but am I imagining or was some new ones coming?

 

[anodos]

I think things with SMB should be more-or-less settling down now in 12.0. 12.1 development nightlies will continue to be... well... under development. For instance, yesterday I just merged in a large patchset from JRA that converts DNS in libads to be asynchronous. This will immensely help scalability in large AD environments, but it needs testing. I also recently ported Samba 4.13, which brought yet more VFS changes that also carry a possibility of regression.

 

[Stilez]

Just to answer my own question 5 months on. I moved my pool from 11.3 to 12-beta1, then upgraded to 12-beta2 and 2.1.

Its been 5 months now. Some bugs popped up that affect data management (eg annoyances in the replication commands, CPU starvation in scrub, but not anything that actually harms data) but using ZFS, SMB and iSCSI, it's basically been rock solid on data care, and I wouldn't go back to 11.x now if I could.

Just in case it's useful for anyone.