8.0.3 RELEASE coming soon

[gcooper]

I'm working on an 8.0.3 RELEASE for mid-December. Basic criterion in my mind for an item making the 8.0.3 release are the following:

1. Going from 8.0.1 -> 8.0.2 regressed critical functionality (example: email without authentication enabled regressed between 8.0.1-p1 and 8.0.1-p2).
2. There's an easy to exercise code path that results in a GUI traceback (example: editing users after adding them in the GUI bombs with a traceback if you press ok and didn't provide an SSH pubkey and didn't add a newline to the textbox).
3. An easy to trigger code path results in unexpected behavior (example: crontab / rsync task generation with AD users was broken because of the fact that '\' wasn't being escaped properly in the ix-crontab script).

with that in mind, here's what I have on my radar...

Goals before the release (incomplete):

Performance Items:

1. Reduce FreeNAS footprint:

a. Reduce default /var/tmp/.cache size from ~1.5GB to 1GB, or smaller. I'm working on the soaking smaller amounts right now; 256MB is the minimum I'm seeing from the ix-cache stuff that's being run against our local test AD server (used to be 1.7GB) so 512MB should be sane, but I need one or more LDAP testers to ensure that that end is sane as well.

Bugfixes:

1. AD/CIFS:

a. Fix the AD interoperability issue at boot and when enabling for the first time (see tickets: 900, 1009).

Items which will be in the release:

Enhancements:

1. Reduce FreeNAS footprint:

a. Use smaller block and frag sizes for /etc and /var (this helped a bit based on internal testing, but again.. it needs more soak).
b. Import a build tweak to nuke /var/db/pkg (I'm sorry hackers, but this is an appliance and not a general purpose machine.. and the fact that /var/db/pkg ate up 20MB+ is heartburn when the memory disk's size is limited).
c. Other build tweaks to remove non-essential features and packages from the image.

1. NAS middleware (generic):

a. Hide Etc/GMT* timezones as they're behavior is counterintuitive and conflicts with Windows 7 semantics (see r8707).
b. Add tunable / sysctl support (we need it for 8.0.3 for iX internal use, and the FreeNAS community would definitely benefit from its inclusion).

2. AD/CIFS:

a. Bump samba from 3.5.11 to 3.6.1.
b. Fix AD clock skew issues and other problems when authenticating to an AD realm (see ticket: 1056, ).

3. AFP:

a. Bump netatalk to 2.2.1.
b. Add knob for controlling the maximum number of configurable connections (see ticket: 847).
c. Only advertise services via avahi if they're enabled (see ticket: 852).

4. Rsync:

a. NFSv4 ACL support (see: r8375, r8414).

5. Misc:

a. Upgrade ataidle

Bugfixes:

1. Rsync task/crontab generation was broken with AD usernames and other unsanitized input.
2. ZFS volume deletion failed when trying to delete zvol with the name 'zvol'.
3. SSH pubkey saving was broken unless you manually add a newline to the text box.
4. Fix email regressions since 8.0.2-RELEASE-p1 (SMART emails now work with more than one recipient, non-SMTP authentication based emails work again).
5. Fix traceback when trying to edit user if ssh public key was not specified.
6. Address simple failure cases with Kerberos tickets and joining AD domains so that users could better rectify improperly configured NAS boxes.
7. There was a bug in 8.0.2 if you created a zvol named 'zvol', it wouldn't allow you to delete it from the GUI. Similar issue with other bits.
8. Fix the bug in rsync tasks where it would strip the trailing '/' off the end of the destination path.
9. Fix a validation error when editing users / groups in 8.0.2+ where if one entered in a path that wasn't valid, the GUI would traceback instead of punting the actual validation error.

Cosmetic items:

1. Change "SSH key" in GUI to "SSH Public Key".
2. Fix GUI trademarks and branding to be more consistent with proper branding, similar to what TrueNAS does today.

 

[ProtoSD]

I just built that branch last night. Here are a few things off the top of my head I'd like to see:

1) Alphabetize the services in the GUI (Think I saw you mention this elsewhere)

2) Fix the FUSE module so it loads and people can mount NTFS ( I just copy /usr/local/modules/fuse.ko to /boot/kernel & add it to loader.conf)

3) I haven't looked into why FAT/msdosfs can't be imported but that would be nice too.

4) Bump NUT to ? (not a problem for me, but think others would appreciate it)

Build version and ISO/xz files etc. are still showing 8.0.2

I can probably think of a few other minor things, but I'm sure you'll get an earful anyway ;)

Thanks for the update/info!

EDIT: Some way of resetting/clearing disks from the database when you delete a volume from the GUI. I've seen several cases where people can't reallocate disks after deleting a pool because the database still has them in use.

 

[gcooper]

I just built that branch last night. Here are a few things off the top of my head I'd like to see:
1) Alphabetize the services in the GUI (Think I saw you mention this elsewhere)

If I do this, the doc writer is probably going to hang me because then she'll have to change release docs :).

2) Fix the FUSE module so it loads and people can mount NTFS ( I just copy /usr/local/modules/fuse.ko to /boot/kernel & add it to loader.conf)

Sounds like the package is broken for starters.. but ok, sure.

3) I haven't looked into why FAT/msdosfs can't be imported but that would be nice too.

Ok. Do you have details on how things are broken?

4) Bump NUT to ? (not a problem for me, but think others would appreciate it)

Hmmm.. is there a compelling argument for doing this (in particular, I don't have access to any network based UPSes)?

Build version and ISO/xz files etc. are still showing 8.0.2

Yeah.. I haven't rebranded or committed anything over to that branch.

I can probably think of a few other minor things, but I'm sure you'll get an earful anyway ;)

Lol -- k.

Thanks for the update/info!

Np -- just trying to increase the length of the release cycles so that less dramatic changes are worked into each release.

EDIT: Some way of resetting/clearing disks from the database when you delete a volume from the GUI. I've seen several cases where people can't reallocate disks after deleting a pool because the database still has them in use.

Do you have more details? Unfortunately this section of code has been rototilled so much that porting changes here will become a nightmare (especially with the django upgrades).

 

[gcooper]

Sysctl and tunable support is another thing that I think people would like as well, but that's where I want to draw the line.

 

[ProtoSD]

Sorry about the formatting/editing. Since the Forum upgrade my editing toolbar is gone and I'm having to type the markup tags by hand/memory.

2) Fix the FUSE module so it loads and people can mount NTFS ( I just copy /usr/local/modules/fuse.ko to /boot/kernel & add it to loader.conf)

Sounds like the package is broken for starters.. but ok, sure.

There's an init script that's supposed to load it but isn't for some reason. I'm not sure if it loads it conditionally when you try to import NTFS or if it's supposed to load it on boot.

3) I haven't looked into why FAT/msdosfs can't be imported but that would be nice too.

Ok. Do you have details on how things are broken?

I'm not exactly sure what's broken, but it seems like middleware. The error is: "The selected disks were not verified for this import rules." I think the mountpoint and database might get updated, but the disk isn't accessible.

4) Bump NUT to ? (not a problem for me, but think others would appreciate it)

Hmmm.. is there a compelling argument for doing this (in particular, I don't have access to any network based UPSes)?

Not specifically from me, but I think there are some people here that may be able to give you a better reason. I guess if no one speaks up it doesn't matter.

EDIT: Some way of resetting/clearing disks from the database when you delete a volume from the GUI. I've seen several cases where people can't reallocate disks after deleting a pool because the database still has them in use.

Do you have more details? Unfortunately this section of code has been rototilled so much that porting changes here will become a nightmare (especially with the django upgrades).

Have a look at this thread, there are several like it:

http://forums.freenas.org/showthrea...e-FreeNAS-forget-about-an-export-d-ZFS-volume

 

[Milhouse]

Hmmm.. is there a compelling argument for doing this

Not specifically from me, but I think there are some people here that may be able to give you a better reason. I guess if no one speaks up it doesn't matter.

Hopefully the bugs previously identified (which affect both local and network UPS support equally) will be in 8.0.3, but since full network support requires additional GUI changes I guess hacking network support into 8.0.x will have to suffice for now.

(in particular, I don't have access to any network based UPSes)?

Presumably you don't have access to a UPS at all then, as most UPS can become a network UPS - after all it's a function that FreeNAS 8.0.2 provides (operating as the UPS master), so if you had a second instance of FreeNAS running on the same network that would be your UPS slave.

 

[nogi]

This is fixed in the trunk so would it be safe to assume it will be included in 8.0.3? I just couldn't see it in your post - https://support.freenas.org/ticket/978

Also would be good to force a rsync job to start so that we can test.

 

[gcooper]

This is fixed in the trunk so would it be safe to assume it will be included in 8.0.3? I just couldn't see it in your post - https://support.freenas.org/ticket/978

Also would be good to force a rsync job to start so that we can test.

(moved release criterion items up to the first post)

If it falls outside the bounds of the set release criterion, we can push it out to 8.2 or 8.0.4; depending on how many early adopters there are for 8.2 -- people who are concerned more with stability should ask for a 8.0.4 release -- which is more of what the commercial customers will want and what I'll be encouraging until the first subminor release is available and all of the kinks have been worked out for the 8.2.0 release.

Your request falls into the third category... I'll see if it's a good candidate for the 8.0.3 release.

 

[gcooper]

And the word on high is that I have 1.5 weeks tops for the release.

I'm gonna also need you to come in Sunday too. We, uh, lost some people this week and we need to sorta catch up. Thanks.

 

[gcooper]

Crud -- 512MB isn't enough... Need at least 1GB (so far). ix-cache can run once at boot and only eat up 256MB, but unfortunately running wbinfo -u again eats up the remaining 500MB (or so) I was using (for some reason it wasn't setting up a 512MB disk.. hmm).

 

[logan54]

inadyn fix

Hello,

a fix of the ddns service would also be great for 8.0.3 too,
see, because it is broken since 8.0RC3
http://forums.freenas.org/showthread.php?3719-I-can-t-activate-DDNS&highlight=inadyn

regards

logan54

 

[gcooper]

Crud -- 512MB isn't enough... Need at least 1GB (so far). ix-cache can run once at boot and only eat up 256MB, but unfortunately running wbinfo -u again eats up the remaining 500MB (or so) I was using (for some reason it wasn't setting up a 512MB disk.. hmm).

Then again... 100k users is a lot:

# getent passwd | wc -l
100039

 

[survive]

Hi gcooper,

Any thoughts on increasing the image size?

Right now I think the image is 2000 megs....it's just a hair to big for my selection of 2BG SanDisk keys & CF cards, so I'm using 4GB flash.

I think it would be a good idea to expand the image to something like 3500MB, which should fit on even the cheesiest 4 Gig key I got in my last box of Cracker Jacks.

-Will

 

[gcooper]

a fix of the ddns service would also be great for 8.0.3 too,
see, because it is broken since 8.0RC3
http://forums.freenas.org/showthread.php?3719-I-can-t-activate-DDNS&highlight=inadyn

WTF. It's installing 32-bit binaries on amd64 (even on trunk)?!

# file /usr/local/bin/inadyn
/usr/local/bin/inadyn: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped

 

[gcooper]

Hi gcooper,

Any thoughts on increasing the image size?

Right now I think the image is 2000 megs....it's just a hair to big for my selection of 2BG SanDisk keys & CF cards, so I'm using 4GB flash.

I think it would be a good idea to expand the image to something like 3500MB, which should fit on even the cheesiest 4 Gig key I got in my last box of Cracker Jacks.

The problem with doing that is that I think we'd be going in the wrong direction... The goal is to shrink things down and put more stuff on permanent storage (like zpools) or SSDs. USB media and CF media are horrible performance wise, and the upcoming plugin work will put apps in /mnt/<volume>/.freenas/<etc> (or that's how things are going as far as I can see..).

 

[gcooper]

Your request falls into the third category... I'll see if it's a good candidate for the 8.0.3 release.

The change merged cleanly into my workspace (which is usually a good sign), but I'll test out the change in an image before committing it.

 

[gcooper]

There's an init script that's supposed to load it but isn't for some reason. I'm not sure if it loads it conditionally when you try to import NTFS or if it's supposed to load it on boot.

This isn't critical, so I think it's better we shoot for 8.2 here.

Not specifically from me, but I think there are some people here that may be able to give you a better reason. I guess if no one speaks up it doesn't matter.

I think this will have to drop off of 8.0.3 until maybe 8.0.4 or 8.2.*. Don't worry though.. we've had a large amount of interest from customers in this area, so once I get the hardware I'll go to town making sure it works :).

Have a look at this thread, there are several like it:

http://forums.freenas.org/showthrea...e-FreeNAS-forget-about-an-export-d-ZFS-volume

Ugh.. that problem. I never got to the bottom of that issue but it sort of "magically" went away with the volumes branch (unfortunately things were sort of regressed in new and interesting ways too after that was collapsed, but I think that the kinks have been worked out now). It's extremely annoying, but there are workarounds, so I think I'll leave it be (otherwise I'll be wasting my effort on that item when the fix has already been done on trunk).

I wish I could pick up your requests -- but you kind of picked hard ones to test in a 1.5 week timeframe :}..

 

[chris]

How about making the simple change of adding

options KVA_PAGES=512

to the kernel build on i386? This allows ZFS to function properly with > 1GB of kernel memory.

(Before you say "don't use i386" or "it won't work", I can say that I've been running solidly for several months with this change on i386 with 2GB).

 

[gcooper]

How about making the simple change of adding

options KVA_PAGES=512

to the kernel build on i386? This allows ZFS to function properly with > 1GB of kernel memory.

(Before you say "don't use i386" or "it won't work", I can say that I've been running solidly for several months with this change on i386 with 2GB).

Ok -- the caveat being then that we don't really support or encourage running i386. I'll depend on you and a handful of other folks to be my dedicated i386 testers :).

I'm also going to cap vfs.zfs.arc_max to 256MB on i386 and 1024MB on amd64 (untuned of course), unless anyone has any objections to me doing that.

Bottom line is that for home users this needs to be a robust solution. For people that want to pimp their NASes they can go about tuning their boxes to make them go faster :).

 

[gcooper]

Ok -- the caveat being then that we don't really support or encourage running i386. I'll depend on you and a handful of other folks to be my dedicated i386 testers :).

I'm also going to cap vfs.zfs.arc_max to 256MB on i386 and 1024MB on amd64 (untuned of course), unless anyone has any objections to me doing that.

Bottom line is that for home users this needs to be a robust solution. For people that want to pimp their NASes they can go about tuning their boxes to make them go faster :).

Adding to this statement: for now they'll be hardcoded limits, but I'll shoot for adding these to a migration script so the preset values can be changed at a later date from the GUI.