Upgrade 11.0 U4 to 11.1 lost gmirror volume

[wil]

Hi,

I have been using FreeNAS for several years now with great satisfaction, thanks for all the valuable work you do.
This is the first time I bump into something like this and as I made a mention about this mishap on twitter, somebody from FreeNAS suggested me to post about it here.

What happened? I ran the upgrade from 11.0 U4 to 11.1 and some of my NFS shares did not come back up. After some troubleshooting it turns out that the RAID1 (non ZFS, UFS) mirror that I had no longer shows up after boot.
This was a couple of weeks ago and I ran in some snags during troubleshooting so I rolled back to 11.0U4 and everything worked again. (yay!)

So today I deleted the new 11.1 boot partition, then verified the install using the "verify install" button and reran the upgrade. The same thing still happens.

So I looked around, apparently the mirror still exists, but FreeNAS is unaware about it.
In 11.1 I ran the following:

Code:

 # gmirror status
		Name	Status  Components
mirror/tuffy  COMPLETE  da4 (ACTIVE)
						da2 (ACTIVE)
mirror/swap0  COMPLETE  da7p1 (ACTIVE)
						da6p1 (ACTIVE)
mirror/swap1  COMPLETE  da5p1 (ACTIVE)
						da3p1 (ACTIVE)
mirror/swap2  COMPLETE  da1p1 (ACTIVE)
						da0p1 (ACTIVE)

mirror/tuffy is the missing volume and it is there, but not available.

Code:

# df -h
Filesystem												  Size	Used   Avail Capacity  Mounted on
freenas-boot/ROOT/11.1-RELEASE							  3.0G	1.7G	1.3G	57%	/
devfs													   1.0K	1.0K	  0B   100%	/dev
tmpfs														32M	 10M	 22M	33%	/etc
tmpfs													   4.0M	8.0K	4.0M	 0%	/mnt
tmpfs														11G	 23M	 11G	 0%	/var
freenas-boot/grub										   1.3G	 11M	1.3G	 1%	/boot/grub
fdescfs													 1.0K	1.0K	  0B   100%	/dev/fd
volumes													 3.0T	192K	3.0T	 0%	/mnt/volumes
volumes/bak												 800G	112G	688G	14%	/mnt/volumes/bak
volumes/dat												 400G	 17G	383G	 4%	/mnt/volumes/dat
volumes/jails											   3.0T	192K	3.0T	 0%	/mnt/volumes/jails
volumes/scripts											 1.0G	192K	1.0G	 0%	/mnt/volumes/scripts
volumes/.system											 3.0T	208K	3.0T	 0%	/var/db/system
volumes/.system/cores									   3.0T	791K	3.0T	 0%	/var/db/system/cores
volumes/.system/samba4									  3.0T	376K	3.0T	 0%	/var/db/system/samba4
volumes/.system/syslog-3832181b18e34de7b6f55927b62eb059	 3.0T	1.1M	3.0T	 0%	/var/db/system/syslog-3832181b18e34de7b6f55927b62eb059
volumes/.system/rrd-3832181b18e34de7b6f55927b62eb059		3.0T	192K	3.0T	 0%	/var/db/system/rrd-3832181b18e34de7b6f55927b62eb059
volumes/.system/configs-3832181b18e34de7b6f55927b62eb059	3.0T	 88M	3.0T	 0%	/var/db/system/configs-3832181b18e34de7b6f55927b62eb059
tmpfs													   600M	106M	494M	18%	/var/db/collectd/rrd

In 11.0U4 there used to also be a line there with "volumes/tuffy"

It now does not show up in /etc/fstab while in 11.0U4 it is in the /etc/fstab file.

Here's my /etc/fstab from 11.0U4:

Code:

# cat /etc/fstab
freenas-boot/grub	   /boot/grub	  zfs	 rw,noatime	  1	   0
fdescfs /dev/fd fdescfs rw	  0 0
/dev/ufs/tuffy  /mnt/tuffy	  ufs	 rw,nfsv4acls,late	   2	   2
/dev/da0p1.eli  none					swap			sw			  0	  0
/dev/da1p1.eli  none					swap			sw			  0	  0
/dev/da3p1.eli  none					swap			sw			  0	  0
/dev/da5p1.eli  none					swap			sw			  0	  0
/dev/da6p1.eli  none					swap			sw			  0	  0
/dev/da7p1.eli  none					swap			sw			  0	  0

whereas now in 11.1 it looks like this:

Code:

# cat /etc/fstab
freenas-boot/grub	   /boot/grub	  zfs	 rw,noatime	  1	   0
fdescfs /dev/fd fdescfs rw	  0 0
/dev/mirror/swap0.eli   none	swap	sw	  0	   0
/dev/mirror/swap1.eli   none	swap	sw	  0	   0
/dev/mirror/swap2.eli   none	swap	sw	  0	   0

As you see no entry for my UFS mirror anymore.
The /dev/ufs/tuffy node still exists though. I guess I could edit /etc/fstab, but that does not survive a reboot I think, so not sure what to do besides rolling back to 11.0U4 again.

Thanks for any tips,
--
Wil

 

[danb35]

FreeNAS hasn't supported UFS since the release of 9.3, over three years ago. What manual hacking have you been doing to make this work this far?

 

[wil]

Hi,

Thanks for your reply.
Ah, no hacking at all. Just installed it a long time ago and upgraded it over time. I seem to remember that I even installed the UFS mirror from the GUI.

Until now the UFS volume just survived any upgrade and that apparently has stopped, at least on 11.1. It is debatable if that is by design or unintended, that is not something for me to decide.
It certainly is a snag for me and am hoping for a fix, but understand if that isn't possible.

 

[danb35]

Just installed it a long time ago and upgraded it over time. I seem to remember that I even installed the UFS mirror from the GUI.

That's entirely possible; the GUI supported that through the 9.2 series. I'm surprised the system retained it this long, though--as I said above, with the release of 9.3 (8 Dec 14), support for UFS was dropped from the GUI, but apparently the backend systems continued to support it until 11.1. I'm afraid I can't give any further insight as to what may have changed, though. A bug report might result in a change, or perhaps at least further explanation of what's going on.

 

[wil]

OK, in that case it has been reported as issue #27521.