Top level directory disappeared during resilver

[rpaw]

I've been running this Freenas box for about 2 years without a hitch. It's running 11.1-u5 with 12 disks on raidz3.

The system was in the process of resilvering when I lost connection to one of the directory shares. I ssh'ed to the box and did an ls on the top level and didn't see that directory anymore.

I ran " zfs list -o space" and it looks like the space is still taken. The amount of free space hasn't changed since before the directory disappeared so the data is there somewhere.

Does anyone have any ideas on what might be going on and how I can recover the directory?

Code:

NAME																AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
freenas-boot														10.5G  3.19G		 0	 64K			  0	  3.19G
freenas-boot/ROOT												   10.5G  3.17G		 0	 29K			  0	  3.17G
freenas-boot/ROOT/11.1-U2										   10.5G   285K		 0	285K			  0		  0
freenas-boot/ROOT/11.1-U4										   10.5G   298K		 0	298K			  0		  0
freenas-boot/ROOT/11.1-U5										   10.5G  3.17G	 2.35G	838M			  0		  0
freenas-boot/ROOT/Initial-Install								   10.5G	 1K		 0	  1K			  0		  0
freenas-boot/ROOT/default										   10.5G   159K		 0	159K			  0		  0
freenas-boot/grub												   10.5G  7.02M		 0   7.02M			  0		  0
media															   14.7T  23.7T		 0   23.7T			  0	  5.14G
media/jailroot													  14.7T  4.68G		 0	488K			  0	  4.68G
media/jailroot/.warden-template-pluginjail						  14.7T   804M	  232K	804M			  0		  0
media/jailroot/.warden-template-pluginjail-11.0-x64				 14.7T   804M	  232K	804M			  0		  0
media/jailroot/.warden-template-pluginjail-11.0-x64-20180524144719  14.7T   804M	  232K	804M			  0		  0
media/jailroot/.warden-template-pluginjail-11.0-x64-20180524152858  14.7T   804M	  232K	804M			  0		  0
media/jailroot/couchpotato_1										14.7T   359M		 0	359M			  0		  0
media/jailroot/mylar_1											  14.7T   426M		 0	426M			  0		  0
media/jailroot/sabnzbd_1											14.7T   355M		 0	355M			  0		  0
media/jailroot/sickbeard_1										  14.7T   437M		 0	437M			  0		  0
system															   113G   680M		 0	 88K			  0	   679M
system/.system													   113G   658M		 0	510M			  0	   148M
system/.system/configs-baf20222858c41e9ac57760956a2756b			  113G  31.1M		 0   31.1M			  0		  0
system/.system/cores												 113G  56.9M		 0   56.9M			  0		  0
system/.system/rrd-baf20222858c41e9ac57760956a2756b				  113G  47.5M		 0   47.5M			  0		  0
system/.system/samba4												113G  1.65M		 0   1.65M			  0		  0
system/.system/syslog-baf20222858c41e9ac57760956a2756b			   113G  11.1M		 0   11.1M			  0		  0

The directory was under

Code:

media

. 23.7TB would include the directory which would be about 10TB of data.

Any help would be appreciated.

Thanks,

 

[Arwen]

One of the first steps is to see if the data / directory is seen by FreeNAS. Simply login, (like you did to get the zfs list. Then change to the base, problably cd /mnt/media and see what is there, ls -la. Continue poking around to see if the files you expect are still there. It's possible your share was dropped.

I tend to perform my FreeNAS data re-organization and verification from command line on the FreeNAS, (using SSH from a client like my desktop or laptop).

Please supply the output of the following commands;
zpool list -v media
zfs list -t all -r media

Last, your hardware and pool layout is helpful.

 

[rpaw]

I ssh'ed in and poked around and didn't see the files. The system rebooted and for some reason, the directory showed back up when the system came back.

I'm a bit nervous about not understanding why the directory disappeared and reappeared, but I can't find anything wrong.

 

[Arwen]

Glad it's back.

Other than it appears to have been un-mounted, I don't have any other reason / cause for the disappearance.

To check the health of the pool, you can use;
zpool status

 

[sretalla]

You could also look at zpool history <pool> for some clues about what might have changed.

 

[GEOptic]

I'm having a similar issue but with a zvol under the pool.
I will post the complete (stressing) situation :)