ZFS Import not showing data

[Alister]

I was doing an upgrade on 9.3 and lost my CIFS service - I think I may have messed with permissions in the past

I created a new clean freenas boot usb and imported the volume. Unfortunately the data is there but I can't get access to it. It's like not in a dataset or the dataset has no name? I have/had files in /mnt/big/<dir1>, /mnt/big/<dir2> etc.



[root@freenas] ~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
big 6.39T 646G 6.39T /mnt/big <<<<<<<
big/.system 57.3M 646G 12.1M legacy
big/.system/cores 35.8M 646G 35.8M legacy
big/.system/rrd-3a1f56673d9d4184b1e2a410be8304bd 288K 646G 288K legacy
big/.system/rrd-537fab4175a746ec895a2b741db9099e 288K 646G 288K legacy
big/.system/samba4 4.25M 646G 4.25M legacy
big/.system/syslog-3a1f56673d9d4184b1e2a410be8304bd 3.36M 646G 3.36M legacy
big/.system/syslog-537fab4175a746ec895a2b741db9099e 1.20M 646G 1.20M legacy
big/huge 288K 646G 288K /mnt/big/huge


[root@freenas] ~# zpool status -v big
pool: big
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub canceled on Tue Feb 17 07:56:07 2015
config:

NAME STATE READ WRITE CKSUM
big ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
gptid/78715744-a0a2-11e4-9863-6805ca226379 ONLINE 0 0 0
gptid/f9a1af72-3d9f-11e4-938e-6805ca226379 ONLINE 0 0 0
gptid/fbbe13c6-3d9f-11e4-938e-6805ca226379 ONLINE 0 0 0
gptid/7bd8521f-8a9f-11e4-a34a-6805ca226379 ONLINE 0 0 0
gptid/09f9236c-3da0-11e4-938e-6805ca226379 ONLINE 0 0 0
gptid/0bdec32f-3da0-11e4-938e-6805ca226379 ONLINE 0 0 0

errors: No known data errors

zdb -h big

2015-02-14.11:51:54 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 4689969378295137911
2015-02-14.11:51:54 zpool set cachefile=/data/zfs/zpool.cache big
2015-02-15.02:42:44 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 4689969378295137911
2015-02-15.02:42:44 zpool set cachefile=/data/zfs/zpool.cache big
2015-02-17.04:22:11 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 4689969378295137911
2015-02-17.04:22:11 zpool set cachefile=/data/zfs/zpool.cache big
2015-02-17.05:25:28 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 4689969378295137911
2015-02-17.05:25:28 zpool set cachefile=/data/zfs/zpool.cache big
2015-02-17.05:48:27 zfs create big/huge
2015-02-17.05:54:29 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 4689969378295137911
2015-02-17.05:54:29 zpool set cachefile=/data/zfs/zpool.cache big
2015-02-17.07:21:23 zpool import -f -R /mnt 4689969378295137911
2015-02-17.07:21:30 zfs inherit -r mountpoint big
2015-02-17.07:21:30 zpool set cachefile=/data/zfs/zpool.cache big
2015-02-17.07:21:31 zfs set aclmode=passthrough big
2015-02-17.07:21:36 zfs set aclinherit=passthrough big
2015-02-17.07:27:57 zpool import -f -R /mnt 4689969378295137911
2015-02-17.07:28:05 zfs inherit -r mountpoint big
2015-02-17.07:28:05 zpool set cachefile=/data/zfs/zpool.cache big
2015-02-17.07:28:06 zfs set aclmode=passthrough big
2015-02-17.07:28:11 zfs set aclinherit=passthrough big
2015-02-17.07:28:35 zfs set mountpoint=legacy big/.system
2015-02-17.07:28:36 zfs create -o mountpoint=legacy big/.system/syslog-537fab4175a746ec895a2b741db9099e
2015-02-17.07:28:42 zfs create -o mountpoint=legacy big/.system/rrd-537fab4175a746ec895a2b741db9099e
2015-02-17.07:45:36 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 4689969378295137911
2015-02-17.07:45:36 zpool set cachefile=/data/zfs/zpool.cache big
2015-02-17.07:45:41 zfs set aclmode=restricted big
2015-02-17.07:50:46 zpool scrub big
2015-02-17.07:56:12 zpool scrub -s big
2015-02-17.07:58:49 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 4689969378295137911
2015-02-17.07:58:49 zpool set cachefile=/data/zfs/zpool.cache big
2015-02-17.11:43:27 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 4689969378295137911
2015-02-17.11:43:27 zpool set cachefile=/data/zfs/zpool.cache big



[root@freenas] ~# ls -a /mnt/big
./ ../ .freenas/ .windows* huge/


if I use zdb I can see zfs plain file & zfs directories - see attached putty.txt

[root@freenas] ~# zpool get all big
NAME PROPERTY VALUE SOURCE
big size 10.9T -
big capacity 88% -
big altroot /mnt local
big health ONLINE -
big guid 4689969378295137911 default
big version - default
big bootfs - default
big delegation on default
big autoreplace off default
big cachefile /data/zfs/zpool.cache local
big failmode continue local
big listsnapshots off default
big autoexpand on local
big dedupditto 0 default
big dedupratio 1.00x -
big free 1.29T -
big allocated 9.59T -
big readonly off -
big comment - default
big expandsize - -
big freeing 0 default
big fragmentation 6% -
big leaked 0 default
big feature@async_destroy enabled local
big feature@empty_bpobj active local
big feature@lz4_compress active local
big feature@multi_vdev_crash_dump enabled local
big feature@spacemap_histogram active local
big feature@enabled_txg active local
big feature@hole_birth active local
big feature@extensible_dataset enabled local
big feature@embedded_data disabled local
big feature@bookmarks enabled local
big feature@filesystem_limits disabled local
big feature@large_blocks disabled local


What I need to recover is

Dataset big [ZPL], ID 21, cr_txg 1, 6.39T, 19394 objects

 

[Alister]

Ran a scrub overnight and it's given access to the data - not pretty, access rights all screwed up, but I can get the data off.