Unable to Mount Full Zpool

Status
Not open for further replies.
P

Plasticjesus690

Cadet
Joined
Sep 6, 2016
Messages
9
Hello Everyone.

I made an idiot mistake and allowed a volume to fill up. During this time I also lost flash drive holding freenas. I have since reinstalled freenas and gotten everything up and running except this volume. Attempting to import it through the GUI fails. Attempting to import it through SSH also fails saying it cannot create mount points. I have also dropped into single user mode and tried to import it. I revived a message saying it could be imported with the -f switch but this made me nervous. I did it anyways. I revived the same message about being unable to create mount points. I ran scrub of the volume and it shows as healthy and no data known data errors. Is there any way I can mount this thing to get my data moved elsewhere or even potentially save the volume?
 
Last edited:
DrKK

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
How do you scrub a volume that can't be mounted?

What's your hardware? Including motherboard make and model, capacities of the thumb drives, etc.
 
P

Plasticjesus690

Cadet
Joined
Sep 6, 2016
Messages
9
Hardware is Supermicro X8DTN+. HBA is IBM M1015. If I do a fresh install and attempt to mount the volume it goes through the process and says it fails to mount /mnt/HiveVault/Media and all other folders or w/e under it. If I attempt to reimport it again it says the zpool is already imported. If I go to single user mode and attempt to import I am presented with a message telling me the last time it was accessed etc and that I can mount the volume using the -f switch trying this gives me the same error message about being unable to create the mount points for /mnt/HiveVault/Media, mnt/HiveVault/Hivevault etc. From either Location I am still able to Do Zpool Scrub and status both show the pool as healthy. I started a new scrub as you can see below.

Code:
[root@Hivenas] /# zpool import HiveVault
cannot mount '/HiveVault/HiveVault': failed to create mountpoint
cannot mount '/HiveVault/Media': failed to create mountpoint
cannot mount '/HiveVault/jails': failed to create mountpoint


zpool import HiveVault
cannot import 'HiveVault': a pool with that name is already created/imported,
and no additional pools with that name were found



~# zpool status HiveVault
  pool: HiveVault
state: ONLINE
  scan: scrub in progress since Mon Sep  5 19:31:22 2016
        4.85T scanned out of 10.5T at 210M/s, 7h53m to go
        0 repaired, 46.00% done
config:

        NAME                                            STATE     READ WRITE CKSUM
        HiveVault                                       ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            gptid/f024a868-bfc0-11e4-8cf5-000c29befef1  ONLINE       0     0     0
            gptid/f08d5072-bfc0-11e4-8cf5-000c29befef1  ONLINE       0     0     0
            gptid/f0f4e05a-bfc0-11e4-8cf5-000c29befef1  ONLINE       0     0     0
            gptid/f15f67f9-bfc0-11e4-8cf5-000c29befef1  ONLINE       0     0     0
 
Last edited:
Mirfster

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
Plasticjesus690 said:
~# zpool status HiveVault
pool: HiveVault
state: ONLINE
scan: scrub in progress since Mon Sep 5 19:31:22 2016
4.85T scanned out of 10.5T at 210M/s, 7h53m to go
0 repaired, 46.00% done
Click to expand...
Looks like it is in the middle of a scrub, so would be best to wait until that is done...

I am guess that since zpool status does show it as "ONLINE", then you may need to delete some stuff off to free up space? *** Not sure because I never allowed my Pool/Volume to reach "Critical Mass".

From the Shell (or better over SSH via Putty), what is the output of ls /mnt/HiveVault (In Code Tags please).
 
P

Plasticjesus690

Cadet
Joined
Sep 6, 2016
Messages
9
Mirfster said:
Looks like it is in the middle of a scrub, so would be best to wait until that is done...

I am guess that since zpool status does show it as "ONLINE", then you may need to delete some stuff off to free up space? *** Not sure because I never allowed my Pool/Volume to reach "Critical Mass".

From the Shell (or better over SSH via Putty), what is the output of ls /mnt/HiveVault (In Code Tags please).
Click to expand...
I know what an idiot mistake filling the drive up now was. If I was able to navigate the drive I could easily free up the space or at the very least. I am hoping to be able to access the data and move it to a different array.When I attempt to ls /mnt/HiveVault

Code:

[root@Hivenas] ~# ls /mnt/HiveVault
./  ../
[root@Hivenas] ~#
 
Last edited:
R

rs225

Guru
Joined
Jun 28, 2014
Messages
878
maybe there is a snapshot you can delete to free some space. The usual strategy is to echo over a file, but if you can't mount anything, that wouldn't be possible.

Try 'zfs list -r -t snapshot HiveVault'
 
Mirfster

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
Can you see if you get anything for zpool history HiveVault?
 
danb35

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,504
I think @Robert Trevellyan is on the right track--investigate mounting your datasets at an alternate mountpoint. Try this:
mkdir /mnt/dataset1
zfs mount -o mountpoint=/mnt/dataset1 HiveVault/HiveVault
ls /mnt/dataset1

This should mount your HiveVault/HiveVault dataset in an alternate location (i.e., one that doesn't require creating a directory inside your already-full pool). If you see stuff when you do the ls, you're good to go. Now you need to free up some space, and due to the copy-on-write nature of ZFS, you can't do that by just deleting files with rm. Instead, you'll need to truncate them like this: echo "" > bigfile.mp4
 
Mirfster

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
While I have never been in the situation, I noticed that on Oracle Blog they mentioned having to use "truncate" instead?

Keep in mind this is several years old, however I did see "truncate" in FreeNAS 9.10.1...
If your ZFS filesystem is completely full, it can be difficult to free up space. Most people's first impulse is to delete files, but that often fails because ZFS can require free space to record the deletion. A colleague of mine ran into this a few weeks ago. I advised him to try truncating a large file using shell redirection (e.g., "cat /dev/null > my_large_file"), but that didn't work. He was able to free up space using the truncate command, which is available in Solaris 11.

truncate -s 0 my_large_file
Click to expand...

/Edit: Thinking about it, its basically the same thing as mentioned by the others.
 
P

Plasticjesus690

Cadet
Joined
Sep 6, 2016
Messages
9
Mirfster said:
Can you see if you get anything for zpool history HiveVault?
Click to expand...

Here is the result of History.
Code:

History for 'HiveVault':
2015-02-28.19:13:34 zpool create -o cachefile=/data/zfs/zpool.cache -o failmode=continue -o autoexpand=on -O compression=lz4 -O aclmode=passthrough -O aclinherit=passthrough -f -m /BoobTube -o altroot=/mnt BoobTube raidz /dev/gptid/f024a868-bfc0-11e4-8cf5-000c29befef1 /dev/gptid/f08d5072-bfc0-11e4-8cf5-000c29befef1 /dev/gptid/f0f4e05a-bfc0-11e4-8cf5-000c29befef1 /dev/gptid/f15f67f9-bfc0-11e4-8cf5-000c29befef1
2015-02-28.19:13:39 zfs inherit mountpoint BoobTube
2015-02-28.19:13:39 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-02-28.19:13:56 zfs create -o mountpoint=legacy BoobTube/.system
2015-02-28.19:13:56 zfs create -o mountpoint=legacy BoobTube/.system/cores
2015-02-28.19:13:57 zfs create -o mountpoint=legacy BoobTube/.system/samba4
2015-02-28.19:13:57 zfs create -o mountpoint=legacy BoobTube/.system/syslog-097c772a7a1e40b5a1773cbed6329f54
2015-02-28.19:14:02 zfs create -o mountpoint=legacy BoobTube/.system/rrd-097c772a7a1e40b5a1773cbed6329f54
2015-02-28.19:15:48 zfs create -o casesensitivity=sensitive BoobTube/Media
2015-02-28.23:26:08 zpool import -f -R /mnt 342812271754280145
2015-02-28.23:26:10 zfs inherit -r mountpoint BoobTube
2015-02-28.23:26:10 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-02-28.23:26:10 zfs set aclmode=passthrough BoobTube
2015-02-28.23:26:15 zfs set aclinherit=passthrough BoobTube
2015-02-28.23:26:45 zfs set mountpoint=legacy BoobTube/.system
2015-03-01.00:03:44 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.00:03:44 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.00:54:50 zpool import -f -R /mnt 342812271754280145
2015-03-01.00:54:51 zfs inherit -r mountpoint BoobTube
2015-03-01.00:54:51 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.00:54:52 zfs set aclmode=passthrough BoobTube
2015-03-01.00:54:57 zfs set aclinherit=passthrough BoobTube
2015-03-01.00:55:23 zfs set mountpoint=legacy BoobTube/.system
2015-03-01.01:17:10 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.01:17:10 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.01:24:18 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.01:24:18 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.01:39:45 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.01:39:45 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.01:58:38 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.01:58:38 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.02:44:42 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.02:44:42 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.02:52:10 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.02:52:10 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.03:35:33 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.03:35:33 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.04:21:59 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.04:21:59 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-01.06:53:56 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-01.06:53:56 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-02.14:07:54 zfs create -o casesensitivity=sensitive BoobTube/HiveVault
2015-03-02.14:11:25 zfs destroy -r BoobTube/HiveVault
2015-03-02.14:41:11 zpool export -f BoobTube
2015-03-02.14:42:59 zpool import -f -R /mnt 342812271754280145
2015-03-02.14:43:01 zfs inherit -r mountpoint BoobTube
2015-03-02.14:43:01 zpool set cachefile=/data/zfs/zpool.cache BoobTube
2015-03-02.14:43:01 zfs set aclmode=passthrough BoobTube
2015-03-02.14:43:06 zfs set aclinherit=passthrough BoobTube
2015-03-02.14:43:27 zpool export BoobTube
2015-03-02.14:43:39 zpool import BoobTube HiveVault
2015-03-02.15:00:13 zpool export HiveVault
2015-03-02.15:00:32 zpool import -f -R /mnt 342812271754280145
2015-03-02.15:00:34 zfs inherit -r mountpoint HiveVault
2015-03-02.15:00:34 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-03-02.15:00:34 zfs set aclmode=passthrough HiveVault
2015-03-02.15:00:39 zfs set aclinherit=passthrough HiveVault
2015-03-02.15:01:17 zfs set mountpoint=legacy HiveVault/.system
2015-03-02.15:42:43 zfs create -o casesensitivity=sensitive HiveVault/HiveVault
2015-03-02.22:17:36 zpool scrub HiveVault
2015-03-11.09:27:56 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-11.09:27:56 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-03-11.09:51:13 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-11.09:51:13 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-03-11.10:05:32 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-11.10:05:32 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-03-14.09:26:35 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-14.09:26:35 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-03-18.09:14:40 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-03-18.09:14:40 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-04-12.00:00:14 zpool scrub HiveVault
2015-04-12.16:38:01 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-04-12.16:38:01 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-04-12.19:22:10 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-04-12.19:22:10 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-04-23.19:22:34 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-04-23.19:22:34 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-05-15.10:37:45 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-05-15.10:37:45 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-05-16.04:54:37 zfs create HiveVault/jails
2015-05-24.00:00:15 zpool scrub HiveVault
2015-05-24.21:37:53 zpool scrub HiveVault
2015-05-25.00:32:59 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-05-25.00:32:59 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-05-25.04:35:30 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-05-25.04:35:30 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-05-25.05:19:19 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-05-25.05:19:19 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-05-25.05:26:51 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-05-25.05:26:51 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-06-06.08:57:50 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-06-06.08:57:50 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-06-06.08:58:24 zfs create -o mountpoint=legacy HiveVault/.system/configs-097c772a7a1e40b5a1773cbed6329f54
2015-06-19.07:51:58 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-06-19.07:51:58 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-06-22.11:27:18 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-06-22.11:27:18 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-06-22.17:34:02 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-06-22.17:34:02 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-06-29.13:07:26 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-06-29.13:07:26 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-07-05.00:00:13 zpool scrub HiveVault
2015-07-13.22:00:41 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-07-13.22:00:41 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-07-13.23:45:44 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-07-13.23:45:44 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-07-21.00:06:37 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-07-21.00:06:37 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-08-16.00:00:13 zpool scrub HiveVault
2015-08-27.07:08:36 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-08-27.07:08:36 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-08-30.16:07:12 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-08-30.16:07:12 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-09-03.07:44:30 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-09-03.07:44:30 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-09-14.02:00:52 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-09-14.02:00:52 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-09-27.00:00:12 zpool scrub HiveVault
2015-10-01.09:24:12 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-10-01.09:24:12 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-10-01.09:47:42 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-10-01.09:47:42 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-10-02.07:36:57 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-10-02.07:36:57 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-10-02.08:14:02 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-10-02.08:14:02 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-10-04.21:47:23 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-10-04.21:47:23 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-10-30.03:17:28 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-10-30.03:17:28 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-11-08.00:00:15 zpool scrub HiveVault
2015-11-08.05:13:41 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-11-08.05:13:41 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-11-12.07:20:18 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-11-12.07:20:18 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-11-13.08:23:19 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-11-13.08:23:19 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-11-15.11:30:04 zpool scrub HiveVault
2015-11-17.21:48:52 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-11-17.21:48:52 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-11-17.19:41:45 zpool scrub HiveVault
2015-11-28.21:37:22 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-11-28.21:37:22 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-11-28.21:42:11 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-11-28.21:42:11 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-11-29.10:19:27 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-11-29.10:19:27 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-11-30.11:58:51 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-11-30.11:58:51 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-11-30.12:11:59 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-11-30.12:11:59 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-12-01.01:42:32 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-12-01.01:42:32 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-12-01.02:00:49 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-12-01.02:00:49 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-12-01.04:20:43 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-12-01.04:20:43 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-12-01.04:28:56 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-12-01.04:28:56 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-12-01.04:45:13 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-12-01.04:45:13 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-12-10.10:05:07 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2015-12-10.10:05:07 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2015-12-27.00:00:12 zpool scrub HiveVault
2016-01-01.18:59:09 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-01-01.18:59:09 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-01-19.12:45:35 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-01-19.12:45:35 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-01-30.17:14:32 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-01-30.17:14:32 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-04.17:30:52 zpool scrub HiveVault
2016-02-05.20:59:16 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-05.20:59:16 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-05.21:01:05 zfs inherit -r freenas:state HiveVault
2016-02-06.21:09:30 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-06.21:09:30 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-06.21:26:08 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-06.21:26:08 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-09.04:05:00 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-09.04:05:00 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-09.03:32:04 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-09.03:32:04 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-09.23:50:33 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-09.23:50:33 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-10.00:07:40 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-10.00:07:40 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-10.00:27:33 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-10.00:27:33 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-10.00:46:56 zpool scrub HiveVault
2016-02-12.21:46:27 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-12.21:46:27 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-12.21:58:02 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-12.21:58:02 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-15.11:23:02 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-15.11:23:02 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-02-20.20:25:51 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-02-20.20:25:51 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-03-01.15:19:38 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-03-01.15:19:38 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-03-14.03:09:10 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-03-14.03:09:10 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-03-17.21:50:56 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-03-17.21:50:56 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-03-20.00:00:22 zpool scrub HiveVault
2016-04-06.06:51:50 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-04-06.06:51:50 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-04-06.13:38:09 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-04-06.13:38:09 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-04-07.08:21:24 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-04-07.08:21:24 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-04-16.22:58:14 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-04-16.22:58:14 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-04-17.22:07:17 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-04-17.22:07:17 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-04-17.22:32:29 zpool import -o cachefile=none -R /mnt -f 342812271754280145
2016-04-17.22:32:29 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-04-17.22:36:06 zpool scrub HiveVault
2016-04-27.03:47:53 zpool import -o cachefile=none -R /mnt -f 342812271754280145
2016-04-27.03:47:53 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-05-01.02:53:33 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-05-01.02:53:33 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-05-21.14:38:17 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-05-21.14:38:17 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-05-29.00:00:25 zpool scrub HiveVault
2016-05-29.15:01:46 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-05-29.15:01:46 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-06-28.20:22:05 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-06-28.20:22:05 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-07-10.00:00:42 zpool scrub HiveVault
2016-08-15.23:44:53 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-08-15.23:44:53 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-08-21.00:00:39 zpool scrub HiveVault
2016-09-01.19:23:39 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-09-01.19:23:39 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-09-05.17:17:55 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-09-05.17:17:55 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-09-05.17:24:32 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 342812271754280145
2016-09-05.17:24:32 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-09-05.17:56:29 zpool import -o cachefile=none -R /mnt -f 342812271754280145
2016-09-05.17:56:29 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-09-05.18:31:24 zpool import -f -R /mnt 342812271754280145
2016-09-05.18:31:26 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-09-05.18:31:26 zfs set aclmode=passthrough HiveVault
2016-09-05.18:31:31 zfs set aclinherit=passthrough HiveVault
2016-09-05.19:31:29 zpool scrub HiveVault
2016-09-05.19:50:53 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-09-05.19:51:25 zfs set mountpoint=legacy HiveVault/.system
2016-09-05.19:54:11 zfs inherit compression HiveVault/HiveVault
2016-09-05.19:54:14 zfs inherit atime HiveVault/HiveVault
2016-09-05.19:54:16 zfs inherit dedup HiveVault/HiveVault
2016-09-05.19:54:19 zfs set reservation=0 HiveVault/HiveVault
2016-09-05.19:54:21 zfs set refreservation=0 HiveVault/HiveVault
2016-09-05.19:54:24 zfs set quota=none HiveVault/HiveVault
2016-09-05.19:54:26 zfs set refquota=none HiveVault/HiveVault
2016-09-05.19:54:33 zfs set aclmode=passthrough HiveVault/HiveVault
2016-09-05.19:54:51 zfs set compression=lz4 HiveVault
2016-09-05.19:54:53 zfs inherit atime HiveVault
2016-09-05.19:54:56 zfs inherit dedup HiveVault
2016-09-05.19:54:58 zfs set reservation=0 HiveVault
2016-09-05.19:55:02 zfs set refreservation=0 HiveVault
2016-09-05.19:55:05 zfs set quota=none HiveVault
2016-09-05.19:55:07 zfs set refquota=none HiveVault
2016-09-05.19:55:10 zfs set aclmode=passthrough HiveVault
2016-09-05.20:16:09 zpool set cachefile=/data/zfs/zpool.cache HiveVault
2016-09-06.06:35:41 zpool export HiveVault
2016-09-06.06:49:26 zpool export HiveVault
2016-09-06.09:10:06 zpool export HiveVault
2016-09-06.14:31:29 zpool export HiveVault


Robert Trevellyan said:
What does zfs mount say?
Click to expand...
Here is what zfs mount says
Code:
[root@Hivenas] ~# zfs mount
freenas-boot/ROOT/default       /
freenas-boot/grub               /boot/grub
Hivesung                        /mnt/Hivesung
Hivesung/Media5                 /mnt/Hivesung/Media5
HGST                            /mnt/HGST
HGST/Media4                     /mnt/HGST/Media4
BoobTube                        /mnt/BoobTube
BoobTube/Media2                 /mnt/BoobTube/Media2
Hivesung/.system                /var/db/system
Hivesung/.system/cores          /var/db/system/cores
Hivesung/.system/samba4         /var/db/system/samba4
Hivesung/.system/syslog-7f4d67ae16c94917b949456bb9f364ad  /var/db/system/syslog-7f4d67ae16c94917b949456bb9f364ad
Hivesung/.system/rrd-7f4d67ae16c94917b949456bb9f364ad  /var/db/system/rrd-7f4d67ae16c94917b949456bb9f364ad
Hivesung/.system/configs-7f4d67ae16c94917b949456bb9f364ad  /var/db/system/configs-7f4d67ae16c94917b949456bb9f364ad
HiveVault                       /mnt/HiveVault


danb35 said:
I think @Robert Trevellyan is on the right track--investigate mounting your datasets at an alternate mountpoint. Try this:
mkdir /mnt/dataset1
zfs mount -o mountpoint=/mnt/dataset1 HiveVault/HiveVault
ls /mnt/dataset1

This should mount your HiveVault/HiveVault dataset in an alternate location (i.e., one that doesn't require creating a directory inside your already-full pool). If you see stuff when you do the ls, you're good to go. Now you need to free up some space, and due to the copy-on-write nature of ZFS, you can't do that by just deleting files with rm. Instead, you'll need to truncate them like this: echo "" > bigfile.mp4
Click to expand...

Here are the results of this. Sadly It doesn't work :(. Once I see files I will be able to resolve the freespace issue. What if I were to replace all the drives with bigger ones and re silver one at at time? Since I seem to atleast have this level of interaction with the pool. I would rather not lose this data and am willing to try almost anything to recover it.

Code:

[root@Hivenas] ~# mkdir /mnt/dataset1
[root@Hivenas] ~# zfs mount -o mountpoint=/mnt/dataset1 HiveVault/HiveVault
cannot mount '/mnt/HiveVault/HiveVault': failed to create mountpoint
[root@Hivenas] ~#
 
Last edited:
R

rs225

Guru
Joined
Jun 28, 2014
Messages
878
In addition to the resilvers, you can also add a new disk to the pool if you plan to recreate the pool later, or add a mirror to the pool if you are going to operate it for a while.
 
R

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Plasticjesus690 said:
HiveVault /mnt/HiveVault
Click to expand...
It appears the pool is mounted in the expected location. My guess is there's an empty directory already there, hiding it.

I suggest trying:
  1. GUI Detach or CLI export the pool.
  2. rmdir /mnt/HiveVault
  3. CLI import the pool read-only.
  4. Recover the data you need via rsync, scp or ZFS replication.
Step 2 will only work if my guess is correct.
 
P

Plasticjesus690

Cadet
Joined
Sep 6, 2016
Messages
9
Has anyone here had any success with any Data recovery programs?
Robert Trevellyan said:
It appears the pool is mounted in the expected location. My guess is there's an empty directory already there, hiding it.

I suggest trying:
  1. GUI Detach or CLI export the pool.
  2. rmdir /mnt/HiveVault
  3. CLI import the pool read-only.
  4. Recover the data you need via rsync, scp or ZFS replication.
Step 2 will only work if my guess is correct.
Click to expand...
Sadly this did not work
Code:
Welcome to FreeNAS
[root@Hivenas] ~# zpool export HiveVault
[root@Hivenas] ~# rmdir /mnt/HiveVault
rmdir: /mnt/HiveVault: No such file or directory
[root@Hivenas] ~# mkdir /mnt/HiveVault
[root@Hivenas] ~# zpool import -R /mnt HiveVault
cannot mount '/mnt/HiveVault/HiveVault': failed to create mountpoint
cannot mount '/mnt/HiveVault/Media': failed to create mountpoint
cannot mount '/mnt/HiveVault/jails': failed to create mountpoint
[root@Hivenas] ~# ls /mnt/HiveVault
./  ../
[root@Hivenas] ~# zpool export HiveVault
[root@Hivenas] ~# ls /mnt/HiveVault
ls: /mnt/HiveVault: No such file or directory
[root@Hivenas] ~# mkdir /mnt/HiveVault
[root@Hivenas] ~# zpool import -o readonly=on -R /mnt HiveVault
cannot mount '/mnt/HiveVault/HiveVault': failed to create mountpoint
cannot mount '/mnt/HiveVault/Media': failed to create mountpoint
cannot mount '/mnt/HiveVault/jails': failed to create mountpoint
[root@Hivenas] ~# zpool status HiveVault
  pool: HiveVault
 state: ONLINE
  scan: scrub repaired 0 in 16h52m with 0 errors on Tue Sep  6 12:24:16 2016
config:

        NAME        STATE     READ WRITE CKSUM
        HiveVault   ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            da2p2   ONLINE       0     0     0
            da5p2   ONLINE       0     0     0
            da6p2   ONLINE       0     0     0
            da1p2   ONLINE       0     0     0

errors: No known data errors
[root@Hivenas] ~#

[root@Hivenas] ~# zpool export HiveVault
[root@Hivenas] ~# mkdir /mnt/HiveVault
[root@Hivenas] ~# zpool import -R /mnt HiveVault
cannot mount '/mnt/HiveVault/HiveVault': failed to create mountpoint
cannot mount '/mnt/HiveVault/Media': failed to create mountpoint
cannot mount '/mnt/HiveVault/jails': failed to create mountpoint
[root@Hivenas] ~# ls /mnt/HiveVault
./  ../
[root@Hivenas] ~# zpool export HiveVault
[root@Hivenas] ~# ls /mnt/HiveVault
ls: /mnt/HiveVault: No such file or directory
[root@Hivenas] ~# mkdir /mnt/HiveVault
[root@Hivenas] ~# zpool import -o readonly=on -R /mnt HiveVault
cannot mount '/mnt/HiveVault/HiveVault': failed to create mountpoint
cannot mount '/mnt/HiveVault/Media': failed to create mountpoint
cannot mount '/mnt/HiveVault/jails': failed to create mountpoint
[root@Hivenas] ~# zpool status HiveVault
  pool: HiveVault
state: ONLINE
  scan: scrub repaired 0 in 16h52m with 0 errors on Tue Sep  6 12:24:16 2016
config:

        NAME        STATE     READ WRITE CKSUM
        HiveVault   ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            da2p2   ONLINE       0     0     0
            da5p2   ONLINE       0     0     0
            da6p2   ONLINE       0     0     0
            da1p2   ONLINE       0     0     0

errors: No known data errors
[root@Hivenas] ~#
 
Last edited:
P

Plasticjesus690

Cadet
Joined
Sep 6, 2016
Messages
9
rs225 said:
In addition to the resilvers, you can also add a new disk to the pool if you plan to recreate the pool later, or add a mirror to the pool if you are going to operate it for a while.
Click to expand...
Will this actually work even though it cannot mount the drive. I cannot really interact with the zpool via GUI. Can I just use the documentation Here or do I need to format the drives first etc.
 
Last edited:
Mirfster

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
Plasticjesus690 said:
Will this actually work even though it cannot mount the drive. I cannot really interact with the zpool via GUI. Can I just use the Zpool replace command or will I need to format drives before hand etc.
Click to expand...
Think @rs225 was more along the lines of adding a vDev (whether it is a single drive vDev or a Mirror vDev) to the Volume in hopes of getting it more space.

That may work and allow you to get to your data, the downside is that you would then want to copy that off somewhere; then blow-away and recreate the Volume and copy back what you really need so the Volume is not left in a vulnerable state.

Got a friend with a FreeNAS System or something with a lot of spare drive space? ;)
 
P

Plasticjesus690

Cadet
Joined
Sep 6, 2016
Messages
9
Mirfster said:
Think @rs225 was more along the lines of adding a vDev (whether it is a single drive vDev or a Mirror vDev) to the Volume in hopes of getting it more space.

That may work and allow you to get to your data, the downside is that you would then want to copy that off somewhere; then blow-away and recreate the Volume and copy back what you really need so the Volume is not left in a vulnerable state.

Got a friend with a FreeNAS System or something with a lot of spare drive space? ;)
Click to expand...
I figured that is what he meant. I was at first considering just replacing the 4 drives one at a time but I like this idea better. I can come up with a few extra drives to add to the pool. I do not really care about saving the pool at this point. I just need the data off of it. So if I add 4x 1tb drives in a raid z config using something like
Code:
zpool add HiveVault raidz drive1  2 3 4


Would this automatically grow the size of the pool allowing me to mount it again?
 
Status
Not open for further replies.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "Unable to Mount Full Zpool"

Similar threads

S
  • Locked
Unable to mount ZFS pool - no space left on pool.
Replies
9
Views
3K
Sasa Petrovic
S
S
  • Locked
zpool import not able to mount
Replies
14
Views
7K
cyberjock
C
A
  • Locked
ZPool Export/Import Encryption with Passphrase Fails
Replies
5
Views
3K
cyberjock
C
C
  • Locked
Lost zpool after upgrading from 8.04 to 8.3
Replies
1
Views
2K
Yell
Yell
rakesh
  • Locked
Unable to mount UFS volume from FreeNAS 9.2.0
Replies
3
Views
3K
cyberjock
C
Top