Pool won't mount after reboot!

[mattlach]

Hey all.

I rebooted my FreeNAS box today to replace a drive.

I scared myself a little when I looked at the volume in the web interface and was greeted by the information:

Code:

Used: 0 (error)
Available: Error getting available space
Size: Error getting total space

I started my resilver anyway and then started investigating.

Zpool status indicates my data is still there, and my resilver is going well: (why is it resilver? I thought it was sliver? Anyway, moving right along)

Code:

# zpool status
  pool: RAIDz2-01
state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Mon May 26 17:31:27 2014
        447G scanned out of 2.79T at 397M/s, 1h43m to go
        54.6G resilvered, 15.64% done
config:
 
NAME                                              STATE    READ WRITE CKSUM
RAIDz2-01                                        DEGRADED    0    0    0
raidz2-0                                        DEGRADED    0    0    0
  da1p2                                        ONLINE      0    0    0
  da2p2                                        ONLINE      0    0    0
  da3p2                                        ONLINE      0    0    0
  da4p2                                        ONLINE      0    0    0
  da5p2                                        ONLINE      0    0    0
  da6p2                                        ONLINE      0    0    0
  da7p2                                        ONLINE      0    0    0
  replacing-7                                  OFFLINE      0    0    0
    2901464765091125052                        OFFLINE      0    0    0
    gptid/17289b87-e51d-11e3-9791-001517168acc  ONLINE      0    0    0  (resilvering)
 
errors: No known data errors

Then I went to look at my data at the mountpoint under /mnt only to find that it wasn't there.
Weird I thought, so I decided to try to mount it with "zfs mount -a" which gave me an error, as the file system is read only, and it tried to mount it under the default / location.
Anyway, does anyone know why it would have failed to mount on boot?
Could it be because my root partition has very little space free?

Code:

# df -h
Filesystem            Size    Used  Avail Capacity  Mounted on
/dev/ufs/FreeNASs1a    926M    829M    23M    97%    /
devfs                  1.0k    1.0k      0B  100%    /dev
/dev/md0              4.6M    3.3M    899k    79%    /etc
/dev/md1              823k    1.5k    756k    0%    /mnt
/dev/md2              149M    42M    94M    31%    /var
/dev/ufs/FreeNASs4      19M    3.8M    14M    21%    /data

For whatever reason the default partition size seems to be ~ 1 gig, even though I installed it on a 4 GB drive. Not quite sure why this is.
It seems like every time I upgrade, I get less and less free space left. Do upgrades leave stuff behind? Is there any way I can remove that stuff?
Also, I installed binutils and gcc a while back in order to compile a driver I needed. I don't need them anymore and wouldn't mind removing them, but ports won't let me because of dependencies:

Code:

# pkg_delete binutils-2.24
pkg_delete: package 'binutils-2.24' is required by these other packages
and may not be deinstalled:
gcc47-4.7.4.20131214
aria2-1.18.2
samba41-4.1.6
 
# pkg_delete gcc47-4.7.4.20131214
pkg_delete: package 'gcc47-4.7.4.20131214' is required by these other packages
and may not be deinstalled:
aria2-1.18.2
samba41-4.1.6

I'm not quite sure what aria is, but I am fairly certain samba was installed by default even before I installed binutils and gcc, so how can it be a dependency now???

Also, if it is space dependent, and I can't free up space, is there a way I can grow the partition?

Anyway, I'd appreciate any help in solving this issue!
Thanks,
Matt

 

[mattlach]

Hmm... So dmesg shows this as the error:

Code:

Mounting local file systems:.
cannot import '17890873096340179923': a pool with that name is already created/imported,
and no additional pools with that name were found
cannot import 'RAIDz2-01': a pool with that name is already created/imported,
and no additional pools with that name were found
cannot mount '/RAIDz2-01': failed to create mountpoint
cannot mount '/RAIDz2-01/.system': failed to create mountpoint
cannot mount '/RAIDz2-01/.system/cores': failed to create mountpoint
cannot mount '/RAIDz2-01/.system/samba4': failed to create mountpoint
cannot mount '/RAIDz2-01/.system/syslog': failed to create mountpoint

I don't know what '17890873096340179923' is, but the pool I need to mount is RAIDz2-01.
I'd really appreciate any help to get my pool to mount again!
Thanks,
Matt

 

[cyberjock]

Yeah.. you need some serious help.. that's for sure.

The fact that you installed anything on the FreeNAS install means you may very well be solely responsible for the problems you are having. FreeNAS needs the little bit of free space it has for it's own internal workings, and you may have broken them. If you read the manual it tells you not to install anything else you will piss off the FreeNAS gods.

If you want to do some paid support feel free to PM me. Otherwise all I can say is good luck.

Do keep in mind that you can break ZFS even worse for the CLI if you don't actually understand what the commands do on their own and whether you should or shouldn't use them for your particular problem. Do things you shouldn't do and you risk losing the pool in a permanent way.

Good luck!

 

[jkh]

I'm not going to be all aggro like cyberjock in my reply since, well, I don't have the time or the energy, but it does sound like your pool is busy resilvering and perhaps you should first let it finish, since you're doing a drive replacement. Then try to import the pool rather than mount it. You only have 1GB of memory on your 4GB drive because that's how FreeNAS rolls - it has a spare partition for upgrades and puts /data on yet another partition, so you're only seeing a small part of the entire picture. That's also a good reason not to install extra software on it. If you prefer, and I would in your shoes, you can just back up your configuration database and then do a fresh erase install of FreeNAS, then re-import your config database and then re-import your pool. That will start things off on the right foot again.

Good luck.

 

[mattlach]

I'm not going to be all aggro like cyberjock in my reply since, well, I don't have the time or the energy, but it does sound like your pool is busy resilvering and perhaps you should first let it finish, since you're doing a drive replacement. Then try to import the pool rather than mount it. You only have 1GB of memory on your 4GB drive because that's how FreeNAS rolls - it has a spare partition for upgrades and puts /data on yet another partition, so you're only seeing a small part of the entire picture. That's also a good reason not to install extra software on it. If you prefer, and I would in your shoes, you can just back up your configuration database and then do a fresh erase install of FreeNAS, then re-import your config database and then re-import your pool. That will start things off on the right foot again.

Good luck.

Appreciate it. Yeah, I was thinking this too. I've already spent more time troubleshooting this, than it would just take me to simply export the volume, do a fresh install and reimport it.

Thank you!