Can't remove snapshot

[nameBrandon]

I've installed 9.10 on a white box server, and I'm shortly moving to proper server-level hardware. Part of that move included migrating my single jail on an SSD to the main storage pool, as I wasn't going to be able to bring that consumer-level SSD over to the new enterprise-level server.

I followed the below directions to move my jail (jail contains PLEX, but via manual git install, not plugin-based). I've included the correct paths for my system in the instructions.

Code:

The pool you are transferring the jails from is SSD
The destination pool is mainarray
The jail root (Jails->Configuration) is /mnt/SSD
The new jail root will be /mnt/mainarray/jails

Turn off all plugins (Plugins->Installed)
Stop all jails (Jails->View Jails)

Run these commands via CLI

zfs snapshot -r SSD@relocate
zfs send -R SSD@relocate | zfs receive -v mainarray/jails
zfs get -rH -o name -s received mountpoint mainarray/jails | xargs -I {} sh -c "zfs set mountpoint=/{} {}; zfs mount {};"

Change the Jail Root to /mnt/mainarray/jails (Jails->Configuration)
Start jails/plugins
Check that everything works and destroy the original jails dataset (SSD/jails)

During step #2 (zfs send -R SSD@relocate | zfs receive -v mainarray/jails), my FreeNAS installation rebooted. I attempted to re-run the commands after rebooting, and then received an error about the new target location for the jail(s) already existing.

At that point I attempted the instructions again with a new jail name, but started noticing random reboots of the FreeNAS box. It appears that for some reason, whenever I stop the jail, the server then reboots within the next ~5-10 minutes. If I leave the jail running, all is well.

At this point I'm out of my depth here, and would like to get to a point where I can ideally migrate the jail to my main storage (mainarray) and be done with that SSD for good. I cannot seem to kill the remaining snapshot from the initial attempt to migrate the jail, nor can I seem to kill the target dataset on my main storage for the 'new' jail (mainarray/jails).

Below is some output showing the situation. Please let me know what further output I can provide that might assist in helping me figure this out.

Code:

ot@greenleo] ~# zfs list -r SSD
NAME                              USED  AVAIL  REFER  MOUNTPOINT
SSD                              10.5G  95.0G   780K  /mnt/SSD
SSD/.warden-template-pluginjail   469M  95.0G   469M  /mnt/SSD/.warden-template-pluginjail
SSD/.warden-template-standard    1.61G  95.0G  1.61G  /mnt/SSD/.warden-template-standard
SSD/plex                         8.47G  95.0G  9.86G  /mnt/SSD/plex
[root@greenleo] ~#

Code:

[root@greenleo] ~# zfs list -r mainarray
NAME                                                         USED  AVAIL  REFER  MOUNTPOINT
mainarray                                                   1.62T  16.5T   238K  /mnt/mainarray
mainarray/.system                                           4.73M  16.5T   402K  legacy
mainarray/.system/configs-0827a2da0c5b428287f6c8d2d52005ef   658K  16.5T   658K  legacy
mainarray/.system/cores                                     1.28M  16.5T  1.28M  legacy
mainarray/.system/rrd-0827a2da0c5b428287f6c8d2d52005ef       219K  16.5T   219K  legacy
mainarray/.system/samba4                                     759K  16.5T   759K  legacy
mainarray/.system/syslog-0827a2da0c5b428287f6c8d2d52005ef   1.46M  16.5T  1.46M  legacy
mainarray/Apps                                              7.28G  16.5T  7.28G  /mnt/mainarray/Apps
mainarray/Downloads                                         29.6G  16.5T  29.6G  /mnt/mainarray/Downloads
mainarray/Misc                                               219K  16.5T   219K  /mnt/mainarray/Misc
mainarray/Music                                              219K  16.5T   219K  /mnt/mainarray/Music
mainarray/PLEX                                              1.58T  16.5T  1.58T  /mnt/mainarray/PLEX
mainarray/jails                                             2.95G  16.5T   219K  /mnt/mainarray/jails
mainarray/jails/.warden-template-standard                   2.95G  16.5T  2.95G  /mnt/SSD/.warden-template-standard
[root@greenleo] ~#

Code:

[root@greenleo] ~# zfs list -t snapshot
NAME                                                       USED  AVAIL  REFER  MOUNTPOINT
SSD/.warden-template-standard@clean                         80K      -  1.61G  -
freenas-boot/ROOT/default@2016-07-10-12:54:10             2.88M      -   594M  -
freenas-boot/ROOT/default@2016-07-10-15:49:04              268K      -   594M  -
freenas-boot/grub@Pre-Upgrade-Wizard-2016-07-10_15:49:04     1K      -  6.33M  -
[root@greenleo] ~#

Trying to kill what was going to be the new jails directory on the main pool.

Code:

[root@greenleo] ~# zfs destroy -r mainarray/jails
cannot destroy 'mainarray/jails/.warden-template-pluginjail': dataset is busy

Trying to get rid of any SSD snapshots that might be causing problems..

Code:

[root@greenleo] ~# zfs destroy SSD/.warden-template-standard@clean
cannot destroy 'SSD/.warden-template-standard@clean': snapshot has dependent clones
use '-R' to destroy the following datasets:
SSD/plex


[root@greenleo] ~# zfs destroy -R SSD/.warden-template-standard@clean
cannot unmount '/mnt/SSD/plex': Device busy
[root@greenleo] ~#

I have not changed any configurations for the jail(s), the default path is still /mnt/SSD.

 

[fracai]

Maybe try setting the jail to not autostart. Once that takes, quit the jail. If the server stays running, delete your datasets. If the server reboots, the jail at least should not start up and you should now be able to delete the datasets.

If you still have issues with that dataset, just wipe the drive itself. You're not going to be using it in the new system anyway, right?

 

[nameBrandon]

Maybe try setting the jail to not autostart. Once that takes, quit the jail. If the server stays running, delete your datasets. If the server reboots, the jail at least should not start up and you should now be able to delete the datasets.

If you still have issues with that dataset, just wipe the drive itself. You're not going to be using it in the new system anyway, right?

Thanks for the reply!

By autostart.. do you mean disable it and reboot? I'll poke around and see if I can find the GUI part relating to startup tasks, I just don't recall seeing it.

I've rebooted a number of times, and still can't delete it, so perhaps that is the issue.

I can wipe the SSD / jail drive, but I would like to copy that jail if possible.. there's ~10GB of metadata on it that takes a while to re-create, along with some configurations I've made. It's not the end of the world, but it's do-able.

The mainarray is not going to be wiped, those are in a DAS and I'll just be moving them to the DL380 G6 i'm picking up, along with an LSI HBA, so those I need to keep in their volume/pool.

edit - saw the jail-specific setting for autostart.. disabled it and no difference..

cannot destroy 'mainarray/jails/.warden-template-standard': dataset is busy

again, this is not even the default path/volume for the single jail I'm running, that is is just SSD/ (/mnt/ssd/)
I have no idea why it would be locked or busy.. nothing (except the snapshot commands previously issued) references it.

 

[depasseg]

Freenas LOVES it when you do ZFS stuff behind the scenes!

Where is your jail storage configured to be stored (via the GUI)?

You can also just copy the plex data from one location to the other.

 

[nameBrandon]

Freenas LOVES it when you do ZFS stuff behind the scenes! :)

Where is your jail storage configured to be stored (via the GUI)?

You can also just copy the plex data from one location to the other.

Thanks for cheering me up.. :)

I did manage to zfs destroy -r /mainarray/jails.. somewhow it worked this time.

Going to try again, and NOT create the destination dataset ahead of time.. which wasn't in the instructions anyway, but somehow I felt like I needed to do that.. :\ ::smacks head on desk::

I may break down and just copy the plex data folder if this doesn't work! Thanks for that tip.

edit aaaan it rebooted again during the snapshot transfer / receive operations.. :( Arghh.. Once I clean up that new jails data set on mainarray I think I'll give up and just kill the SSD dataset and create a new plex data set on the redundant drives (mainarray)..

Code:

[root@greenleo] ~# zfs snapshot -r SSD@bh1
[root@greenleo] ~# zfs send -R SSD@bh1 | zfs receive -v mainarray/jails
receiving full stream of SSD@bh1 into mainarray/jails@bh1
received 1.09MB stream in 1 seconds (1.09MB/sec)
receiving full stream of SSD/.warden-template-pluginjail@bh1 into mainarray/jails/.warden-template-pluginjail@bh1
received 817MB stream in 3 seconds (272MB/sec)
receiving full stream of SSD/.warden-template-standard@clean into mainarray/jails/.warden-template-standard@clean
received 2.21GB stream in 9 seconds (252MB/sec)
receiving incremental stream of SSD/.warden-template-standard@bh1 into mainarray/jails/.warden-template-standard@bh1
received 312B stream in 1 seconds (312B/sec)
found clone origin mainarray/jails/.warden-template-standard@clean
receiving incremental stream of SSD/plex@bh1 into mainarray/jails/plex@bh1

**it rebooted at this point...**

SUPER DOUBLE EDIT - I gave up and killed the entire SSD pool/dataset/volume and the whole jail setup it contained and just started over. It works fine and I now I don't have to worry about migrating the config to the new DL 380 with a missing SSD drive.