"Cannot destroy" while cleaning up iocage jail snapshots

[ctag]

Hi,

I'm trying to slim down my iocage jail storage on FreeNAS 11.3-U4.1. One jail in particular takes 240GB of space, despite only having ~12GB of files.

Code:

root@bns-normandy:~ # df -h
Filesystem                                  Size    Used   Avail Capacity  Mounted on
main-pool/iocage/jails/bns-normandy/root     18T     12G     18T     0%    /

I suspect the issue here is an abundance of snapshots. So I went to SSH and dug around in `zfs list -t snapshot` until I had an idea of what was going on. Right now this system is configured to keep auto-generated snapshots for both 1 and 6 months. But I'm finding snapshots for every dataset going back almost a year, so something seems off.

I found a snapshot before my latest freebsd jail release, and tried removing it and everything earlier:

Code:

# iocage stop ALL
# zfs unmount -f tank/iocage
# zfs destroy -r main-pool/iocage@%auto-20200617.0900-6m
...
would destroy main-pool/iocage/releases/11.2-RELEASE@auto-20200615.0900-6m
would destroy main-pool/iocage/releases/11.2-RELEASE@auto-20200616.0900-1m
would destroy main-pool/iocage/releases/11.2-RELEASE@auto-20200616.0900-6m
would destroy main-pool/iocage/releases/11.2-RELEASE@auto-20200617.0900-1m
would destroy main-pool/iocage/releases/11.2-RELEASE@auto-20200617.0900-6m
cannot destroy 'main-pool/iocage/releases/11.2-RELEASE/root@bns-normandy': snapshot has dependent clones
use '-R' to destroy the following datasets:
main-pool/iocage/jails/bns-normandy/root@auto-20200313.0900-6m
main-pool/iocage/jails/bns-normandy/root@auto-20200421.0900-6m
main-pool/iocage/jails/bns-normandy/root@auto-20200506.0900-6m
main-pool/iocage/jails/bns-normandy/root@auto-20200131.1129-1m
main-pool/iocage/jails/bns-normandy/root@auto-20200502.0900-6m
main-pool/iocage/jails/bns-normandy/root@auto-20200530.0900-1m
main-pool/iocage/jails/bns-normandy/root@auto-20200317.0900-6m
...

That.. Didn't seem to work. Apparently iocage uses cloning or something to create the jails, and it's locking me into having all of these snapshots?

If you can help me out of this bind at all, I'd really appreciate it!

 

[ctag]

I found this excellent post by Jurgen Sergaert that explains what's going on with the snapshots and cloning:

iocage jail type: base jail vs clone. Which to choose?

Trying to figure out when I would use which kind of jail (base vs clone). It all started when I was trying to clean up my snapshots and found one named for my one and only jail. I read the manual which explains each. It seems a base jail has a mount point of the base OS whereas clone is a...

www.ixsystems.com

That post also references this webpage which may help explain what's going on for me:

ZFS clones: Probably not what you really want – JRS Systems: the blog

The two jails that are taking up so much space are jails that I manually migrated from Warden by copying the filesystem files... So they're likely 100% divergent from the underlying release jail.

Here's the interesting line of output from `zfs list -t all -o name,origin,used,refer,mountpoint -S origin -r main-pool/iocage`:

Code:

NAME                                                                                      ORIGIN                                                       USED  REFER  MOUNTPOINT
main-pool/iocage/jails/bns-normandy/root                                                  main-pool/iocage/releases/11.2-RELEASE/root@bns-normandy     248G  11.7G  /mnt/main-pool/iocage/jails
/bns-normandy/root

I'm still a little confused, because from within the jail it claims to be 11.3 release:

root@bns-normandy:~ # freebsd-version
11.3-RELEASE-p10

 

[ctag]

I wound up removing all of the jail snapshots, which appeared to lessen the bloat.

Command used:

Code:

# iocage snaplist -H jailname | awk '{print $1;}' | while read line; do iocage snapremove -n $line jailname; done

iocage snapremove ALL not implemented · Issue #1098 · iocage/iocage

It seems the man page and the code disagree. iocage snapremove [-n | --name TEXT] UUID | NAME | ALL Then: snapremove Delete snapshots of the specified jail. If the keyword [ALL] is used, all snapsh...

github.com