Installing plugins fails at zfs clone of jails/.warden-template-pluginjail@clean snapshot

[Nicola Worthington]

I'm running FreeNAS-9.3-STABLE-201512121950 on a dual CPU Intel(R) Xeon(R) CPU E5-2603 v2 @ 1.80GHz machine with 81861MB of RAM. There is very low usage of this machine as it is a home media and storage server.

I have a mirrored SSD pool called "jails" for my jails to be installed on:

brienne# zpool status jails
pool: jails
state: ONLINE
scan: scrub repaired 0 in 0h11m with 0 errors on Sun Dec 13 00:11:21 2015
config:

NAME STATE READ WRITE CKSUM
jails ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/71ea795a-809a-11e5-884a-0cc47a546fcc ONLINE 0 0 0
gptid/7223d49f-809a-11e5-884a-0cc47a546fcc ONLINE 0 0 0

errors: No known data errors
brienne#

brienne# warden template list
Jail Templates:
------------------------------
VirtualBox-4.3.12 - FreeBSD (amd64)
pluginjail - FreeBSD (amd64)
standard - FreeBSD (amd64)
brienne#

brienne# zfs list -t snapshot | grep clean
jails/jails/.warden-template-VirtualBox-4.3.12@clean 80K - 661M -
jails/jails/.warden-template-pluginjail@clean 96K - 449M -
jails/jails/.warden-template-standard@clean 88K - 1.59G -
brienne#

brienne# ls /mnt/jails/jails/.warden-template-pluginjail
.cshrc COPYRIGHT dev libexec proc sbin usr
.plugins bin etc media rescue sys var
.profile boot lib mnt root tmp
brienne#
​

I have successfully installed plugins before, and am running a few jails. However today I tried installing the ownCloud and firefly plugins, but I get the following error when doing so:

Jan 7 18:03:18 brienne warden: Building new Jail... Please wait...
Jan 7 18:03:18 brienne warden: zfs clone jails/mnt/jails/jails/.warden-template-pluginjail@clean jails/jails/owncloud_1
Jan 7 18:03:18 brienne warden: ERROR: Failed creating clean ZFS base clone
Jan 7 18:03:18 brienne manage.py: [middleware.exceptions:37] [MiddlewareError: [Errno 2] No such file or directory: u'/mnt/jails/jails/owncloud_1/.plugins/PLUGIN']
​

I am not aware of having done anything strange or unusual on this machine recently. I have tried rebooting and trying to install the plugin through the web UI again, but I still get the same error.

To my untrained eye it looks like the GUI is executing the wrong zfs clone command, because jails/mnt/jails/jails/.warden-template-pluginjail@clean doesn't exist. Surely it means the following instead: jails/jails/.warden-template-pluginjail@clean ?

For example, if I execute the command from the log output above, manually on the command line:

brienne# zfs clone jails/mnt/jails/jails/.warden-template-pluginjail@clean jails/jails/owncloud_1
cannot open 'jails/mnt/jails/jails/.warden-template-pluginjail@clean': dataset does not exist
brienne#​

So if I test my theory by correcting this command, the zfs clone of the warden template works:

brienne# zfs clone jails/mnt/jails/jails/.warden-template-pluginjail@clean jails/jails/owncloud_1​

cannot open 'jails/mnt/jails/jails/.warden-template-pluginjail@clean': dataset does not exist
brienne# zfs clone jails/jails/.warden-template-pluginjail@clean jails/jails/owncloud_1
brienne# ls /mnt/jails/jails/owncloud_1/
.cshrc COPYRIGHT dev libexec proc sbin usr
.plugins bin etc media rescue sys var
.profile boot lib mnt root tmp
brienne#

The jail root is set to /mnt/jails/jails under the jails configuration page in the GUI.

Am I doing something silly? It feels to me like there has been a bug introduced in to the most recent update maybe?

Any help here would be greatly appreciated.

Thanks!

 

[depasseg]

It's like you copied and pasted my exact issue from last week. any chance this has been upgraded from 9.2.xx?

Long story short, create a snapshot of the warden dataset in question called clean and you should be all set. Exact command is in my post (don't have easy access ATM).

 

[Nicola Worthington]

It's like you copied and pasted my exact issue from last week. any chance this has been upgraded from 9.2.xx?

Long story short, create a snapshot of the warden dataset in question called clean and you should be all set. Exact command is in my post (don't have easy access ATM).

No, this was a fresh 9.3 install. I installed my existing jails and plugins a few weeks ago. This has only been an issue since I applied the most recent update a few days ago, ... so I'm suspecting something from that update is causing this, but I'm not sure. It may be something I've done, but I really don't know.

The snapshot already exists:

brienne# zfs list -t snapshot | grep jails/jails/.warden-template-pluginjail
jails/jails/.warden-template-pluginjail@clean 96K - 449M -
jails/jails/.warden-template-pluginjail@auto-20151202.1258-2m 88K - 449M -
jails/jails/.warden-template-pluginjail@auto-20151209.1258-2m 88K - 449M -
jails/jails/.warden-template-pluginjail@auto-20151216.1258-2m 0 - 449M -
jails/jails/.warden-template-pluginjail@auto-20151223.1258-2m 0 - 449M -
jails/jails/.warden-template-pluginjail@auto-20151230.1258-2m 88K - 449M -
jails/jails/.warden-template-pluginjail@auto-20160104.1109-1m 88K - 449M -
jails/jails/.warden-template-pluginjail@auto-20160105.1109-1m 0 - 449M -
jails/jails/.warden-template-pluginjail@auto-20160106.1109-1m 0 - 449M -
jails/jails/.warden-template-pluginjail@auto-20160107.1800-3h 0 - 449M -
jails/jails/.warden-template-pluginjail@auto-20160107.1800-1m 0 - 449M -
jails/jails/.warden-template-pluginjail@auto-20160107.1900-3h 0 - 449M -
brienne#
​

To my untrained eye it looks like the GUI is executing the wrong zfs clone command, because jails/mnt/jails/jails/.warden-template-pluginjail@clean doesn't exist. Surely it means the following instead: jails/jails/.warden-template-pluginjail@clean ?

For example, if I execute the command from the log output above, manually on the command line:

brienne# zfs clone jails/mnt/jails/jails/.warden-template-pluginjail@clean jails/jails/owncloud_1
cannot open 'jails/mnt/jails/jails/.warden-template-pluginjail@clean': dataset does not exist
brienne#​

So if I test my theory by correcting this command, the zfs clone of the warden template works:

brienne# zfs clone jails/jails/.warden-template-pluginjail@clean jails/jails/owncloud_1
brienne# ls /mnt/jails/jails/owncloud_1/
.cshrc COPYRIGHT dev libexec proc sbin usr .plugins bin etc media rescue sys var .profile boot lib mnt root tmp
brienne#
​

Could this possibly be a bug?

 

[dlavigne]

I think https://bugs.freenas.org/issues/6077 fixes it. The SU containing that fix is due out early next week.

 

[Nicola Worthington]

I think https://bugs.freenas.org/issues/6077 fixes it. The SU containing that fix is due out early next week.

Ah, yes that looks like it's very probably the same thing.

Thank you for your assistance. It's very appreciated! :)

I have disabled the recursive replication of my jails pool, and deleted the replicated archive/jails-backup/.warden-template-plugin dataset from my archive pool... doing so fixed the problem and I can now install the plugin okay.

I'll keep an eye out for the patch fix.