Moving Jails from spinners to NVMe

[rmccullough]

I bought a Supermicro AOC-SLG3-2M2 and two Western Digital 500GB Blue NVMe drives withe the intent to move my bhyve VMs and jails to this.

Spinners are in pool named Tank. NVMe's are in a pool named vm.

I installed the drives in the card, the card in my server, set bifurcation, and then created a new pool using the 2 drives in a mirrored configuration.

I was able to move my bhyve VM using a zfs send / receive between pools. This is running and doesn't seem to have any issues thus far.

Then, to move my jails, I roughly followed the instructions found here: [11.2] Migrating the iocage dataset to an other pool ?

My issue is roughly described in this post: https://www.truenas.com/community/t...ge-dataset-to-an-other-pool.70168/post-630995

But I will copy/paste here:
I didn't follow the instructions exactly, but I think I may have a slightly different setup.

I did not execute the 2 zfs destroy commands.

However, my iocage does not seem to reside at /mnt/Tank/iocage. Instead, mine seems to show up at /mnt/iocage.

Also, now I have iocage listed in both of my pools, and it almost appears as if they are somehow linked. I tried to run the zfs destroy -f Tank/iocage after completing everything and it was getting stuck saying:

Code:

root@freenas[/mnt]# zfs destroy -f Tank/iocage
cannot destroy 'Tank/iocage': filesystem has children
use '-r' to destroy the following datasets:
...

So I added the -r flag and tried again with this result:

Code:

root@freenas[/mnt]# zfs destroy -rf Tank/iocage
cannot destroy 'Tank/iocage/jails/organizr': dataset is busy

Very odd. So I went and looked at the jail and it said organizr was corrupt. I deleted the jail and the volume from my Tank pool and it disappeared from my vm pool.

I feel like I am all messed up here. My jails show that it is using the vm pool, but I don't really trust that now.

How can I figure this out and free up the space on my Tank pool? I would really like to take advantage of the faster performance of the mirrored NVMe drives I just added.

 

[pschatz100]

How about giving us the specs of the new system? Did you create the pool manually, or by using the GUI? What pool is iocage activated on?

 

[rmccullough]

Its the same system, just a 2nd pool. I used the GUI to create the pool. The GUI is showing the iocage is on the vm pool.

 

[pschatz100]

Its the same system, just a 2nd pool. I used the GUI to create the pool. The GUI is showing the iocage is on the vm pool.

Is that where you activated your iocage jails? The iocage dataset should be on the pool that was activated. According to the post you referred to, early versions of iocage mounted the first iocage dataset in /mnt/iocage - you probably don't need that one anymore.

I don't think you can just delete jails. That is what "iocage destroy" command is for. I would go back to the post you referenced and follow the directions more closely.

Good luck.

 

[rmccullough]

@pschatz100 I am worried that I have an early version of iocage mounted in /mnt/iocage instead of in my pool.

I am also concerned that if I follow the instructions in that post, that I will lose my jails entirely.

Is this because I am still running FreeNAS 11.2 instead of 11.3 or TrueNAS?

It has the appearance now that if I remove my jails from my 'Tank' pool that it is also removing them from 'vm' as well.

 

[rmccullough]

Let me ask if I understand this. Is the idea that the dataset is under a pool, but mounted at /mnt/iocage?

Is the issue that there are still some of the data referenced from my old pool? I am very reluctant to run the destroy before I know exactly what it is going to do.

 

[Samuel Tai]

iocage uses whatever pool has a root dataset with the org.freebsd.ioc:active=yes property. If you're confused, there's 2 ways to check this:

• zfs get org.freebsd.ioc:active <name of your pool's root dataset>. If this returns yes, then this is the pool into which iocage will mount its datasets.
• In the GUI, click the little gear under Jails, in the upper right corner. In the ensuing dialog, the iocage pool will display.