Relocate Jails to SSD helping HDD sleep?

Status
Not open for further replies.

CrazySurfaNZ

Cadet
Joined
Apr 7, 2014
Messages
4
Actually, nothing nasty happens. The jails won't start of course, but FreeNAS will continue running without problems.

In reality, the config DB contains only single reference to the jails location and you can change it via GUI. The reason why you can't just cp it over is that the jails (warden) uses ZFS features to use disk space effectively (that's the reason you can't use UFS with FreeNAS jails). One plugin jail consumes about 700MB of disk space. With 5 plugins/jails you would waste 4*700MB=2.8GB of disk space by basically identical files. Instead of creating copies, warden does a snapshot of the plugin template dataset and then creates individual plugin jails as ZFS clones of the template snapshot. If you cp this to a new location you will lose the snapshot/clone links. Therefore you need to use ZFS replication to relocate the jails. Another small complication is that warden explicitly sets the template dataset mountpoint.

Enough theory, this is the "Relocate jails how-to" :) :
Assumptions:
  • The pool you are transferring the jails from is main_pool
  • The destination pool is ssd_pool
  • The jail root (Jails->Configuration) is /mnt/main_pool/jails
  • The new jail root will be /mnt/ssd_pool/jails
Steps:
  1. Turn off all plugins (Plugins->Installed)
  2. Stop all jails (Jails->View Jails)
  3. Run these commands via CLI:[PANEL]zfs snapshot -r main_pool/jails@relocate
    zfs send -R main_pool/jails@relocate | zfs receive -v ssd_pool/jails
    zfs get -rH -o name -s received mountpoint ssd_pool/jails | xargs -I {} sh -c "zfs set mountpoint=/{} {}; zfs mount {};"[/PANEL]
  4. Change the Jail Root to /mnt/ssd_pool/jails (Jails->Configuration)
  5. Start jails/plugins
  6. Check that everything works and destroy the original jails dataset (main_pool/jails)

I just did this myself on my freenas box.

A couple of things I found out:
1) Change to /mnt before you run the commands. You could probably include /mnt in the command itself, but I wanted to use the command exactly as Dusan posted it for maximum chances of success.
2) Don't try to be smart and create the 'jails' dataset on the new pool yourself before you run the commands, the commands will complain like follows:

[root@freenas /mnt]# zfs snapshot -r megaPool/jails@relocate
[root@freenas /mnt]# zfs send -R megaPool/jails@relocate | zfs receive -v dotsystem/jails
cannot receive new filesystem stream: destination 'dotsystem/jails' exists
must specify -F to overwrite it
warning: cannot send 'megaPool/jails@relocate': Broken pipe
cannot send 'megaPool/jails': I/O error

After I deleted my 'dotsystem/jails' dataset it worked as expected, though I got an error when I tried to create the snapshot again:

[root@freenas ~]# cd /mnt/
[root@freenas /mnt]# zfs snapshot -r megaPool/jails@relocate
cannot create snapshot 'megaPool/jails@relocate': dataset already exists
cannot create snapshot 'megaPool/jails/sabnzbd_1@relocate': dataset already exists
cannot create snapshot 'megaPool/jails/plexmediaserver_1@relocate': dataset already exists
cannot create snapshot 'megaPool/jails/sonarr_1@relocate': dataset already exists
cannot create snapshot 'megaPool/jails/.warden-template-pluginjail@relocate': dataset already exists
no snapshots were created
[root@freenas /mnt]# zfs send -R megaPool/jails@relocate | zfs receive -v dotsystem/jails
receiving full stream of megaPool/jails@relocate into dotsystem/jails@relocate
received 111KB stream in 1 seconds (111KB/sec)
receiving full stream of megaPool/jails/.warden-template-pluginjail@clean into dotsystem/jails/.warden-template-pluginjail@clean
received 1.19GB stream in 13 seconds (93.7MB/sec)
receiving incremental stream of megaPool/jails/.warden-template-pluginjail@relocate into dotsystem/jails/.warden-template-pluginjail
@relocate
received 48.6KB stream in 1 seconds (48.6KB/sec)
found clone origin dotsystem/jails/.warden-template-pluginjail@clean
receiving incremental stream of megaPool/jails/sabnzbd_1@relocate into dotsystem/jails/sabnzbd_1@relocate
received 234MB stream in 4 seconds (58.5MB/sec)
found clone origin dotsystem/jails/.warden-template-pluginjail@clean
receiving incremental stream of megaPool/jails/plexmediaserver_1@relocate into dotsystem/jails/plexmediaserver_1@relocate
received 2.84GB stream in 54 seconds (53.9MB/sec)
found clone origin dotsystem/jails/.warden-template-pluginjail@clean
receiving incremental stream of megaPool/jails/sonarr_1@relocate into dotsystem/jails/sonarr_1@relocate
received 545MB stream in 7 seconds (77.9MB/sec)
[root@freenas /mnt]# zfs get -rH -o name -s received mountpoint dotsystem/jails | xargs -I {} sh -c "zfs set mountpoint=/{} {}; zfs
mount {};"
 

Bish

Cadet
Joined
Aug 1, 2014
Messages
8
I just did this myself on my freenas box.

A couple of things I found out:
1) Change to /mnt before you run the commands. You could probably include /mnt in the command itself, but I wanted to use the command exactly as Dusan posted it for maximum chances of success.

I've never found that necessary, and indeed I think the first time I did it I added the full path and got an error. I think the zfs command works with dataset names rather than Unix paths, so maybe your two names were too similar somehow? Cleverer people than me may be able to clarify.

2) Don't try to be smart and create the 'jails' dataset on the new pool yourself before you run the commands...
Yeah, someone else in the thread above made that mistake, but it bears repeating.

After I deleted my 'dotsystem/jails' dataset it worked as expected, though I got an error when I tried to create the snapshot again...
Again, there are plenty of people more qualified than me to explain, but it looks like it's just pointing out that it already has an up to date snapshot with the @relocate name. I dunno, I might've misunderstood how snapshots work.

While I'm here again, though, I'm just going to reiterate how useful these steps have been for me. The reshuffle I did the other night has enabled me to solve some niggling performance issues, and my box is now performing like an absolute monster. Thanks again, Dusan! I'm curious to know why the 'relocate button' idea doesn't seem to have made it into the build, though?
 

CrazySurfaNZ

Cadet
Joined
Apr 7, 2014
Messages
4
Again, there are plenty of people more qualified than me to explain, but it looks like it's just pointing out that it already has an up to date snapshot with the @relocate name. I dunno, I might've misunderstood how snapshots work.

While I'm here again, though, I'm just going to reiterate how useful these steps have been for me. The reshuffle I did the other night has enabled me to solve some niggling performance issues, and my box is now performing like an absolute monster. Thanks again, Dusan! I'm curious to know why the 'relocate button' idea doesn't seem to have made it into the build, though?
Yes, I think the @relocate part of the command is actually specifying the name of the snapshot... so running it again just tries to recreate the same snapshot again which fails since you haven't deleted the existing one. The error saying use -F to force overwriting it seems to indicate this too.

And yeah before I went and did the manual steps I actually went and looked for a 'relocate jails' button. Maybe in 10.0 ;)
 

CraftyClown

Patron
Joined
Apr 5, 2014
Messages
214
Just wanted to add my thanks to Dusan for this quick and handy solution.

It really couldn't have been easier :D
 

Krowvin

Explorer
Joined
Jul 24, 2014
Messages
60
Thanks for this Question/Response! I'll be giving Dusan's method a try when I get home tonight.

What started off as research in to how to speed up Plex transcoding/Poster lookup has turned in to a full blown migration of all my jails and .system files. I'm seriously considering letting my drives spin down on non-peak times. (Thanks to PlexWatch i'll know when those times are)

Going to make a separate post about whether or not to choose SSD in a Mirror or a RaidZ1 (Not trying to hijack this thread!)

Here's a link to the above question: (Posted it in the hardware section)
https://forums.freenas.org/index.php?threads/raidz-vs-mirroring-ssd-for-plex-system-files.36986/
 
Last edited:

Krowvin

Explorer
Joined
Jul 24, 2014
Messages
60
Silly Question:

If we move our jails over to the SSD dataset do the snapshots go with?

More directly, they are linked to that location. But are they saved to that location? If the SSD dataset were to fail, would I/others be able to recover from snapshots if they are saved on another dataset/pool? i.e. main_pool vs ssd_pool
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Snapshots are intrinsically part of the pool that contains the data they reference.
 

angelus249

Dabbler
Joined
Dec 19, 2014
Messages
41
I actually just added a new item to my list of dev things -- to add a "Relocate Jails" button to FreeNAS :).

How is that going? Any progress on the relocate button? ;)

And since I don't have any SATA ports availbe for SSD drives, are there any caveats to maybe use e.g. 2x64GB USB drives, mirrored as jailroot?
 

theCMack

Cadet
Joined
Feb 15, 2016
Messages
4
Actually, nothing nasty happens. The jails won't start of course, but FreeNAS will continue running without problems.

In reality, the config DB contains only single reference to the jails location and you can change it via GUI. The reason why you can't just cp it over is that the jails (warden) uses ZFS features to use disk space effectively (that's the reason you can't use UFS with FreeNAS jails). One plugin jail consumes about 700MB of disk space. With 5 plugins/jails you would waste 4*700MB=2.8GB of disk space by basically identical files. Instead of creating copies, warden does a snapshot of the plugin template dataset and then creates individual plugin jails as ZFS clones of the template snapshot. If you cp this to a new location you will lose the snapshot/clone links. Therefore you need to use ZFS replication to relocate the jails. Another small complication is that warden explicitly sets the template dataset mountpoint.

Enough theory, this is the "Relocate jails how-to" :) :
Assumptions:
  • The pool you are transferring the jails from is main_pool
  • The destination pool is ssd_pool
  • The jail root (Jails->Configuration) is /mnt/main_pool/jails
  • The new jail root will be /mnt/ssd_pool/jails
Steps:
  1. Turn off all plugins (Plugins->Installed)
  2. Stop all jails (Jails->View Jails)
  3. Run these commands via CLI:[PANEL]zfs snapshot -r main_pool/jails@relocate
    zfs send -R main_pool/jails@relocate | zfs receive -v ssd_pool/jails
    zfs get -rH -o name -s received mountpoint ssd_pool/jails | xargs -I {} sh -c "zfs set mountpoint=/{} {}; zfs mount {};"[/PANEL]
  4. Change the Jail Root to /mnt/ssd_pool/jails (Jails->Configuration)
  5. Start jails/plugins
  6. Check that everything works and destroy the original jails dataset (main_pool/jails)

Hey just wanted to say thanks so much for the post I followed it and it worked like a champ. One note is I did need to restart the server for the jails to show up in the new path when changing the Jail Root.. Thanks again Dusan!!
 

meku

Dabbler
Joined
May 4, 2014
Messages
34
Are people aware of the 'warden' command for managing jails? I didn't have success with the snapshot-copy method to move a jail, but below worked.

Do this for each of your jails to stop and export to myjail.wdn in the current jail root.
warden stop myjail
warden export myjail


Change jail root in FreeNAS settings, then import each jail. Had to include 'host' so that it would name the jail correctly otherwise it was named 'OFF'
warden import myjail.wdn --host=myjail
 
Joined
Apr 23, 2014
Messages
6
Do this for each of your jails to stop and export to myjail.wdn in the current jail root.
warden stop myjail
warden export myjail


Change jail root in FreeNAS settings, then import each jail. Had to include 'host' so that it would name the jail correctly otherwise it was named 'OFF'
warden import myjail.wdn --host=myjail
I have just done these commands and worked pretty well. The import function gave all jails an id of 1 so had to manually edit them but other than that I have move my jails.
My next problem is that my plugins look to have lost their links to the jails.
Does anyone know how to restore the link between the plugins and the newly imported jails?
Thanks for the help so far.
 

Attachments

  • plugins.png
    plugins.png
    137.7 KB · Views: 269

ric

Contributor
Joined
Dec 22, 2013
Messages
180
Hey just wanted to say thanks so much for the post I followed it and it worked like a champ. One note is I did need to restart the server for the jails to show up in the new path when changing the Jail Root.. Thanks again Dusan!!
I did exactly the same how you did here, and followed exactly in the instructions one at a time but nothing seems to be working for me, and still my freenas system/HDDs doesn't go to sleep.
I am currently using WD NAS Red HDD (RAIDZ2), and I've Freenas 9.3 Stable ver. installed into usb thumb drive running just fine . All Jails are relocated to SSD. I've snapshotted as well.

Please help!
 
Joined
Apr 23, 2014
Messages
6
I have had some success! I installed another plugin and noticed the created jail was of type "pluginjail" where as my warden imported jails were "standard".

So I edited the /mnt/path/to/jails/.jailname.meta/jailtype file to contain the text "pluginjail" and after rebooting the plugins are working with imported jails!

I still have two jails that wouldn't get DHCP address but I think this jailtype issue is the problem we both had.
 

Simplicity

Dabbler
Joined
Jun 13, 2015
Messages
28
I am trying to execute this command but nothing happens zfs get -rH -o name -s received mountpoint ssd_pool/jails | xargs -I {} sh -c "zfs set mountpoint=/{} {}; zfs mount {};"
 

Simplicity

Dabbler
Joined
Jun 13, 2015
Messages
28
I rebooted after following the steps and my jails have been migrated, but all my plugins are missing?
 

rm-r

Contributor
Joined
Jan 7, 2013
Messages
166
I rebooted after following the steps and my jails have been migrated, but all my plugins are missing?

you need to change the location of the jails in the web UI under "jails" > "configuration" to the new location :)
 

Simplicity

Dabbler
Joined
Jun 13, 2015
Messages
28
you need to change the location of the jails in the web UI under "jails" > "configuration" to the new location :)
Hi buddy, thanks for the reply. I have already done that as per the original steps. I already re-located my new jail settings to ssd > jails. My jails have loaded properly off the SSD (i already deleted the jails on my HDD) I was able to manually turn on the plugins via editing the rc.config file but they are still missing from GUI :(
 

PnoT

Dabbler
Joined
Apr 12, 2017
Messages
41
I setup FreeNAS about a month ago and neglected to setup a dedicated jail dataset and instead used the root of my ssd pool as a dumping ground =/

current pool: ssd | /mnt/ssd
current jails: mariadb | /mnt/ssd/mariadb

I have since created a dataset, /mnt/ssd/Jails, and would like to move them to that location but was unable to using the commands listed earlier in the thread.

My steps have been:

1. zfs snapshot ssd/mariadb@relocate
2. zfs send ssd/mariadb@relocate | zfs receive -v ssd/Jails/mariadb (I can see the snapshot under the correct Jail dataset at this point)
3. zfs get -rH -o name -s received mountpoint ssd/Jails | xargs -I {} sh -c "zfs set mountpoint=/{} {}; zfs mount {};"
This command has no output and running just the beginning of it "zfs get -rH -o name -s received mountpoint ssd/Jails" doesn't either.
4. changed Jail root from /mnt/ssd to /mnt/ssd/Jails

At this point I can't see any jails and a reboot didn't solve that problem either. If I change the jails root back to what it was before the jail shows up. Can someone help me out on this one?

thanks
 
Last edited:

pschatz100

Guru
Joined
Mar 30, 2014
Messages
1,184
You are missing a couple of options in your zfs commands. Also, be careful to honor upper case or lower case options in the command - it makes a difference. Adjust the dataset names to fit your installation.

Here are the steps, where NAS is the old volume and JAILS is the new volume (replace NAS and JAILS with your volume names)
1. Turn off all plugins and jails.
2. Run these commands via the WebGUI Shell:
zfs snapshot -r NAS/jails@relocate
zfs send -R NAS/jails@relocate | zfs receive -v -F JAILS
zfs get -rH -o name -s received mountpoint JAILS | xargs -I {} sh -c "zfs set mountpoint=/{} {}; zfs mount {};"
3. Then change the Jail Root to /mnt/JAILS/
4. Reboot
5. Restart jails then restart plugins
 
Status
Not open for further replies.
Top