Howto: migrate data from one pool to a bigger pool

Apollo

Wizard
Joined
Jun 13, 2013
Messages
1,449
Hello folks. I am in a process of upgrading my storage very soon, and I found this very helpful guide, and I do see some additional information provided by other people in this thread. I am a bit slightly confused in procedures combining depasseg, alexr and CPLeyden1282's posts. So here is the completed procedures from my understanding.


1.) Move off the system dataset.

2) Create system config backup.

3) Create a snapshot of the dataset I'd like to move.

4) Replicate the dataset from the old one to the new one using CPLeyden1282's instruction. Using GUI for 11.2 ver.

5) Detach both old and new volumes.

6) Import & rename pools using CLI, then export them.

7) Use the GUI to import the renamed pools so FreeNAS can understand them.

8) Using the backup from 2, bring back the old share.

9) Fix the GUID so that I can detach/wipe the old volume without problem, using alexr's instruction.

10) Wipe out the old volume if I like.

Are these procedures correct?

Also, if I do this, does Jail get moved as well? Since I will be moving enitre data from one to other, I assume Jail datasets are being copied as well, or is it not? Do I have to follow the steps for moving Jail (mentioned in OP) separately?
If you can affords replicating to another machine on the network, things would be almost easier.
Export should work if your pool isn't encrypted.
 

Lighthouse

Dabbler
Joined
Nov 15, 2018
Messages
15
If you can affords replicating to another machine on the network, things would be almost easier.
Export should work if your pool isn't encrypted.

Yes, in fact I have one small miniPC running Ubuntu server for learning docker. I could use that computer. But, I can't always go for the 'easy' way for everything.

The pool is not encrypted. Today is the day I pick up HDDs from Best Buy, and will spend some days to check the drive, install LSI IT-moded controller then I shall see how it goes.
 
Last edited:

Lighthouse

Dabbler
Joined
Nov 15, 2018
Messages
15
So I finished initial HDD tests, backup the data, created the pool, backup the config, created snapshot.

However, as usual the GUI of web interface is bugged, thus I could not use CPLeyden1282's method; my old pool/snapshot just does not show up to be chosen as source. I know GUI is useless for many cases because of bugs but I really think the bugs have to be ironed out. Like, at least dashboard should be reliably displaying status of the file server, for instance. This is one of the reasons I was really thinking about ditching FreeNAS after all.

I used the command zfs send -R <old pool>@<name of snapshot> | zfs receive -F <new pool>, with the addition of '-F'. I have to give the command to overwrite the existing <new pool>.

Now the data is being copied from my old pool to new pool. I will keep posting as I find new problems and/or new symptoms. I will update my initial procedure post.

Edit : initial procedure post is here : https://www.ixsystems.com/community...ool-to-a-bigger-pool.40519/page-3#post-517319
 

Apollo

Wizard
Joined
Jun 13, 2013
Messages
1,449
So I finished initial HDD tests, backup the data, created the pool, backup the config, created snapshot.

However, as usual the GUI of web interface is bugged, thus I could not use CPLeyden1282's method; my old pool/snapshot just does not show up to be chosen as source. I know GUI is useless for many cases because of bugs but I really think the bugs have to be ironed out. Like, at least dashboard should be reliably displaying status of the file server, for instance. This is one of the reasons I was really thinking about ditching FreeNAS after all.

I used the command zfs send -R <old pool>@<name of snapshot> | zfs receive -F <new pool>, with the addition of '-F'. I have to give the command to overwrite the existing <new pool>.

Now the data is being copied from my old pool to new pool. I will keep posting as I find new problems and/or new symptoms. I will update my initial procedure post.

Edit : initial procedure post is here : https://www.ixsystems.com/community...ool-to-a-bigger-pool.40519/page-3#post-517319
Nothing better than the old CLI commands.
And because it will take time for the replication to take place and in the event your current pool/volume is still in use, I would take a recursive snapshot of the original pool when it as competed, followed by a recursive incremental snapshot to the new pool and then stop all jails, disable services related to shares and take one last recursive snapshot. At that point performing recursive replication shouldn't take too long and modification of the original volume be almost inexistant. Perform a last recusive replication and both original and new pool should be identical.
At that point you can export new pool to to the original name and possibly shut down server, remove the original pool and restart. Hopefully, you will not need importing the new volume as it should be picked up as if the old one.
 

Lighthouse

Dabbler
Joined
Nov 15, 2018
Messages
15
So, the replication is completed, and I renamed the pools. One difference is that instead of depasseg's method where he
"zpool import <old pool> old<old pool>" | "zpool import <new pool> <old pool>", changing old pool name into something eise, I just swapped the names, like:

"zpool import <old pool> <new pool>" | "zpool import <new pool> <old pool>"
I import the both pools and the data is all fine. The jails are shown as empty unless I detach one of the pools. I checked the data, deleted old pool which is now renamed as new pool, then the jails are one again shown and I could just activate them.

Also, when I detach the pools, FreeNAS actually asks whether shares data should be saved or not. With this backup the config is actually not needed.

And... that's it. With this procedure, alexr's procedure is not needed.
 

Lighthouse

Dabbler
Joined
Nov 15, 2018
Messages
15
Nothing better than the old CLI commands.
And because it will take time for the replication to take place and in the event your current pool/volume is still in use, I would take a recursive snapshot of the original pool when it as competed, followed by a recursive incremental snapshot to the new pool and then stop all jails, disable services related to shares and take one last recursive snapshot. At that point performing recursive replication shouldn't take too long and modification of the original volume be almost inexistant. Perform a last recusive replication and both original and new pool should be identical.
At that point you can export new pool to to the original name and possibly shut down server, remove the original pool and restart. Hopefully, you will not need importing the new volume as it should be picked up as if the old one.

Good old CLI commands is good but it does not mean those bugs can be left as they are. What if it is bugged in a way that does malicious behaviour, say instead of the replication it corrupts the source pool? The whole point of making FreeNAS appliance-like and installing packages outside of the jail is discouraged do not help the situation either.

I completed editing the procedure. This should work for majority of people.
https://www.ixsystems.com/community...ool-to-a-bigger-pool.40519/page-3#post-517319
 

Snow

Patron
Joined
Aug 1, 2014
Messages
309
Hello folks. I am in a process of upgrading my storage very soon, and I found this very helpful guide, and I do see some additional information provided by other people in this thread. I am a bit slightly confused in procedures combining depasseg, Snow and CPLeyden1282's posts. So here is the completed procedures from my understanding.

Edit (3-13-2019): With this procedure I successfully migrated my pool into bigger pool.


<old pool> = current pool's name
<new pool> = any tempory name for added pool.


1.) Move off the system dataset.

2) Create system config backup. May not be needed but just in case.

3) Create a snapshot of the dataset I'd like to move, which will be <name of snapshot>.

4) Replicate the dataset from the old one to the new one using CPLeyden1282's instruction. Using GUI for 11.2 ver.
Or with high chance GUI is bugged and not working, then use the commend "zfs send -R <old pool>@<name of snapshot> | zfs receive -F <new pool>"

***If you use remote ssh connection, when you close it the replication will stop. Use the command "zfs send -Rv <old pool>@<name of snapshot> | pv | zfs receive -Fdu <new pool>" provided by Snow.***

5) Detach both old and new volumes. When detaching the volume, do not select "destroy" and select to save share-related configs.

6) Import & swap the names of pools using CLI, then export them.
"zpool import <old pool> <new pool>" and "zpool import <new pool> <old pool>"
Then
"zpool export <old pool>" and "zpool export <new pool>"

7) Use the GUI to import the renamed pools so FreeNAS can understand them.

8) If something happened to share and not working, use the system config backup did on 2).

9) Wipe out the old volume if I like.
Thanks for Updating it. :)
For some reason I wasn't getting automated replies from this thread after the date I posted.
I never received any notifications that anybody else posted after I did> Thanks again
Lighthouse
 
Last edited:

phomchick

Dabbler
Joined
Oct 2, 2017
Messages
26
I'm about to move my data to a larger reconfigured pool, but it's scary.

I don't have a machine big enough to contain two copies of all of my data. Currently I have two pools, a zfs1 3-drive pool (old_pool_1 for discussion), and a zfs2 5-drive pool (old_pool_2). I want to recreate the same dataset structure on a new zfs3 8-drive pool and have everything including shares and jails continue to work.

Here's my plan.
  1. Replicate my two pools to another machine where they will be stored as dup_pool/old_pool_1 and dup_pool/old_pool_2
  2. Save the system config on the original machine
  3. Detach old_pool_1 and old_pool_2 on the original machine, erasing all data (Marking the drives as new [destroy data]). Do I also want to "Also delete the share's configuration:"?
  4. Create a new zfs3 8-drive pool (new_pool) on the original machine.
  5. Set up a replication scheme to replicate dup_pool/pool_1 and dup_pool/pool_2 from the backup machine to new_pool on the original machine.
  6. Export new_pool on the original machine
  7. Restore the original system config
  8. Import new_pool
  9. Fix the pointers to jails and shares
I'm not sure I even need to do the export and import, would someone please clarify? And it seems this should be very similar to recovering from a backup after a hardware failure. Are there good threads on that?

Thank you,

P. Homchick
 

Apollo

Wizard
Joined
Jun 13, 2013
Messages
1,449
@phomchick , If you have a second computer, the better approach would be to create the zfs3 8 disk pool on it with the same volume name as the source. Do recursive replication which is going to take from a couple hours to a few days depending on LAN speed and number of snapshots.
Once replication is complete, stop all the jails from the source, perform a recursive snapshot and then do one last run of incremental replication. This shouldn't take too long. Mere hours in the worst case, a few minutes in the fastest.
Then you can swap your volumes and run on the same boot drive or move the boot drive to the new machine.
If you swap the drives (new pool to the old machine) you can keep your boot drive and upon start up, it should map all the drives automatically.
Jails should be restarted as if nothing happened.

It is normally a strait forward procedure.
If however the new pool isn't detected properly, just detach it and attach it again without erasing the shares.
 

phomchick

Dabbler
Joined
Oct 2, 2017
Messages
26
@Apollo , thank you for the clever and easy suggestion. Unfortunately, while I have two computers, they are not equivalent. The one where I want to destroy two pools and recreate a new z3 pool has eight 3.5" 4tb drives. The computer I'm using to back it up, has six 2.5" 4tb drives. I guess I'm going to go with my original plan and see what happens. The jails are the only thing I'm really concerned about.
 

Apollo

Wizard
Joined
Jun 13, 2013
Messages
1,449
@phomchick , Reading up up the post more than once I better understand the issue.
I would add some notes of caution before you proceed away.

First of all, which pool is assigned the Jails as I would try to leave this one as is (from the structure point of view), this will allow you to use the same config file without too much issues I believe, but you will still have to changes configs at the jail level if they have path pointing to the second volume. This is doable but can be time consuming.
I think it shouldn't be too difficult to create the main pool and have the 2 datasets, each one assigned as you suggested.
Freenas should be able to locate the jails on its own I believe.

Where you might have some problemes are the various user account has you will have to remap the home folder for each one otherwise permission will/may not work.

If your down time is not too important, I would try to experiment replicating back on a single drive ( Not destroying the 2 pools you currently have) and see if you can manage the process you have outlined. It will require performing replication to just the few datasets, such as user accounts, jails and see how it works.
 

Jessep

Patron
Joined
Aug 19, 2018
Messages
379
Why not buy another HBA and do it all at once in the same machine?
 

phomchick

Dabbler
Joined
Oct 2, 2017
Messages
26
@Jessep "Why not buy another HBA and do it all at once in the same machine?" Not enough drive bays and not enough spare hard drives. This is a home computer lab, not a commercial environment. But I was looking at a 16-bay 2U server for cheap. I could do it, but I'd rather keep the $1K that the "easy" route would consume.
 
Last edited:

Apollo

Wizard
Joined
Jun 13, 2013
Messages
1,449
Doing it on the same system will make things a bit more risky to implement. Would be definitely faster at replicating though.
Regardless, @phomchick intent is to combine the two volumes into one which in itself going to require some tinkering with the shares, iocage jails and config.
 

phomchick

Dabbler
Joined
Oct 2, 2017
Messages
26
I've started the process. In two days, I'll know if it worked out. :smile:
 

phomchick

Dabbler
Joined
Oct 2, 2017
Messages
26
The system is back up and running, using the process I proposed in post #70. It was pretty straight-forward but here are a few things I learned.

If you change the name of the pool (as I did) you'll have some extra work, but it's obvious and won't take much time.
  • I had to change the jail root under jails configuration. After that, the jails came up and ran.
  • I had an iSCSI share that needed to be reconfigured
  • My Windows SMB share needed to be upgraded with the new dataset path
  • When replicating and filling up the new pool, the mount points of the new data didn't show up. Detaching (without destroying the data) and then importing the pool fixed that.
  • Two jails had added storage. These needed to be updated
  • Don't forget to re-add your snapshot tasks (and replication if you are using it).
All in all, my fears about the difficulty of the task were way exaggerated.
 
Last edited:

Apollo

Wizard
Joined
Jun 13, 2013
Messages
1,449
Well, good job.
Not many people manage to go through this without either loosing sleep or screwing things up.
Make sure the now active pool dosen't have write protected datasets.
 
Top