Migrating to new disks

[rdhir]

I am trying to migrate my current set up of a bunch of datasets across
2 old zpools (4x 1TB [zpool1], 4x 2TB[bigpool])
to a single new zpool (4x4TB) [hugepool]

I used the following to copy my zpool datasets

zfs snapshot -r bigpool@migrate
zfs send -R bigpool@migrate | zfs recv -Fv hugepool

This seems to have copied the datasets off one pool on to the new one.
When I tried to then do the same for zpool1, I get an error saying there is already a snapshot on the destination drive.

I don't really understand snapshots and despite reading the documentation on Oracle I am still struggling. If I delete the snapshot will that delete the datasets on the new system as well?

Reading around some other threads it would seem the correct approach is to

a) Make the snapshot
b) Replicate the snapshot
c) Make the snapshot a clone - so it is now writable
d) Promote the clone so the old zpool is a clone of the new zpool
e) destroy the old zpool
f) Now I can destroy the snapshots on the new zpool
g) repeat with the second old zpool

Is this correct or is there a better method of migrating datasets to new zpools(disks)?

It would be useful to know as in a months time, I would like to take the disks from the destroyed pool and bring them back in as a new zpool into the system. So knowing how to migrate a zpool or datsset from one device to another would be good.

thanks

 

[depasseg]

I've recently gone through this and I just wrote down the steps. Take a look at see if they make sense or where I can add some clarification.

https://forums.freenas.org/index.php?threads/how-to-migrate-data-from-one-pool-to-another.40519/

 

[Robert Trevellyan]

If I delete the snapshot will that delete the datasets on the new system as well?

No.

The issue is probably that you're trying to use the name "migrate" for the snapshot from both source pools. Just use a different name for the 2nd pool's snapshot if you don't want to delete the replicated snapshot yet.

 

[rdhir]

No I had a different name for the snapshot as I was appending a date eg migrate-20160116.
It still gave me an error when I tried to migrate the zpool and had destroyed all the snapshots in the destination zpool .
However it worked fine when I moved the datasets one at a time.

eg

veryoldpool
__veryoldpool
____dataset1
____dataset2
oldpool
__oldpool
____dataset3
____datase4
newpool
__ newpool


so

zfs snapshot -r veryoldpool@migrate-20160109
zfs snapshot -r oldpool@migrate-20160116

zfs send -R oldpool@migrate-20160116 | zfs recv -Fv newpool

and we get

veryoldpool
__veryoldpool
____dataset1
____dataset2
oldpool
__oldpool
____dataset3
____datase4
newpool
__newpool
____dataset3
____dataset4

then to clear out snapshots but not data on new datsets

zfs destroy -R newpool@migrate-20160116

you would think you could then do (not options changed to -v as we dont' want to trash the datasets we just migrated)

zfs send -R veryoldpool@migrate-20160109 | zfs recv -v newpool

but this error saysing snapshots exist which they don't according to zfs list -t snapshot

Anyway what does then work is this

zfs send -R veryoldpool/dataset1@migrate-20160109 | zfs recv -v newpool/dataset1
zfs send -R veryoldpool/dataset2@migrate-20160109 | zfs recv -v newpool/dataset2


and after destroying the snapshots in the newpool and detaching the old pools without destroying data, I have


newpool
__newpool
____dataset1
____dataset2
____dataset3
____dataset4



So mission accomplished, but I'm still curious as to why I can't copy a second pool, but I can copy the datsets.

Thanks Robert for re-assuring me. I eventually spent a couple more hours reading about snapshots to help.
Depasseg, I appreciate the desire to do it through the GUI and I'm amazed what I wanted to do is not better supported, but I really needed to understand what I was about to do and I'm old fashioned in that for multi-hour migration I prefer the command line, not a GUI where one is never quite sure if its kept up to date with the underlying processes.

 

[depasseg]

So mission accomplished, but I'm still curious as to why I can't copy a second pool, but I can copy the datsets.

The reason is that ZFS receive, by default, will initialize the filesystem. Since you already had data in the destination, zfs gave you a warning. You might have been able to send the veryoldpool data as incremental snapshots, but I'm not sure if it would have worked.

From http://docs.oracle.com/cd/E19253-01/819-5461/gbchx/index.html
"When you send a full stream, the destination file system must not exist."