Best and quickest way to move Folders between datasets

[Darren Myers]

I am currently undergoing a transformation from 3 datasets to 5, and I would like to move individual folders (some as much as ~500Gb) from dataset A to dataset B.

Is it easier to use a medium such as Windows to facilitate the move, or can I do it via putty/ssh?

Or any other insight would be greatly appreciated.

 

[danb35]

Quickest is going to be to do it via SSH. The mv command will do it, as long as you aren't dealing with Windows permissions.

 

[Darren Myers]

Quickest is going to be to do it via SSH. The mv command will do it, as long as you aren't dealing with Windows permissions.

Sadly there are some Windows permissions ,but its just for 1 Windows user.

 

[danb35]

In that case, you may need to reset ownership and permissions after the move, if you use mv. I'm not aware of a way to do it that preserves ACLs.

 

[Nick2253]

Can't you do a "cp -p" to preserve the ACLs?

 

[Mlovelace]

You can rsync if you've got space for a copy of the dataset and not just a move. Rsync can preserve permissions, executability and timestamps 'rsync -pEt'

 

[Darren Myers]

You can rsync if you've got space for a copy of the dataset and not just a move. Rsync can preserve permissions, executability and timestamps 'rsync -pEt'

I will have space for a copy of the dataset, youre saying just copy the dataset and the data then delete what I dont need?

 

[Mlovelace]

I will have space for a copy of the dataset, youre saying just copy the dataset and the data then delete what I dont need?

Yes, you can rsync the data from one dataset to the next and preserve the permissions and timestamps in doing so, then after confirming everything is solid, delete the old dataset.

Edit: Your command would look something like this:

Code:

rsync -rvpEt /mnt/<pool name>/<dataset name>/<directory>/ /mnt/<pool name>/<new dataset name>/

 

[Darren Myers]

Yes, you can rsync the data from one dataset to the next and preserve the permissions and timestamps in doing so, then after confirming everything is solid, delete the old dataset.

Edit: Your command would look something like this:

Code:

rsync -rvpEt /mnt/<pool name>/<dataset name>/<directory>/ /mnt/<pool name>/<new dataset name>/

That is awesome! Just what I need, and just what I am going to do.

 

[Mlovelace]

That is awesome! Just what I need, and just what I am going to do.

You may or may not want the verbose flag (v) in the options, depending on how many files are in the directory... ;)

Edit: Also if you want to copy the root directory of the path and not just the files in the directory, leave the trailing / off the source. Example:

Code:

rsync -rvpEt /mnt/<pool name>/<dataset name>/<directory> /mnt/<pool name>/<new dataset name>/

 

[Spearfoot]

You may or may not want the verbose flag (v) in the options, depending on how many files are in the directory... ;)

Edit: Also if you want to copy the root directory of the path and not just the files in the directory, leave the trailing / off the source. Example:

Code:

rsync -rvpEt /mnt/<pool name>/<dataset name>/<directory> /mnt/<pool name>/<new dataset name>/

I hope rsync works for you... but I haven't had much luck with it preserving permissions correctly for Windows datasets, gory details here.

 

[Darren Myers]

You may or may not want the verbose flag (v) in the options, depending on how many files are in the directory... ;)

Edit: Also if you want to copy the root directory of the path and not just the files in the directory, leave the trailing / off the source. Example:

Code:

rsync -rvpEt /mnt/<pool name>/<dataset name>/<directory> /mnt/<pool name>/<new dataset name>/

I had asked a similar question last year actually, but it was to move an entire dataset not just some folders. Would this still be relevant to accomplish the same thing, without destroying the original of course.

Or, if you want to move all existing snapshots and properties, you may do something like this:
1. Create final source snapshot:

Code:

zfs snapshot -r Data2/Storage@copy

2. Copy the data:

Code:

zfs send -Rv Data2/Storage@copy | zfs receive -F Data1/Storage

3. Delete created snapshots:

Code:

zfs destroy -r Data1/Storage@copy ; zfs destroy -r Data2/Storage@copy

Source: https://forums.freenas.org/index.php?threads/copy-move-dataset.28876/#post-189799

 

[Mlovelace]

If like @Spearfoot suggests that rsync won't keep the windows permissions, you could robocopy the data via CIFS between the two datasets.

 

[Mlovelace]

I had asked a similar question last year actually, but it was to move an entire dataset not just some folders. Would this still be relevant to accomplish the same thing, without destroying the original of course.

Yes, you can rsync the entire contents of a dataset. I do this to keep some data sync'd between freeNAS and a non-zfs Linux system.

 

[Mlovelace]

@Darren Myers Did rsync keep the window's permissions when you migrated your data?