migrate/convert/downgrade dataset to regular folder?

[digity]

I'm in the middle of a server migration of tens of TBs of data, I just realized a mistake (for my application/project) of putting folders in separate sub-datasets on the target FreeNAS server. I wiped the source NAS server already and have no space on the target server to maneuver by copying (with cp or rsync) the dataset contents into regular folders under the main dataset. How do I quickly migrate/convert/downgrade datasets to regular folders and keep the same timestamp, permissions, etc. for the data inside them?

 

[garm]

How do you migrate the data? ZFS Send?

 

[anmnz]

Assuming the source server is still available for use (even though wiped) you could ZFS send (or rsync) your datasets back to it, destroy the datasets on the target server, then rsync to directories on the target.

If you are stuck with just the target server you could "mv" files one at a time from your unwanted datasets to the main dataset, having made sure you don't have any snapshots of the unwanted datasets (and no job set up to create snapshots).

Both those approaches would do what you say you want and avoid creating copies of the data.

But this just sounds like trouble waiting to happen. How come you have already run out of free space on your new server?

 

[sretalla]

How do I quickly migrate/convert/downgrade datasets to regular folders and keep the same timestamp, permissions, etc. for the data inside them?

There is no shortcut.

If the target location is the parent dataset of the current one, the only way to have the data arriving there is to have it copied... you said you have insufficient space for that, so there's no way to do it without adding some temporary storage to make an intermediate step. (mv doesn't get around it as it's still a copy until the deletion at the end)

 

[anmnz]

mv doesn't get around it as it's still a copy until the deletion at the end

Yes though what I was trying and maybe failing to say was you could do it a file at a time, and then (absent snapshots) you'd only have one file duplicated at any time, instead of a whole dataset.

Though I do think that if OP has so little space they can't afford to make temporary copies, they're probably heading for disaster anyway...

 

[garm]

Even if the source system is wiped, the data can be moved back, then the desired dataset layout can be created on the target system and the data moved again

 

[digity]

yeah, I used mv in the past and as mentioned it appears to copy first, doubling storage usage.

I'm using rsync with the --remove-source-files option and that actually appears to be copying files and deleting during transfer instead of after transfer, so I guess this is working. It's slow though - I was hoping there was something fast like zfs send & recv