Captain Proton
Cadet
- Joined
- Sep 15, 2017
- Messages
- 4
I'm migrating from 11.3-U5 to 12.0-U2.1. Both NAS's are VMs within the same hypervisor. I've got a Dataset called "Media" on a pool named "sweet" and I want to replicate it to a new pool on the new NAS. The old/original "Media" datasets use 128k records, I want to go to 1MiB on the new instance of "Media". I created a new Dataset called "Media" on the new system/pool thusly:
http://nothingunreal.com/dump/MediaCreation.PNG
And then a replication task to copy the data:
http://nothingunreal.com/dump/MediaPull.PNG
Based on all the tooltips, documentation, and forum posts I have read; this replication task, as shown in the linked screenshot, should not change the attributes of the target dataset. But it does. The record size on the target dataset changes to 128k as soon as I start the replication.
late night edit: I decided to make it a local transfer by shutting down the old VM and giving the new one its controller, thereby directly connecting the legacy pool "sweet" directly to the 12.0-U2.1 VM. Even doing it as a local transfer, with similar settings, the recordsize attribute immediately changes to 128k after starting the replication task. Last option I can think of is a command line copy from one dataset to the other. Never done that with 44TiB before.
a couple of hours after taking my usual sleeping pill but pushing through because madness edit: The command line copy is going at a good clip, about as fast as SSH+NETCAT. The recordsize attribute of the new Media dataset has not changed; still 1MiB. Used to copy via command line
A day and a half later: So, in the end my new dataset is 810GB smaller and I can't figure out why.
Thanks for all the help, everyone.
About 5 days later? Given that I wasn't able to use ZFS replication to copy my Media dataset, I used an external program to read both datasets, generate, and compare hash values for all the files. Two files had identical properties but different hash values. They were movies and a little bit-rot or whatever wouldn't likely have made any perceptible difference in playback, but I copied and checked those two files again and the hash values are a match. yay.
Best guess on the change in size of the dataset is almost certainly due to differences in compression, ashift misalignment, and other factors I've forgotten to consider or am blatantly unaware of.
http://nothingunreal.com/dump/MediaCreation.PNG
And then a replication task to copy the data:
http://nothingunreal.com/dump/MediaPull.PNG
Based on all the tooltips, documentation, and forum posts I have read; this replication task, as shown in the linked screenshot, should not change the attributes of the target dataset. But it does. The record size on the target dataset changes to 128k as soon as I start the replication.
late night edit: I decided to make it a local transfer by shutting down the old VM and giving the new one its controller, thereby directly connecting the legacy pool "sweet" directly to the 12.0-U2.1 VM. Even doing it as a local transfer, with similar settings, the recordsize attribute immediately changes to 128k after starting the replication task. Last option I can think of is a command line copy from one dataset to the other. Never done that with 44TiB before.
a couple of hours after taking my usual sleeping pill but pushing through because madness edit: The command line copy is going at a good clip, about as fast as SSH+NETCAT. The recordsize attribute of the new Media dataset has not changed; still 1MiB. Used
Code:
cp -R -ipv
A day and a half later: So, in the end my new dataset is 810GB smaller and I can't figure out why.
Thanks for all the help, everyone.
About 5 days later? Given that I wasn't able to use ZFS replication to copy my Media dataset, I used an external program to read both datasets, generate, and compare hash values for all the files. Two files had identical properties but different hash values. They were movies and a little bit-rot or whatever wouldn't likely have made any perceptible difference in playback, but I copied and checked those two files again and the hash values are a match. yay.
Best guess on the change in size of the dataset is almost certainly due to differences in compression, ashift misalignment, and other factors I've forgotten to consider or am blatantly unaware of.
Last edited:
