Question about replicating sync always datasets

[MikeyG]

I have a primary NAS and a backup NAS. The primary has a RAIDZ2 pool for media, and a mirror pool with 4X SSDs in it that's used to store a zvol for iSCSI set to sync always.

When replication runs from the primary NAS for the media datasets on RAIDZ2, i get about 250MBps via SSH to the backup NAS (10gbe). Backup NAS has 8X 8TB RAIDZ2 pool and can usually sustain at least 600MBps of write activity so this seems great.

However, when replication runs from the primary NAS for the zvol dataset, it goes at about 20-30MBps. gstat on the target (the backup NAS) shows disks are very busy, so clearly there are disk limitations when running this task. I also notice that when I view properties on the backup for the dataset, sync is set to always, which given that the sources is set that way makes sense.

My question is, when replication is run from a zvol set to sync always, is it written to the target with the same write penalty (due to the double writes without a SLOG) that would normally be seen on a zvol or dataset set to sync always? I would assume it would be common to run a fast SSD mirror pool for iSCSI set to sync always, and then back that up to slower disks. Is everyone in a similar situation just taking a massive write penalty since the speed of a backup isn't super important?

Also, is data for replication read and written sequentially since it's reading and replicating storage blocks, and not actual files? If so, and it's not double writing on the target (sorry if I'm saying that wrong), why would the zvol be so much slower to replicate?

Thanks for your insights!

 

[sretalla]

Depends what you're using the backup target as... if it's a hot-spare (to be substituted for the production one in the case of some major failure) or if it's just backup file storage.

If the intention is just part of your 1-2-3 backup story, then let it be slow as long as it completes in your backup window.