Storage migration strategy from Synology to ZFS

Status
Not open for further replies.

massib80

Cadet
Joined
Aug 15, 2017
Messages
3
Hello all,
tomorrow I'll get an HP Microserver Gen8 wiht 4x1TB HDD to which I would like to migrate 1.99TB of data from a Synology with 2x3TB HDD (In SHR "RAID1")

My plan is:

Create ZFS Pool as follow
VDEV Mirror 1
> disk1 -1TB + disk2-1TB
VDEV Mirror 2
> disk3 - 1TB + disk 4-1TB

Migrate 1.99 TB of data (I can reduce migration by scarifying some data if necessary) to the ZFS POOL.

Then, the plan is to take 1 of the 3TB HDD from the Synology and replace one disk in the pool:
VDEV Mirror 1
> - disk1 -3TB + disk2-1TB
VDEV Mirror 2
> disk3 - 1TB + disk 4-1TB

Once resilvering is done I'll take the second 3 TB HDD and change the pool as follow:
VDEV Mirror 1
> - disk1 -3TB + disk2-3TB
VDEV Mirror 2
> - disk3 - 1TB + disk 4-1TB

After resilvering I will exand the pool and I should then end up with a total storage of about 4TB with the option to increase the Pool later one by replacing the disks in the VDEV Mirror 2

I'm choosing Mirroring vs RAIDZ1 as this will be used mainly as a streaming server so I'd like fast reads and as well to make it easier to increase storage in the future. Of this migration strategy I like the fact that in case things go wrong I will still have my data on one of the 3TB drive in the Synology so seems pretty safe to me... but I'd like to know from the experts :D
 

saikee

Explorer
Joined
Feb 7, 2017
Messages
77
There could be a problem of fitting 1.99TB data into a 2x2x1TB Mirror raid since zfs filing system recommends 80% used storage and part of the storage on each disk needs a reserved area for swap.

The vdev calculator indicates 4x1TB mirror has only a usable space of 1.574TB.

I would probably do a 3x1TB strip giving me about 2.36 usable space for ta new zfs pool, copy all the 1.9TB data onto it, verify all the data accurately transferred, use the spare 2x3TB from the Synology to create a new mirror zpool to run parallel with the 3x1TB stripe zpool, transfer finally all the data to the 2x3TB mirror. You can then decide what to do with the four 1TB disks.
 
Last edited:

Stux

MVP
Joined
Jun 2, 2016
Messages
4,367
I suspect the microserver only has 4 bays.

Another option

Do the triple stripe. Then once your data is on that stripe, scrub it. You still have the synology.

Now make a new 3TB pool with a single drive. Replicate the stripe to the 3TB pool.

You have three copies of your data.

detach the stripe. Remove a drive and put the other 3TB in. Attach it to the other 3TB and let it resilver. You still have two copies.

Now wipe the other two disks and add them as a mirror to the 3TB mirror.

Put the two remaining 1TB drives in your sinology and use it as a backup ;)

A benefit of this approach is you end up with a TB free in both vdevs. Should make for more Optimal further filling.
 

massib80

Cadet
Joined
Aug 15, 2017
Messages
3
Thanks guys, for now I did a RAIDZ1 with 3x1TB drives giving me 1.8TB... I had few data that I could drop and some duplicate backups...

I've preferred the RAIDZ1 just in case I have an HDD failure and then I will have wasted all the rsynching time which I'm going through at the moment...

@Stux when you say:
>> detach the stripe. Remove a drive and put the other 3TB in. Attach it to the other 3TB and let it resilver. You still have two copies.

You mean that I can create the pool with one disk and when I've attached the second it will be a mirror?
At the moment I'm not on FreeNAS, I'm trying to get more knowledgeable on ZFS using a Debian installation with ZFS On Linux.

I create a new pool "solo" with 1 disk (imagine UID: WD00A)

I replicate and then I'm adding the 2 disk (imagine UID: WD00B) like this:
zfs attach solo WD00A WD00B

this will effectively make the vdev in the "solo" pool mirrored.

Am I correct?
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,367
Thanks guys, for now I did a RAIDZ1 with 3x1TB drives giving me 1.8TB... I had few data that I could drop and some duplicate backups...

I've preferred the RAIDZ1 just in case I have an HDD failure and then I will have wasted all the rsynching time which I'm going through at the moment...

@Stux when you say:
>> detach the stripe. Remove a drive and put the other 3TB in. Attach it to the other 3TB and let it resilver. You still have two copies.

You mean that I can create the pool with one disk and when I've attached the second it will be a mirror?
At the moment I'm not on FreeNAS, I'm trying to get more knowledgeable on ZFS using a Debian installation with ZFS On Linux.

I create a new pool "solo" with 1 disk (imagine UID: WD00A)

I replicate and then I'm adding the 2 disk (imagine UID: WD00B) like this:
zfs attach solo WD00A WD00B

this will effectively make the vdev in the "solo" pool mirrored.

Am I correct?

Yes. The trick is you want to format the disk so that its formatted the same as if FreeNAS had done it. This attaching a disk to a stripe to make it a mirror functionality really should be in FreeNAS, but it is possible from the command line.

Here is how to do it:
https://forums.freenas.org/index.php?threads/create-zfs-mirror-by-adding-a-drive.14880/#post-81348
 

massib80

Cadet
Joined
Aug 15, 2017
Messages
3
Thanks, I'm chugging along with rsync and then I will start to work on the rest, I will report here my experience in case will be useful for someone else, hopefully all will work out :D
 
Status
Not open for further replies.
Top