Data Migration

[Tony Self]

I am getting to grips with migrating my data from my Ext4 formatted drives, but before I go much further I would like to get some validation on my storage strategy before I get to a point of no return.

I have 8 x 3TB drives (5 WD Red and 3 Seagate Barracuda). At the moment I had managed to clear two of the drives which I had used to setup my first pool. But the trouble is this isn't a Z1 or Z2. Fortunately, I have found some spare space on a Lenovo IX4 NAS box, so at the moment I am copying data from two other drives of to this.

Once that is completed and I have two additional drives I plan to install those in my FreeNAS server, zap my initial ZFS pool and create a new one using 4 disks to create a Z2 pool.

I can then start copying data from the remaining 4 drives into datasets in the Z2 pool. Some guidance here please - Most of this data is for a Plex Media Server. I am best to create one Plex dataset or separate datasets for each Plex library (i.e. TV, Movies, Pictures, Music, etc)?

As I clear each Ext4 disk, I plan to move the disk into the FreeNAS server and add it to the Z2 pool. I will repeat this process until the remaining disks have been added to the Z2 pool.

Finally I will copy files that were transferred to the Lenovo IX4 to the FreeNAS box.

Does this strategy make sense or can you suggest a better course of action?

Thanks for any input you can give.

 

[nojohnny101]

As I clear each Ext4 disk, I plan to move the disk into the FreeNAS server and add it to the Z2 pool. I will repeat this process until the remaining disks have been added to the Z2 pool.

You can not do this. Once you create a vdev, you can not add/remove disks from the vdev (exception is with mirrors). You could stripe another vdev to the existing vdev which would expand the pool, but I don't think that is what you want.

There is no easy way to temporarily store a ton of data. How much data do you have? Some of the most common methods or stopgaps that I have seen on here are:
1) find a friend with massive storage space, use that.
2) buy a single very large external hard drive (have a backup!) - this option is becoming easier with the 8TB, 10TB, etc drives coming out.

For a very easy to understand guide explaining this very fundamental knowledge about FreeNAS and how it handles storage, please see this: Cyberjock's Noob Slideshow

Most of this data is for a Plex Media Server. I am best to create one Plex dataset or separate datasets for each Plex library (i.e. TV, Movies, Pictures, Music, etc)?

I don't think this particularly matters as far as function goes. FreeNAS and your jail doesn't care. I have all my plex data in one dataset but buried down 2-3 levels in specific folders for movies, tv, photos, etc. All that matters if that you link the storage properly when you are setting up the jail.

Good luck.

 

[Tony Self]

Have I miss read what I have seen in the ZFS Primer. I thought you could add additional disks to a pool once it had been created. Also, I thought Datasets were dynamic and could grow to whatever size was required. I understood that zvol was of fixed sized and couldn't be resized. What is a vdev and where does this come into play?

 

[Tony Self]

I will have a look at Cyberjock's Noob Slideshow

 

[Tony Self]

Right having read through the Noob Slideshow and getting a better understanding of how ZFS works, I am going to reorganise my strategy.

1. I will continue with freeing up two of my disks and transferring the data onto the IX4
2. Once I have four disks free, including the initial two I had install, I will zap the existing zdev and pool and create a new Z1 zdev.
3. That should give me enough room to copy the data from the remaining four disks on to the first zdev.
4. With the second set of four disks, I can create a second Z1 zdev to increase the pool size and give me room to copy the data from the IX4.
5. Once I have copied the data across from the IX4, I may consider using the four 3TB drives that contains to create a further Z1 zdev. Depending how things go with the first two.

Any comments appreciated.

 

[Kevin Horton]

I assume this data is relatively important, judging by the efforts you are taking to transfer it. If it is important, I recommend you drop any thought of using RAIDZ1, due to the risk of data loss if a second disk fails when the new drive is resilvering after a first disk fails. The drives work very hard during resilvering, and this apparently can trigger a disk that is near the end of its life to die.

If future data loss is not a concern, then press on with your current plan.

 

[Tony Self]

Thanks Kevin. The data is not critical. It is just my video library I have collected over the years. Not irrecoverable, it would just take a long time to put back together again. I must admit after reading Cyberjock's presentation, I am wavering towards Z2, it's just a matter of logistics trying to find enough temporary space to create the Z2 in the first place. It would make sense to use all 8 disks in one zdev.

Decisions, decisions....

 

[nojohnny101]

I am wavering towards Z2, it's just a matter of logistics trying to find enough temporary space to create the Z2 in the first place. It would make sense to use all 8 disks in one zdev.

It's a common problem people face on here. Reference my options above, those are the most common solutions that people come up with.

 

[SweetAndLow]

Thanks Kevin. The data is not critical. It is just my video library I have collected over the years. Not irrecoverable, it would just take a long time to put back together again. I must admit after reading Cyberjock's presentation, I am wavering towards Z2, it's just a matter of logistics trying to find enough temporary space to create the Z2 in the first place. It would make sense to use all 8 disks in one zdev.

Decisions, decisions....

A single 8tb external drive isn't that expensive and can probably be returned.

Sent from my Nexus 5X using Tapatalk