8 x 4tb drives, want to create 2 two mirrored Z2 vdevs

[M H]

Ok, so I must admit, it's been about 4 years since I set up my pools (because FreeNAS is so damn good) and I've gotten a little rusty. It's time for me to upgrade and expand my storage. I was hoping that I could just run my plan of action past the gurus here to make sure I'm attacking this correctly.

I want to create two mirrored Z2 vdevs, but I want to create them in stages.

I want to create my first Z2 volume of 4 x 4TB drives so that I can migrate my existing data off of my current pool. I then want to add another 4 x 4TB Z2 vdev that will mirror the first. What's the best way of doing this? Can I create the 4 x 4TB Z2 vdev, move my data onto it and then add the 2nd 4 x 4TB vdev? I have to do this because 3 of my 8 4TB drives are currently in use.

I hope I explained that correctly and I'm pretty sure what to do, but couldn't find any graphical examples of doing so in the GUI. Thanks again, you guys rock!

 

[Stux]

a mirror of RaidZ2 vdevs is not a valid ZFS configuration.

What are you trying to accomplish?

Why not just an 8-way RaidZ2 vdev? (yes, I know that you *think* you have a chicken/egg problem... but you don't ;))

 

[M H]

a mirror of RaidZ2 vdevs is not a valid ZFS configuration.

What are you trying to accomplish?

Why not just an 8-way RaidZ2 vdev? (yes, I know that you *think* you have a chicken/egg problem... but you don't ;))

Thanks for the quick reply. I want the mirrors for performance because we have TONs of small random writes like thousands of 500K files and I want at least 3 drive parity (or even 4 drive under some conditions) because the data is extremely important. As you can see, in our current setup, we have a 3 way mirror setup.

 

[MrToddsFriends]

Thanks for the quick reply. I want the mirrors for performance because we have TONs of small random writes like thousands of 500K files and I want at least 3 drive parity (or even 4 drive under some conditions) because the data is extremely important. As you can see, in our current setup, we have a 3 way mirror setup.

In ZFS if several vdevs make up a pool it is striped over them (not mirrored).

For example you could add a second 3-way mirror vdev to the existing one. In this way each vdev had 2 parity disks.

Next example: You could convert the existing 3-way mirror into a 4-way mirror and add a second 4-way mirror vdev to the (then) existing one. In this way each vdev had 3 parity disks.

AFAICT creating 3-way and 4-way mirror vdevs as well as converting a 3-way mirror into a 4-way mirror needs to be done using the console, but I'm not sure about that.

 

[Stux]

Also, adding more mirrors to a vdev doesn’t increase write speed. Adding more mirrored vdevs does.

It sounds like you may be after 3 striped triple mirrors. Gets you double redundancy on each vdev and 3 vdevs for writing.

Or two quad mirrors.

 

[M H]

Got it, thank you. I did have to create the current 3 way mirror via console, including swap etc., so that doesn't scare me much. I just don't want to screw up my topology. So, if I create 4 mirrored vdevs which will be striped when combined, that should give me 16TB available with maximum performance?

 

[M H]

Also, if I just make one Z3 vdev of all 8 disks, will I get a huge performance hit? I may be able to swing that by splitting off two of my mirrored drives and making the vdev from the start.

 

[Zredwire]

Also, if I just make one Z3 vdev of all 8 disks, will I get a huge performance hit? I may be able to swing that by splitting off two of my mirrored drives and making the vdev from the start.

Z3 has a huge write penalty. Were the write penalty for mirrors is 2x (or I guess 3x for a three way mirror) the penalty for Z3 is 8x. If you have a lot of writes, Z3 is not what you want.

 

[M H]

Z3 has a huge write penalty. Were the write penalty for mirrors is 2x (or I guess 3x for a three way mirror) the penalty for Z3 is 8x. If you have a lot of writes, Z3 is not what you want.

Excellent, thank you. Last question, I would like to keep either 16 or 20TB of usable space. Of those two possibilities, which topology would give me the best performance for TONS of small random reads and writes?

 

[Stux]

The best performance for small random reads and writes is a collection of mirrors.

Perhaps you could use mirrors and a replication for backup?

 

[M H]

The best performance for small random reads and writes is a collection of mirrors.

Perhaps you could use mirrors and a replication for backup?

Thanks. I have an identical machine off site that is already set up for replication so that's not a problem. It won't have as many drives, but will have at least 2 disk parity.

When you say mirrors, what topology do mean? 4TB x 2 mirrored and then striped across 4 sets of them?

 

[Stux]

Yes. Assuming you’re happy with single parity per vdev.

Alternate would be to go to 9 disks and have 3x 3-way mirrors.

 

[M H]

So just to confirm, creating a 4x4TB RaidZ2 pool and then expanding it with another 4x4TB RaidZ2 vdev is not a valid configuration as mentioned in the earlier posts? I know that it doesn't have mirrors so performance will suffer but it seems to suite my needs OK. Not ideal, but acceptable.

And would this topology perform better than 8x4TB RaidZ2?

 

[Stux]

Yes. That is a valid configuration.

Random performance will be better (more vdevs) streaming performance will be worse (less data disks)

And it results in 4 disks used for party.

So your choice.

 

[M H]

Yes. That is a valid configuration.

Random performance will be better (more vdevs) streaming performance will be worse (less data disks)

And it results in 4 disks used for party.

So your choice.

I think this is the way I'm going to go. It will also give me a chance to move my data over and free up the additional drives. I really can't thank you guys enough for the help and the quick refresher. You guys rock! Have a safe and happy Cinco de mayo.