Sanity check on 18 drive setup

[Jr922]

When extending a pool should the new vdevs match exactly the existing vdevs? I keep seeing somewhat conflicting answers to this.
I've been using 6 disk raidz2 vdevs and I setup one pool per vdev for a max of 4 pools per 24 bay case.
I want to move away from this and create a single pool for all the vdevs.
I liked that a vdev could die and I would only lose that vdevs data but balancing the free space was getting silly as they filled up and now that I'm on 10Gbit I am bottle necking on a 6 disk z2.
I want to keep the same 6 disk z2 vdevs since I like that amount of redundancy/it fits the case nicely/buying 8 or 12 disks at a time is too $$.
I have 12x 8tb WD RED and 6x 6tb HGST NAS.
I would like to make 3x 6 disk z2 vdevs out of that (6x 8tb, 6x 8tb, 6x 6tb) and put that all under one pool. Is this a valid setup or is mixing the 6tb drives in no good?
In the near future I want to extend this with another 6 disk z2 for a total of 24 drives, 4 vdevs, 1 pool.

Secondary question, when extending a pool with a new vdev and the pool already has lots of existing data , does it make sense to copy everything to backup and then rewrite it to the pool to get all the data striped evenly across the drives?

 

[Spearfoot]

When extending a pool should the new vdevs match exactly the existing vdevs? I keep seeing somewhat conflicting answers to this.
I've been using 6 disk raidz2 vdevs and I setup one pool per vdev for a max of 4 pools per 24 bay case.
I want to move away from this and create a single pool for all the vdevs.
I liked that a vdev could die and I would only lose that vdevs data but balancing the free space was getting silly as they filled up and now that I'm on 10Gbit I am bottle necking on a 6 disk z2.
I want to keep the same 6 disk z2 vdevs since I like that amount of redundancy/it fits the case nicely/buying 8 or 12 disks at a time is too $$.
I have 12x 8tb WD RED and 6x 6tb HGST NAS.
I would like to make 3x 6 disk z2 vdevs out of that (6x 8tb, 6x 8tb, 6x 6tb) and put that all under one pool. Is this a valid setup or is mixing the 6tb drives in no good?
In the near future I want to extend this with another 6 disk z2 for a total of 24 drives, 4 vdevs, 1 pool.

Secondary question, when extending a pool with a new vdev and the pool already has lots of existing data , does it make sense to copy everything to backup and then rewrite it to the pool to get all the data striped evenly across the drives?

Yes, you can 'mix-and-match' vdevs made up of different size disks, so your proposed pool configuration of 3 x RAIDZ2 vdevs (6x8TB, 6x8TB, and 6x6TB) will work just fine. You can also extend this pool with an additional RAIDZ2 vdev later, as you described above, using whatever size disks you like in the new vdev.

Caveat: you don't want to create a RAIDZ2 vdev using disks of varying size. Certainly you can do it, but it's inefficient because the RAIDZ2 vdev capacity will be limited to the size of the smallest disk.

I suppose you could balance the data across the 3 vdevs in your pool by making a backup, creating the new pool, and then restoring from backup... but I wouldn't worry about doing that. If you have a lot of data it could take a long time to do...

 

[Chris Moore]

does it make sense to copy everything to backup and then rewrite it to the pool to get all the data striped evenly across the drives?

The only potential benefit of that is that you might improve performance because the data might be spread across more vdevs. I have done that on systems where I work. It made a difference to increase from four vdevs to ten vdevs because of read and write paralleling, but if you are not having performance problems, it might not be worth the trouble.

 

[Jr922]

Ok, sounds good, I'll just have to test and see if I can saturate a good bit of my 10Gbit if not ill rewrite it to the pool after.
I'm getting 2-400 Mbps right now from one of the 6 disk z2 pool to a similar pool on another sever so thats not to shabby but its definitely the drives bottlenecking so far.
I can setup a pool with 2 of those vdevs before ill need to start copying data over to stay backed up. Then I can add the third one and see, probably should be around atleast 600 Mbps just for a 2 vdev pool

 

[CraigD]

I liked that a vdev could die and I would only lose that vdevs data

If a vdev dies you lose the whole pool

Have Fun

 

[Jr922]

If a vdev dies you lose the whole pool

Have Fun

Yeah I understand that, maybe I explained it to you poorly.
I originally had one pool per vdev so if a vdev failed it would take out the pool but since that vdev was the only thing in the pool the other vdevs each on their own pools werent affected

I saw that as an advantage in terms of how safe the data was, Now I am essentially trading that in for a peformance boost so I can utilize more of the networks potential and the ability to not have to shuffle shit around as drives fill up.

I still think that a pool of 3x 6 disk z2 vdevs is pretty safe, and im mirroring it to a similar server with an identical pool setup.