12 drives RAID-Z config

[m0t0rh3ad]

Hello, right now I have Raid-Z2 with 8 drives (HGST 10Tb SAS) on Supermciro X10SDV-4C-7TP4F (Xeon D-1518) 32Gb Ram and LSI 3008. My server's purpose for home usage: Plex (90% Full HD and 10% 4K HDR), torrents, and video surveillance (4 cameras ZoneMinder).

I want to expand my storage to 12 Drives. Is it optimal to add 4 additional drives to my Raid-Z2 pool or not? Any other configurations?

 

[Alecmascot]

2 X 6 RaidZ2 VDEVS is probably best

 

[joeschmuck]

I want to expand my storage to 12 Drives. Is it optimal to add 4 additional drives to my Raid-Z2 pool or not?

Realize that you just cannot add four more drives to your RAIDZ2 pool to make it a 12 drive RAIDZ2 pool, you would need to destroy your 8 drive pool then recreate the pool as a 12 drive pool. 12 drives is a very wide pool (hence the recommendation of 2X6 RaidZ2) but for what you are using it for, shouldn't be an issue. The problem for you is if you have your current 8 drive pool full, where could you store all that data while you destroy and recreate your pool.

A crappy option is to just build another separate pool of four drives as RAIDZ2 and just share that data with Plex as well. It's not a great option and you waste two drives for redundancy but you would gain more storage.

One other thing to mention, since you are using these large 10TB drives, it takes a long time to resilver these so I'd recommend that you perform a SMART Long test on each drive, ensure all of them are working well and have no errors, I'd also recommend you build a RAIDZ3 vdev/pool, but that is your call since it does take one more drive away for redundancy.

 

[m0t0rh3ad]

Realize that you just cannot add four more drives to your RAIDZ2 pool to make it a 12 drive RAIDZ2 pool, you would need to destroy your 8 drive pool then recreate the pool as a 12 drive pool. 12 drives is a very wide pool (hence the recommendation of 2X6 RaidZ2) but for what you are using it for, shouldn't be an issue. The problem for you is if you have your current 8 drive pool full, where could you store all that data while you destroy and recreate your pool.

A crappy option is to just build another separate pool of four drives as RAIDZ2 and just share that data with Plex as well. It's not a great option and you waste two drives for redundancy but you would gain more storage.

One other thing to mention, since you are using these large 10TB drives, it takes a long time to resilver these so I'd recommend that you perform a SMART Long test on each drive, ensure all of them are working well and have no errors, I'd also recommend you build a RAIDZ3 vdev/pool, but that is your call since it does take one more drive away for redundancy.

Thanks for the answer. Right now Used Space - 68%. Is it possible to "remove" 1 or 2 drives (I have 13x10Tb Drives) and move to RaidZ2 with 6 drives and then create additional RaidZ2 with 6 disks (4+2)?

 

[blanchet]

If you recreate your pool from scratch, you may be interested by this article

ZFS: You should use mirror vdevs, not RAIDZ. – JRS Systems: the blog

It explains that, for a home user, a mirror vdevs may be a good option.
At the first sight, it seems a waste of space, but at the end you win because it is easier to expand.

 

[Heracles]

Thanks for the answer. Right now Used Space - 68%. Is it possible to "remove" 1 or 2 drives (I have 13x10Tb Drives) and move to RaidZ2 with 6 drives and then create additional RaidZ2 with 6 disks (4+2)?

Hi,

It is possible but not safe...

1-Remove 2 drives from your Raid-Z2

That will drop you to no redundancy. A single failure and you loose it all. High risk and not recommended but possible.

2-Create a new 6 drives Raid-Z2 vDev
3-Create a new pool from that single vDev
4-Migrate your data from your actual pool to that new one

Only from now you are back in safety. Of course, all of your data must fit in this single vDev

5-Destroy your old pool
6-Create a new 6 drives Raid-Z2 vDev
7-Add that vDev to your new pool

Here, almost all of your data will be in the first vDev and the second one will be almost empty. ZFS will rebalance this over time but it is not ideal. The best is to migrate your actual data out of the pool and create a new and complete pool before moving everything back.

 

[joeschmuck]

@Heracles has provided you sound advice on how to get from point A to point B and also cautioned you that if you have a failure, your data is all gone. I would recommend that you obtain one extra 10TB drive and pull only one drive from your RAIDZ2 vdev, this way you have at least one drive for a safety net. Or if you could just buy two more drives and after the rebuild process you would have two spare drives to use when you have a drive failure. It's not the cheapest way to do things but it's far safer.