What is "2 x RAIDZ2"?

[allanonmage]

I had a TrueNAS scale server with RAIDZ2 with 6x WD 14TB drives, giving me a usable 50.x TB of storage. I got an HBA from Art of the Server, some cables, and 6 more drives (Toshibas) and doubled the drive count to 12, all of them are 14TB enterprise drives. 6 are on the motherboard SATA ports, and 6 on the HBA. I was mucking about the interface and saw that the existing pool could be expanded, and since I only have test data on the machine right now anyways, figured I'd give it a shot. I added the drives to the pool, and wound up with double the capacity. I was anticipating more than that since I figured 2 drive redundancy was good enough for now. It seems that I'm missing 2 drives worth of capacity, leaving me at 100.x TB usable space (I expected anotehr 24 TB or so of usable space). I think what happened is that I wound up with a 2nd VDEV and each VDEV is a RAIDZ2, so now I have 4 drives of redundancy. The system shows it as "2 x RAIDZ2".

In Devices, I see two different VDEVs, each with 6 disks. I split the WD and Toshiba drives across both the motherboard SATA and HBA ports, thinking that since the drives are different, that should split my odds across two controllers against issues later down the line. I am pleased to report that TrueNAS DGAF that I moved the drives from one controller to another, which I had heard is what it's supposed to do. However, the VDEVs are 6x WD (the original) and 6x Toshiba (the new ones).

I did buy one of the Mastering ZFS books on Amazon, but the osmosis technique is not working, and squirrels showed up to steal my attention.

 

[chuck32]

It seems that I'm missing 2 drives worth of capacity, leaving me at 100.x TB usable space (I expected anotehr 24 TB or so of usable space). I think what happened is that I wound up with a 2nd VDEV and each VDEV is a RAIDZ2, so now I have 4 drives of redundancy. The system shows it as "2 x RAIDZ2".

You cannot currently expand a raidzx system, i.e. your 6 wide RAIDZ2 will always stay a 6 wide RAIDZ2, you cannot make it 12 wide RAIDZ2.
If you want that, you would need to destroy your pools, create a 12 wide RAIDZ2 and upload your data again from backup.

 

[NugentS]

Output of "zpool status" please

 

[Davvo]

I had a TrueNAS scale server with RAIDZ2 with 2x WD 14TB drives

Not possible, with two drives you can only have a two-way mirror VDEV: what you mean it's maybe 6x WD 14TB drives in RAIDZ2 (which means a single VDEV composed of 6 drives in RAIDZ2); as you correctly guessed 2x RAIDZ2 VDEVs means two different VDEVs, @chuck32 gave you a good explaination.

Please read the following resources:

Introduction to ZFS

Ericloewe submitted a new resource: Introduction to ZFS - A short introduction to ZFS, oriented towards FreeNAS users This is a short introduction to ZFS. It is really only intended to convey the bare minimum knowledge needed to start diving into ZFS and is in no way meant to cut Michael W...

www.truenas.com

ZFS Storage Pool Layout

This amazing document, created by iXsystems in February 2022 as a "White Paper", cleanly explains how to qualify pool performance touching briefly on how ZFS stores data and presents the advantages, performance and disadvantages of each pool...

www.truenas.com

 

[allanonmage]

Not possible, with two drives you can only have a two-way mirror VDEV: what you mean it's maybe 6x WD 14TB drives in RAIDZ2 (which means a single VDEV composed of 6 drives in RAIDZ2); as you correctly guessed 2x RAIDZ2 VDEVs means two different VDEVs, @chuck32 gave you a good explaination.

Please read the following resources:

Introduction to ZFS

Ericloewe submitted a new resource: Introduction to ZFS - A short introduction to ZFS, oriented towards FreeNAS users This is a short introduction to ZFS. It is really only intended to convey the bare minimum knowledge needed to start diving into ZFS and is in no way meant to cut Michael W...

www.truenas.com

ZFS Storage Pool Layout

This amazing document, created by iXsystems in February 2022 as a "White Paper", cleanly explains how to qualify pool performance touching briefly on how ZFS stores data and presents the advantages, performance and disadvantages of each pool...

www.truenas.com

Yes, 2x was a typo and I meant 6x.

I had always thought that there was no expanding the storage, but until yesterday was never really clear on VDEVs vs pools. I found an Ars Technica article with a picture and it clicked instantly. Now I understand why there are so many posts talking about how to architect your storage.

I will look into how to architect it a good way, but I'm leaning towards 1 VDEV in RAIDZ2

 

[Davvo]

I will look into how to architect it a good way, but I'm leaning towards 1 VDEV in RAIDZ2

Just keep in mind that the maximum reccomended VDEV size is 12 drives.

 

[allanonmage]

Yeah I saw that while looking around yesterday.