Smallest upgrade increment.....2 disk vdevs?

[jstore]

So, I am not "new" to TrueNas but I am doing my first capacity expansion and you guessed it, I just learned that I have to add equal amounts of disks to a pool. Since I created an 8 disk vdev and only have room for 4 more drives and only bought 2 new drives....here is my thought.

All data is replicated so I am okay if I have to blow away the data (while that's super frustrating, I probably didn't read carefully enough a year a ago when I set it up)

If I create 5 vdevs in the pool each with 2 disks is that okay? (best practice even?) That way I can add my two disks today and give myself an option of adding 2 more down the road?

Am I missing a new constraint that I will regret a year from now?

 

[Davvo]

Introduction to ZFS

This is a short introduction to ZFS. It is really only intended to convey the bare minimum knowledge needed to start diving into ZFS and is in no way meant to cut Michael W. Lucas' and Allan Jude's book income. It is a bit of a spiritual...

www.truenas.com

ZFS Storage Pool Layout

This resource was originally created by user: @Davvo on the TrueNAS Community Forums Archive. https://www.truenas.com/community/resources/zfs-storage-pool-layout.201/download [1] This amazing document, created by iXsystems in February 2022 as a “White Paper”, cleanly explains how to qualify...

www.truenas.com

If you want to expand your existing pool, you can only do so with an equally-wide VDEV (since I doubt you have an 8-way mirror).
If you want to change your layout you have to destroy the pool and recreate it as something else.
You could do two 6 wide vdevs in RAIDZ2 for best space efficency and good redundancy, but you would need to buy two more drives; or you could create five 2-way mirrors.

First option is safer, if you choose the second option I would strongly suggest using at least one hostpare.

 

[Patrick M. Hausen]

Btw ... equally wide vdevs is by no means mandatory. It is recommended for redundancy and performance considerations. If you add a two disk mirror to a pool consisting of a RAlDZ2 vdev, you worsen your resiliency to disk failure. Also mixed topology pools might exhibit unexpected performance patterns, although probably nobody can tell in advance how exactly. Hence the lots of "might" etc.

Tldr; if the pool in question is a "family data dump" it might be ok to add just another mirror. If a complete redesign with a new pool is planned, anyway, even more so. It all depends on your use case.

 

[jstore]

Thank you. Got it. I wasn't thinking about space and spares. A 2 disk vdev can only be mirrored so I'd be wasting a lot of space and taking a bigger risk. So now thinking about capacity and risk here are the options I see.

• 2 - 6 drive vdevs using RaidZ2 would give me ~40TB per vdev (using easy math with 10TB drives) total Pool would be ~80TB. Low Risk, Good Space.
• 3 - 4 drive vdevs using RaidZ1 would give me ~30TB per vdev (making the argument that a smaller vdev size reduces the risk so I could use RasiZ1, I just have to replace the drive quickly) Total Pool size would be ~90TB. Higher risk, better space.

It's home system so I am inclined to go 3 - 4 drive RAIDZ1 vdevs, if this were for something more important than personal stuff, I'd got RAIDZ2.

 

[samarium]

If you want to use Z1, have a cold spare on hand, and be prepared to use it. If you can figure out a way to attach it, while leaving a partially broken drive operational until after the replacement you will be have less chance of data loss as the partial drive can add some parity while it operates, it can't help at all if you remove it before replacement has completed.

 

[jstore]

Gotcha. Thanks. I have to buy 2 drives, might as well make it 3. Did I forget to mention that I have a second machine sitting next to it and I replicate the important data? I split the workload between the two systems but replicate the data but since they both use the same drives it would be silly not to have a spare on hand.

 

[Davvo]

 

[jstore]

Hard to argue with math. I like that visual. In my case with the relicataed system it's 2 seperate - 4 dev:1 Parity RaidZ1 Pools, which is not quite dual parity but is tolerance for two drive failures. I wonder if having it in a seperate machine increases or decreases the odds? (took stats to long ago)

 

[Davvo]

The curves relative to each other stay the same, the numbers vary.
Assuming the same vdev type and size, having two pools of a single vdevs is safer than a single pool of two vdevs.

 

[samarium]

https://jro.io/r2c2/ if you want to run the model yourself