Need help with ZFS Layout

[lukazy88]

Hi,
I want to setup a truenas system as a backup (offsite) vault.
I need 100 TB of useable space.
What layout would be the best for my use case?
I though about 16 TB drives (very good price per TB).
I could do:

• 2 vdevs of Raid-Z 2 using 6 drives each giving me ~ 123 TB of useable space
• one 9 drive Raid-Z 2 giving me 105 TB of space

Smaller vdevs are better for expanding in the future I heard so in the second option I would need to get another 9 drive vdev to add more to the pool. So maybe not a smart move to "save 3 drives" initially but have way more cost to expand later?

Does going even smaller on vdevs like 3 vdevs of 5 drives each make sense? "Only" 3 drives more than option 1 but each new vdev now is only 5 and not 6 drives.

I like going Raid-Z 2 because I'm still safe when replacing/resilvering a faulty drive.
That makes sense right?

Or is there a layout I'm missing completely?

regards

 

[Davvo]

With those kinds of drives RAIDZ2 is the minimum I would suggest.

You have to subtract 15-20% of the total space since you want the pool to remain below the 80-85% mark in order to avoid a serious performance degrade.

A single RAIDZ3 VDEV composed of 12 (16TB) drives (the maximum reccomended to put in a single VDEV) would give you around 107 TB of available space and great resilience.

If you haven't already, please read the following resources.

ZFS Storage Pool Layout

This amazing document, created by iXsystems in February 2022 as a "White Paper", cleanly explains how to qualify pool performance touching briefly on how ZFS stores data and presents the advantages, performance and disadvantages of each pool...

www.truenas.com

Assessing the Potential for Data Loss

This guide was written to be read from top to bottom without jumps, with the intent of spreading awareness to both new and experienced users; the author of this document assumes the understanding of the concepts explained in the following...

www.truenas.com

 

[lukazy88]

I was using thos website to calculate my Raid-Z: https://wintelguy.com/zfs-calc.pl

I read through the 2 links you provided and understood a little (there is a lot of in depth stuff which I just barely get though)

So I guess the layout of 2 vdevs with like 6 drives would be good because that one can be expanded more easily.

What I need help with is the 20% free space.
When I only use it to do over the internet backups to an offsite location is that performance hit even noticeable? When using it for daily work with multiple users I get it but for backup maybe not required?

regards

 

[ChrisRJ]

No offense meant, but assuming that this is in a commercial context, to understand only a little bit carries a huge risk. Please give yourself a couple of weeks time to truly understand all the relevant details. ZFS is different, sometimes very much so, from other storage solutions. If there is not enough time, please consider hiring a professional with the required knowledge to speed things up.

 

[Davvo]

I was using thos website to calculate my Raid-Z: https://wintelguy.com/zfs-calc.pl

As I did: according to it, two RAIDZ2 VDEVs composed by six 16TB drives each will give you a low space efficency and below 100TB of usable space.

Show : Images

So I guess the layout of 2 vdevs with like 6 drives would be good because that one can be expanded more easily.

In this case, it's basically the only benefit over the single VDEV config: going two VDEVs means less space and less reliability; the IOPS performance increase is not of concern since this will be a backup pool.

What I need help with is the 20% free space.
When I only use it to do over the internet backups to an offsite location is that performance hit even noticeable? When using it for daily work with multiple users I get it but for backup maybe not required?

It's totally required, ZFS is a COW (Copy on Write). You could go beyond 80 and stop at 85, but you really don't want to go beyond that.

(there is a lot of in depth stuff which I just barely get though)

I agree with @ChrisRJ, take your time to digest stuff. Reading the ZFS Introduction is a good place to start.

Introduction to ZFS

This is a short introduction to ZFS. It is really only intended to convey the bare minimum knowledge needed to start diving into ZFS and is in no way meant to cut Michael W. Lucas' and Allan Jude's book income. It is a bit of a spiritual...

www.truenas.com