Pool build performance question..

[z2Dylan]

I currently have 5 10TB WD Reds in Z2 and looking to rebuild my pool soon with performance being not very important, but certainly a consideration. My understanding is you can (roughly) subtract 2 disks from your pool (Z2) and multiply the rest of the disks by their read/write speeds to get a estimate of your expected pool speed. I'm considering 2 options: 2 vdevs (4 drives each) in Z2 into a single pool or a single 6 disk Z2 vdev. with regards to the 2 vdev pool... I know IOPS will double, but am I to assume that I'll also get more read/write speed compared to my current 5 drive Z2 and roughly the same read/write as the 6 disk single vdev? Thank you for any advice/help!

 

[sretalla]

am I to assume that I'll also get more read/write speed compared to my current 5 drive Z2 and roughly the same read/write as the 6 disk single vdev?

Speed isn't a single measure...

IOPS are one factor, throughput is the other.

If you're copying large files, few IOPS are needed (even fewer if you've used a 1m recordsize on the dataset) to move the data and RAIDZ type VDEVs can hum along at relatively impressive "speeds" due to the number of spindles (specifics of the system will govern some aspects, but mostly will be at some multiple of the individual disk read/write speeds x something like 3-5 of the disks in most layouts).

If you're working with lots of small files or doing block storage (ZVOLs), then you're going to hit the bottleneck of IOPS as metadata becomes a much higher proportion of the data in total and the number of operations sent to the pool increases, but the number of bytes remains relatively low.

That will mean that the same pool that could transfer 300MB/s for large files a minute ago suddenly seems only capable of 20MB/s now with smaller files.

 

[Davvo]

You might find the following resources useful.

ZFS Storage Pool Layout

This amazing document, created by iXsystems in February 2022 as a "White Paper", cleanly explains how to qualify pool performance touching briefly on how ZFS stores data and presents the advantages, performance and disadvantages of each pool...

www.truenas.com

Resource - The path to success for block storage

ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle bits modified after creation. This is optimal for RAIDZ. It's what most...

www.truenas.com