best pool config for 6 drives?

[jjb2018]

Hi - I'm thinking of rebuilding my zfs pool to increase capacity. I currently have 6 x 2TB drives in a 3 x 2 drive mirror config. I'm guessing the only sensible options for changing this is 1 x 6 drive raidz2 or 2 x 3 drive raidz1 (ignore 1 x 6 drive raidz1 option). Am i right in thinking i would get the same capacity from the two raidz options? And would there be a significant difference in performance between the two? My machine is used for storage of static data (photos, videos, documents etc ...) but also for live VM images. This is actually an Arch Linux machine running the VMs with KVM.

 

[HoneyBadger]

Live VMs on RAIDZ-anything isn't exactly a recipe for performance - you've actually got the best performance setup with the 3x2-way mirror setup above.

Personally, I don't consider single-parity RAID to be safe after seeing too many people get bitten by it - I would advise the 6-drive Z2 for that reason alone. Capacity will be roughly equal to the 2x3-drive Z1 but you'll have two-drive failure tolerance before your data is at risk.

Regardless of pool configuration, I'd strongly suggest two separate datasets - one with recordsize=128K for your photos/videos/documents, and one with recordsize=16K for your KVM images; also consider manually tuning the internal block size of your QCOW2 images to match this, unless you're presenting raw ZVOLs to each VM.

 

[jjb2018]

Thanks for the tips. Before I setup my pool, in it's current 3 x 2 mirror config, I did read somewhere that this would give the best read throughput and IOPS, but I have since read that with the RAIDZ1 / RAIDZ2 config the reads will be a little slower but writes will be a little faster. I do not have a SLOG in my system (can't afford a decent reliable device!). At the moment all of my VM images are RAW images stored in separate datasets for each VM. This is so I can tune and snapshot each VM independently via dataset properties. Would your recommendation of recordsize=16K still be necessary for RAW images? Or is that specifically for QCOW2 images?

 

[HoneyBadger]

Large sequential writes to a RAIDZ2 would theoretically be faster (as you have four data disks versus 3 in your stripe of mirrors) but most VM I/O is both small and random, so real-world performance of the VMs would likely degrade.

As far as your images, if they are .RAW files on a dataset, as opposed to each VM being given a ZVOL block device, yes the recordsize tuning will still probably be beneficial.

 

[jjb2018]

Thanks. I'll experiment a little with 16K recordsize.

 

[Stux]

What about a 6-way raidz2 and then a mirrored pair of SSDs for your VMs?

Or even a single ssd for the VMs which is replicated to the raidz2