RZ2: Do special vdevs, l2arc, metadata, etc. increase performance?

[winstontj]

I have 8x 8tb drives in a fractal 804 with a x11spm-tpf mobo. Not really sure how to set it up as it will be the first time I'm running Core as a VM (esxi) and I'm also considering rz2 vs. doing mirrors.

For vm boot drives: Do I need to give a core VM 2x boot drives and mirror them? Or just give it one drive and override the warnings? (the vhd is going to be on a bare-metal TN-Core machine)

Do things like special and metadata vdevs increase rz2 performance? Or is it always going to be about the speed of one single drive in the pool?

Thx.

 

[Arwen]

Don't know about VMs, but;

L2ARC is not suggested until you max out your memory. And generally >=32GBytes. There are exceptions, like a metadata only L2ARC, especially persistent one.

L2ARC size is suggested to be no more than 5 to 10 times the memory size. However, their is new thought on that because improvements in the L2ARC pointers, (which exist in RAM), are smaller now.

Metadata vDevs do improve performance, but generally just for Metadata, (directory info, file attribute, etc...). You can use a Metadata vDev for small files, like <64KByte. However, if the Metadata vDev gets full, you can loose additional Metadata benefits due to putting in too many small files.

You have to balance a Metadata's redundancy with the pool's redundancy. So, if the main pool is RAID-Z2, (2 disks of redundancy), you want 2 disks of redundancy in the Metadata vDev, thus a 3 way mirror. (Loss of a Metadata vDev means LOSS of the ENTIRE POOL!)

Last, both L2ARC and Metadata vDevs can "fill up", and if they do, any additional writes is through the main pool's vDev(s). This can allow a user to replace Metadata vDev with larger devices. Or add a second Metadata vDev.

 

[winstontj]

Thanks Arwen. The only VM related question I want to confirm is about boot drives. The guide: https://www.truenas.com/docs/solutions/integrations/vmware/deploytninvmware/
does not specifically say or state the recommended # boot drives/config. I'm assuming either one virtual boot drive, or pass through 2x physical boot drives and mirror them as if it was bare metal? (I'm currently giving it 1x boot .vhd)

The rest of my questions are about pool and dataset performance. I have a 6x 2tb ssd rz2 pool in another tn-core machine but because that is SSD's I don't notice a huge performance hit with it being rz2 vs. mirrors. This machine will be a general-purpose home nas that in a pinch, I'd like to be able to run a few VMs on it (if I need to do maintenance or reboot my other tn-core machines built for esxi vms, nfs, etc.). It'll do TimeMachine backups, have our photos, music and movies on it. It'll do frigate and zoneminder backups as well as have all of our paperless files (scans of papers, files, tax documents, house crap, etc.).

If a 8x 8tb rz2 pool will perform about the same as one single 8tb wd red drive, then I'll do mirrors. Also, I did a bunch of reading through jgreco's posts/info, as well as I found this thread: https://www.truenas.com/community/threads/the-problem-with-raidz.114529/ that talks about actual storage size expectations between raidz and mirrors. My takeaway from reading was that because of stripe size and other things, I'm not going to have a whole lot more space from rz2 vs. running mirrors. And additionally, mirrors will give a bunch better performance. So I'm really just looking to confirm that.

Another question: What is the rule of thumb when running multiple hba's? My mobo (x11spm-tpf) has 8x sata that I'm passing through to the VM. I'm thinking about running a second hba (dell h310) so I can pass through 8x more sata to the VM that I can use for ssd's, possibly run 10x 8tb drives (possibly 5x mirror vdevs), and have 6x sata left over for other vdevs.

This thing is going to be a general purpose NAS. Mostly a file server. Wondering if I'm being crazy trying to chase performance with mirror vdevs, and really wondering how slow a rz2 pool will be compared to mirrors. My current mirrored tn-core machines fully saturate their 10GbE uplinks. I don't need that from this VM, however I want it to be faster than 1GbE.

Thanks.

 

[Arwen]

I can't give you performance suggestions, perhaps someone else can.

Note that their are specific guidelines for reliable, (and data safe), methods for running TrueNAS as a VM;

Resource - "Absolutely must virtualize TrueNAS!" ... a guide to not completely losing your data.

[---- 2024/01/16: Still relevant. Virtualization really doesn't change much. Updates made as appropriate. ----] [---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some...

www.truenas.com

 

[Davvo]

You should have posted into the right section.

 

[winstontj]

You should have posted into the right section.

Sorry. If mods want to move then I would appreciate it.
I thought my questions were more basic about rz performance vs. mirrors, and questions like general guidelines for multiple hbas, etc. That stuff shouldn't matter whether it is virtual (with passthrough) or bare metal.