ZFS: RaidZ2 or Striped Mirror?

[jakemarnatti]

Hello!

I am building an all-in-one box as part of a new media server setup. I already have all the parts which include 128gb of ECC RAM with dual E5-2650s. I have 8 identical 3TB HDDs. They were eBay purchases for next to nothing so they are old and reliability is questionable. I also have 2 120gb ssds for the OS install. How should I setup these HDDs because I will also host several VMs that will use the storage pool so higher performance is also a concern? I also would like advice on whether or not to use one of the ssds as a cache and use the other for an OS install?

I read that striping mirrors is more ideal than a RAIDZ2? But I also need expandability for the future as the media pool expands.

Thanks!

 

[Ericloewe]

Reliability is better with RAIDZ2, but performance - namely IOPS - will be better with mirrors.

 

[NugentS]

IMO - for VMWare you need IOPS - otherwise the VM's feel like they are running in treacle.
So MIrrored vDevs all the way.
However the flip side to that is that as @Ericloewe says - the reliability will be worse.
8 Disks RAIDZ2 = 6 Disks of Storage, and you can lose any two disks
8 Disks Mirrored vdevs = 4 Disks of Storage and you can lose up to one disk per vdev - so you could lose 4 disks and be OK (each in a different vdev) or you could lose 2 disks and lose the pool (both disks in the same vdev)

RAIDZ2 = (simplistically) IOPS of a single disk
Mirrored vDevs = IOPS of 4 disks

Also the time to rebuild (resilver) a RAIDZ2 >> resilver mirrored vdevs as with the mirrors - so less stress on the array during the resilver

YMMV.

What are the SSD's?

 

[jakemarnatti]

They are consumer grade Kingston 120GB A400 SATA 3 2.5".

 

[NugentS]

The Kingstons probably (99%, without checking) don't have PLP (Power Loss Protection) - so don't use them as SLOG.
With 128GB RAM you won't need L2ARC
And I wouldn't use one as a MetaData Special - that just asking to loose the Pool

Put the HDD's through a badblocks sequence before using them in anger - that will likely show up any that are due to fail soon. (Note that this will take days)

Just use the SSD's as boot devices.

How are you attaching the HDD's (& SSD's) to the motherboard? Onboard SATA or HBA?
Posting the full hardware spec helps

 

[jakemarnatti]

The SSDs are attached to the only 2 SATA III ports on the motherboard, the HDDs are attached via an LSI HBA card flashed to IT mode. Also, how would I go about adding the specs as you have done to your profile?

 

[NugentS]

Look in https://www.truenas.com/community/threads/my-content-not-working.94653/ where I asked the same question and @Heracles was kind enough to point the solution out