Raidz best configuration for 8 disks

alex87alex

Dabbler
Joined
Jun 28, 2017
Messages
17
Good morning, I have an HP ML110 Gen10:
CPU: Intel Xeon Bronze 3104
RAM: 32gb DDR4
HBA: LSI SAS3008
HDD: 8x4tb
ETH: 10gbe X540-T2
TrueNAS-12.0-U8.1

What's your opinion about raidz configuration? Actually I have an HP Microserver Gen10 with 4x4tb on raid 10 with truenas (striped mirror) and I need more space and good performance (fileserver, plex, backup center for all pc and vm), so I tought these hypothetical setup:
  • 2x raidz1 striped (raid 50) or
  • striped mirror (raid 10) or
  • easy and "slow" raidz2
Do you think is it a good improvement add to the pool storage an l2arc cache ssd?

I have a backup system, I will shortly implement also a cloud backup, so the only problem to lost the entire pool is the recovery time...

Thank you for help
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
fileserver, plex, backup center for all pc
These use cases are well suited to RAIDZ.
This is suited to Mirrors.

What's your opinion about raidz configuration?
Depends how concerned you are about pool loss... Red cells indicate pool loss with the 2nd disk.
1650888459653.png


1650888652058.png


Clearly you can see RAIDZ2 is the only case where 2 lost disks isn't fatal for the pool.

Running with 2x RAIDZ isn't necessarily the IOPS boost that you might think for the VMS either: https://www.truenas.com/community/threads/the-path-to-success-for-block-storage.81165/

Do you think is it a good improvement add to the pool storage an l2arc cache ssd?
Not until you run arc_summary and see you're missing ARC hits at a high level. (and in that case add more RAM first anyway)
 

awasb

Patron
Joined
Jan 11, 2021
Messages
403
If you can (and I guess so, since you asked about adding L2ARC), add one or two SSDs (if two: mirrored) as an exclusive VM-pool. Replicate that pool’s snapshots to your storage pool (raidz2).
 

alex87alex

Dabbler
Joined
Jun 28, 2017
Messages
17
Thank you for reply, I think the best configuration for me is raidz2 and 1 sata ssd for vm storage. What's the performance of 8 hdd on raidz2 config? (mb/s transfer rate 100% read and 100% write)
 

Bikerchris

Patron
Joined
Mar 22, 2020
Messages
208
Only chiming in (far from being as experienced as others here) but I have 8 x 4TB ironwolfs in RAIDZ2 and so far I've seen 200-220MB/s reads over a 10G network. I could do with making the destination an NVME, but I don't have one spare at the moment. I could use iperf sometime, but not sure when.

This is a handy webpage: https://calomel.org/zfs_raid_speed_capacity.html
 

Morris

Contributor
Joined
Nov 21, 2020
Messages
120
I think you are on the correct track Alex with a pool designed for all your file operations and another for the VMs. RAID Z2 is the way to go for the file operations unless you need faster transfer rates in which case 2x raidz1 striped (raid 50) will work well. I use it for a large backup target across a 10-Gb lan and run about 4-Gb/sec. Two SSD mirrored would be ideal for the VMs or you could go with mirrored spinning disks and an L2 ARC.
 
Top