Pool & Vdev design question

[tiberiusQ]

Dear all,

For the upcoming Truenas 13 I may go a new way designing my Pool & vdevs....
One server has 8 x sata nas drives, 64gb ecc and 10gb nic.
The main usecase is smb for about 100 users (local & vpn) + periodic snapshots + replication to a different server + this server is a desitnation for replication daily from elsewhere...Currently this box has one pool with one vdev using 6 x sata disks raidz2 and the performance is just ok but not great.

So I think about the following versions for a new design:

01. 8 x sata disks (1 vdev 1 pool raidz2)

02. 8 x sata disks (2 vdev 4 each - 1 pool raidz2)

03. 8 x sata disks (1 vdev 7 disks 1 pool raidz1 + 1 hot spare)

Or does this all make not a big diff and I should switch to ssds and if so, which pool design is then recommended with 8 slots if i aim a performance gain ?

Thanks & Best !

 

[sretalla]

You may find that you're bottlenecked on metadata depending on the number of files and directories.

You may also be bottlenecked on IOPS (since RAIDZ2 has the IOPS of only one disk for the entire VDEV/pool).

If it's the second issue, you may find option 02 works to improve things.

if it's the first, either SSD(s) for L2ARC with metadata only settings for the pool datasets or a metadata special VDEV may help.

 

[tiberiusQ]

Ok cool and how do I examine which one is the bottleneck to be able to make a proper decision ;-) ?

 

[sretalla]

You can start with a look art arc_summary

That should show if you're fully using your ARC and/or if metadata is missing a lot of hits.

 

[tiberiusQ]

What do you think....

 

[tiberiusQ]

Further and in general is it recommended to create 1 vdev with 8 sata disks radz2 or better 2 vdevs with 4 disks each raidz2 and why ?

 

[tiberiusQ]

And what about version 04: 8 x sata disks - 2 vdevs 3 disks each raidz1 + 2 hot spare) ?

 

[tiberiusQ]

This doc tells me anything I wanted to know:

https://static.ixsystems.co/uploads/2020/09/ZFS_Storage_Pool_Layout_White_Paper_2020_WEB.pdf

Thanks @sretalla

​

 

[sretalla]

Generally I don't like the hot-spare concept... it makes sense where you have a pool (or multiple pools) of lots of mirrors where those spares can start the resilvering immediately to reduce the time where your pool is without redundancy, but for RAIDZ, (IMO) it's much better to have those "spare disks" already in the pool and up to date, providing you the fault tolerance without the need to resilver to have it.

If it's difficult to phsically access your system, hot spares can be useful in all types of pool.

Further and in general is it recommended to create 1 vdev with 8 sata disks radz2 or better 2 vdevs with 4 disks each raidz2 and why ?

for capacity, it's clearly better to have a single VDEV (you get 6 disks of capacity and lose 2 to parity)... but for IOPS, it's better to have 2 VDEVs (but you lose 4 disks to parity... and would probably be better with mirrors, giving you 4x the IOPS if that's what you're after with the same loss of disks to parity).