What to expect from this setup.

[kasmafan]

Hi.
I'm long time user of Freenas, built several machines but now i'm a bit lost.
The last machine i've built have these specs.

TrueNAS-12.0-U8
Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz 10 Cores
128GB ECC RAM
Supermicro MOBO and Case
12x Seagate 16TB Exos Drives (ST16000NM001G-2KK103)
10GB Fiber card

My pool has 2x Z2Vdevs of 6 disks each combined in a Pool "VolumeX"
Network wise everything is working fine, Iperf to and from the machine I can use the 10G.

But reading from the SMB pool I'm getting around 450MB read and 250 Write speeds, Is this normal?

Thanks

 

[sretalla]

Depending what you're reading and writing (and the associated IOPS that are required), it may well be normal.

With 2 VDEVs, you're working with the IOPS of only 2 of your 12 disks.

As soon as you exhaust the buffers, you're going to slow to around 400 IOPS at most, which will be a limit you'll notice for 10Gbits and a bunch of small files or for block storage.

If the intended purpose of the server is file sharing of large files only, you may want to do a bit of tuning to improve your throughput (like MTU of 9000) and perhaps use TrueNAS 13 where SMB is multithreaded by default.

 

[kasmafan]

Hi sretalla, thanks for the reply.
So if I need to speed this up, if I add another vdev (equal to the others) this will give me the speed of 3 disks and so on?
Is any kind of SSD cache advisable?

Once again, thanks for the reply's

 

[sretalla]

So if I need to speed this up, if I add another vdev (equal to the others) this will give me the speed of 3 disks and so on?

Sort-of... you'll get additional IOPS capability (if you clear out your pool or do something to rebalance it after adding th eadditional VDEV), but throughput won't necessarily change, so if you're IOPS bound, then things may get better, but if it's just mechanical throughput, maybe not much will improve.

Is any kind of SSD cache advisable?

You have plenty of RAM, so adding L2ARC isn't likely to win you a lot.

If it's IOPS (and Sync writes), maybe a SLOG could help, but you should really do some research on that one as you can't just use a cheap SSD and expect any good things.