TrueNAS 12U1 less than half the read speed of FreeeNAS 11.3U5?

[cyr]

So I have been hesitant to upgrade my trusty FreeNAS machine to TrueNAS 12, even though it would bring a couple of features I'd like (most notably native encryption and encrypted replication). I thought I would check it out on a spare machine and also run some tests and benchmarks with the encryption...

The system I used is a Dell Poweredge R420, with a single E5-2403 CPU, 48G RAM and 4 2TB SATA 5400rpm drives.

I never got around to running tests with encryption, because no matter what I try I can't get decent read speeds with TrueNAS 12. I have run various pool configurations (4-way stripe, 2 mirror pairs, RAIDZ1) and various simple tests (dd, cat > /dev/null, iozone, scrubs). They all result in only about 40-50MB/s per drive read speed, while on FreeNAS 11 it's > 100MB/s per drive. Write speeds on the other hand is about the same...

As an example, this is a dd test with a 4-way striped pool, compression and encryption off. All default settings.

FreeNAS 11.3U5:

root@freenas[/mnt/testpool/plain]# dd if=/dev/zero of=test.bin bs=1M count=100000
100000+0 records in
100000+0 records out
104857600000 bytes transferred in 216.760091 secs (483749567 bytes/sec)
root@freenas[/mnt/testpool/plain]# dd of=/dev/null if=test.bin bs=1M count=100000
100000+0 records in
100000+0 records out
104857600000 bytes transferred in 230.173066 secs (455559817 bytes/sec)

TrueNAS 12U1 (upgraded, same exact pool):

root@freenas[/mnt/testpool/plain]# dd if=/dev/zero of=test.bin bs=1M count=100000
100000+0 records in
100000+0 records out
104857600000 bytes transferred in 229.388011 secs (457118920 bytes/sec)
root@freenas[/mnt/testpool/plain]# dd of=/dev/null if=test.bin bs=1M count=100000
100000+0 records in
100000+0 records out
104857600000 bytes transferred in 503.398958 secs (208299199 bytes/sec)
root@freenas[/mnt/testpool/plain]#

I also tried TrueNAS scale alpha with very similar results, maybe this is a OpenZFS 2.0 issue?

Any ideas what I could try to get my linear read speed back so I can maybe run TrueNAS 12?

 

[Mark St.]

try this:
sysctl net.iflib.min_tx_latency=1


https://www.truenas.com/community/t...r-truenas-12-upgrade.88312/page-3#post-621334

 

[cyr]

Thanks for the suggestion, but my problem isn't network related. I haven't even gotten to trying out network performance, just trying to get something decent out of the pool locally first. (I did try, and it made no difference unfortunately).

 

[Kris Moore]

@cyr Please go ahead and make a ticket on jira.ixsystems.com with an attached debug so we can take a look at this in more depth.

 

[cyr]

Done, correctly I hope.

https://jira.ixsystems.com/browse/NAS-108818