TrueNAS 10Gbe Intel NIC, very slow read speed, write speed 'normal'

derkxr

Cadet
Joined
Apr 8, 2017
Messages
5
Hi all, best wishes for 2021, to start with.

I have a HP Microserver Gen8 (base model with Celeron G1610t, 16 GB ECC, 2 broadcom 1Gbe NIC’s) and an additional Intel X540-T1 10Gbe card in the PCI slot.

5x 500GB WD Red NAS SSD’s make up for a RAID-Z1 pool.

Using FreeNAS 11.3-U3.2 I always had nice sequential (SMB) read/write speeds for bigger files between 400-500MB/s over a direct 10Gbe connection between the FreeNAS box (X540-T1 NIC) and my workstation (Lenovo P330, with 10Gbe Intel X550-T2 PCI card).

I upgraded to TrueNAS 12.0-U1 by a clean install on to a pair of fresh USB keys and recreated user, pool and shares.

Unfortunately to find out that read speed over the 10Gbe link is extremely slow, where write speed is the usual 400-500MB/s. Read speed by CrystalDiskMark (SEQ1M Q1T1) shows 6-7MB/s. A real life test copying a ~150 GB folder with thousands of photo’s to and from the pool also showed substantial faster write than read speeds (200-300 write speed vs 20-50 MB/s read speed).

Even stranger, an SMB share on one of the 1Gbe NIC's gives ~100/100 read/write speeds as expected, so much faster reads than over 10Gbe.

Reverting back to my still available FreeNAS 11.3 USB-keys I immediately got the higher ‘on par’ read/write speeds again. CrystalDiskMark shows 420/480 read/write for SEQ1M Q1T1 (4GB test file).

I find it strange there is such an imbalance between read/write speeds in the TrueNAS setup, specifically on the 10Gbe link but not on a 1Gbe link. Faulty NIC or cabling, maybe , but exact same setup with FreeNAS is fine…

I searched and found several mentions of speed issues over 10Gbe, with ample suggestions for testing and resolve, but none specifically on the huge imbalance between read and write speeds I found.

Thanks for any clues/suggestions;-)



Regards,

Richard
 

MikeyG

Patron
Joined
Dec 8, 2017
Messages
442

Kris Moore

SVP of Engineering
Administrator
Moderator
iXsystems
Joined
Nov 12, 2015
Messages
1,448
Yep, seems to be a regression on the network driver side of things. Fix is being worked on and should merge into the next update.
 
Top