Slow read performance in TrueNAS Core 12 U2

pk167

Cadet
Joined
Jul 18, 2020
Messages
7
Seems I am not only one encountering this problem. The reading performance of SMB can not reach 10Gbps which suppose be limited by 10G network and won't lower than 10Gbps. I have tried to set "aio read size = 0" and a few tunable parameters (please refer the below), somehow it didn't work at all, the reading speed still be around 6xx-7xxMB/s, but the writing speed is normally 9xxMB/s. The most frustrated is it is not allowed to rollback to FreeNAS. I think the worst case will be migrating all data into an other server and rebuild a FreeNAS server. Please kindly suggest me if there is any solution. Thank you.

Here are my current configuration
Dell R720xd E5-2660* 2/ 192G RAM/ H710mini in IT mode/ WD80EZAZ *12 (2 stripped Raidz2 of 6 Disks)/ Dell Intel X520-DA2

Tunables
1614046849688.png


SMB setting
1614046939249.png


Sharing setting
1614046994518.png


Pool setting:
1614109741752.png
 
Last edited:

qbix

Cadet
Joined
Dec 6, 2020
Messages
4
Sorry for the bad english. Perhaps some turns of speech will be incomprehensible or impolite, once again I apologize in advance - I use an online translator.

Please answer me:
1.When testing TrueNAS for reading, is Windows the client? If so, does the disk on it have time to receive the file over the 10G network? Is there enough bandwidth?
2. Sorry, as far as I understand, you have 12 disks organized into two vdevs of 6 disks each.
Perhaps the problem is that with such an organization of the pool, you can get a read speed not higher than twice the speed of a single disk and it is not clear whether ARC in RAM can help in this situation.

On my TrueNAS 12.0-U2, I see a similar problem with reading from TrueNAS (SMB server) on Windows Server 2016 (client), and the disk on the client (mirror of two server NVMe) allows the X520DA2 adapter to be completely utilized.
I see how reading (file size 20-50gb) starts at 1.1gb/s, then drops sharply to 650-670mb/s.
In one case out of five - copying is completely done at full speed, but this behavior is not at all satisfied.
The fact that sometimes copying still occurs at full speed allows us to conclude that it is not a matter of the size of the fast cache on SSD drives on Windows.
In one case out of five - copying is completely done at full speed, but this behavior is not at all satisfied.
All the necessary settings for the network adapter in Windows Server 2016, RSS bindings by processor cores have been completed.
If we carry out tests using iperf2, where the program server is running on TrueNAS, it follows from them that the network adapter is completely utilized in both directions.
I don't understand what the problem is and how to solve it.
By the way, if both network adapters on the X520DA2 board are combined into lagg0/lacp, then reading from TrueNAS always occurs only at a speed of 650-670mb/s.
If you then remove the lagg, then you can no longer return to the reading speed above this value.
Resetting the settings does not help, you have to reinstall the server.
As far as I remember there was an issue with Intel 10G NICs in version 12.0-U1.1 that should be fixed in 12.0-U2.
Perhaps this problem has not been fully resolved?
 

qbix

Cadet
Joined
Dec 6, 2020
Messages
4
Поправка: зеркало двух дисков NVMe для серверов
 

qbix

Cadet
Joined
Dec 6, 2020
Messages
4
Addition.
In my scenario of using TrueNAS, I am primarily interested in how large files, over 10gb, will be copied.
I ran tests to actually copy large files between TrueNAS and 2 Windows Server 2016 computers. Windows computers have Intel X520DA2 NICs.
All participants are connected through a switch with spf+ 10g ports, the separate testing of which has shown that it does not impose restrictions on the transmission of 10G traffic.
Above, I wrote that "single-threaded" copying a file from TrueNAS to a Windows computer, most of the copying process in progress is performed at a speed of 650-670 mb/s.
If we perform "multi-threaded" reading from TrueNAS on two or more Windows computers or run several copying processes on one Windows computer, then we see that the copying speeds create a total traffic that completely utilizes the network adapter 10G on TrueNAS.
The total read traffic of TrueNAS reaches 1.18 gb/s.
Recording speed on TrueNAS - even in one stream, from one computer Windows is equal to 1.10-1.13 gb/s.

The problem lies in the answer to the question of how to achieve a read speed that completely utilizes the capabilities of the adapter in one thread?
What's stopping this?
I would like to understand.
 
Top