Slow read speeds / Fast Write Speeds / 10G

[eretron]

Hello,

recently I have upgraded the whole network to 10Gbps interfaces, and was doing some read/write tests for a few days. I have a few issues that I hope someone could help me with.

First of all, TrueNAS Core hardware is:

Intel(R) Xeon(R) Silver 4116 CPU @ 2.10GHz
192GB ECC Memory
Dual intel 10G NICs
16x WD Red 4TB HDDs

So, pool is made of 2 vDevs with 8 Hdds each in raidz2 setup about 75% full

Code:

state: ONLINE
  scan: scrub repaired 0B in 05:18:26 with 0 errors on Sun Nov 15 07:18:28 2020
config:

        NAME                                            STATE     READ WRITE CKSUM
        Pool                                              ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            gptid/7b677ef4-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/7c3e1be6-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/7d0dcf38-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/7ddd4ec3-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/7eb3fa5b-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/7f995fa1-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/80732fe1-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/81463489-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
          raidz2-1                                      ONLINE       0     0     0
            gptid/821add09-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/82fbde4d-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/83e04eb9-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/84c7b3e8-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/85a897ca-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/86824c39-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/876cb52f-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0
            gptid/88516ef0-a7aa-11e9-a955-ac1f6bbb2cf6  ONLINE       0     0     0

Write speeds to the pool via SMB are locked at 1.1 GBps all the time, so no issues there at all... On the other hand, copying from NAS I get speeds varying from 200-500MBps on the first run.

If I try to copy the same file again the speed is locked to 1.1 GBps, since I guess now the file is read from the Memory instead from the HDDs.

Vast majority of the files on NAS, something like 95%, are large video files, ranging from 10GB to 500GB each, so I would need a setup improvements for this kind of load.

My question is, what could I do to improve read speeds from the HDDs? And is this a normal behavior, for this amount of HDDs?

Thanks in advance

 

[joeschmuck]

This sounds like the SMB problem experienced by many already.
Check out this posting...

 

[eretron]

Thanks, Ill give it a try over FTP to se if I can replicate the results.

 

[eretron]

This sounds like the SMB problem experienced by many already.
Check out this posting...

Well actually over FTP I get a steady 670 MBps, and that by itself is a big improvement, but over FTP as well, second copy of the same file, when its read from the Memory instead of the HDDs its locked at 1.1 GBps all the time.