NFS read speed half of write speed

[winnielinnie]

I'm still struggling to see how testing sync writes which go to the ZIL and the disk will give me insight into my reads. What are you trying to test? That writes to disk are slower than reads from them?

NFS read speed half of write speed​

Transferring a 40gb file from my client to my TrueNAS I get 7-800MiBps. Transferring a 40gb file from TrueNAS to my client, I get 350MiBps. iperf3 shows a bit over 7Gbps each way.

Why would the sync property have anything to do with read speeds?

Because you're claiming slow reads speeds relative to your write speeds. (It's in your thread's title.)

Since your dataset is set to sync=disabled, then it means all writes are enforced as "async", even though NFS by default would use "sync" (for example, if the property was set to "sync=standard".)

Set your dataset property to sync=standard, then see if your write speeds still maintain 800 MiB/s.

Because you're comparing the 800 MiB/s write speeds to 350-450 MiB/s read speeds. You may in fact be getting the best possible read speeds for your setup of spinning drives, 11-wide RAIDZ3 shucked Western Digitals. From how you introduced your initial post, it seems like the glaring comparison of "read speed half of write speed" is what set you off to search for a solution (in which one might not be needed, since everything is working optimally.)

If you had left the dataset to sync=standard, then done your tests before making this forum thread, you might notice that your read and write speeds are fairly comparable over NFS, and perhaps may not have felt like "I'm missing out on something, I need to find a solution."

I'm not even implying you're wasting your time or that you shouldn't have made a forum post. In fact, I think it's great that people want to get the best out of what they have. However, you don't want to search for a solution if none really exists in the first place because your initial premise was skewed.

 

[CJRoss]

Copying 120G worth of 6G files from my client to the NAS PCIe3 nvme pool results in the same 700MiBps speed. Copying that back to the client SSD results in the same 350MiBps.

Also, the nvme pool drive is set to sync=standard.