SOLVED SMB and NFS Slow Read Speeds, iSCSI Performs as Expected

[bobert84]

Hello!

I have tried this with TrueNAS Core and Scale and get similar results.

Here are the specs of the machine:
Dell R640 Server
Dual Intel Gold 6132 CPU's (14c, 28t)
256 GB Ram
2 - LSI 9207-8e HBA's connected to 2 - 45 bay Supermicro JBODs
90 - 20 TB Drives
Boot is on 2 - 240 GB SSDs in R640
Drives are 9 drives per VDEV, 10 VDEV's, RAIDz
Connected to network with 1x 10 GB Intel 710 NIC

dd utility consistently gets over 2 GB/s read and write (approaches 3 GB/s most of the time) on the array. Seems appropriate for the HBA's and connected busses.
iperf gets ~8,000 Mbps between TrueNAS and clients, as expected.

iSCSI gives me good numbers on CrystalDiskMark (Basically maxing out the 10 Gb NIC):

Upload to TrueNAS from Windows or Linux NFS averages 800-900 MBps, which seems reasonable. This is with a large file ( > 10 GB).

Download from TrueNAS to Windows/Linux SMB or FTP or Linux NFS is where the problem lies: It averages a very constant 150 MBps to 250 MBps with large sequential files. Not the performance I was looking or hoping for.

Here's what I've tried:

• TrueNAS Scale seemed to have more variance in performance, so I switched to TrueNAS Core and found performance to be more consistent (especially with iSCSI), but little change on SMB and NFS shares. (the rest of the tries below are on TrueNAS Core)
• Turning on AutoTune (Helped some, but only 150 to 200 MBps)
• Connecting from different Windows and Linux boxes.
• Enabling and disabling flow control on the switch.
• Read through a ton of forums trying to find an answer, but couldn't find anything outside of what I've outlined above.
• Watching CPU and Ram usage (both stay low) During download from NAS, 1 or two threads may get up to 20% usage.

It almost seems like there's a speed limit of some kind for outbound file shares. Am I missed a setting or throttle that is causing this? When watching the array work while downloading files from it, the drives are barely working. They work more when uploading (4x the data being moved).

The hardware seems like it is performing as needed and even the Storage pool is working as needed, especially since iSCSI and Uploads are running as I'd expect.

What else should I try?

Thanks for any help you can give!

 

[bobert84]

Issue solved. Unfortunately, I cannot delete this thread or I would. Discovered it was a client/switch issue. Server seems to be running fine.

 

[HoneyBadger]

I'll mark the thread as SOLVED for you, but in general we prefer to leave solved problems up for future community users to benefit from the experience shared within.

Is there anything else specific that you can share with regards to the client/switch issue, such as symptoms or steps that led you to this conclusion, and might be able to help other future users in a similar situation?