FreeNAS - Disk Read goes slow over time

[Anthony Segelhorst]

All,

After a reboot of FreeNAS we can get around 700 MB/s for read and write for SMB connections. After times passes (usually less than a day). The following will occur:
write: 700 MB/s
read: 200-300 MB/s

If we reboot the SMB connection goes back to normal. Does anyone have any thoughts on what/why/where to look?

All of the connections to FreeNAS are SMB. We need SMB for FCP.
If we perform an FTP when the read is slow via SMB, the read/write is still around 700 MB/s. This makes me believe is an SMB issue and not a network/disk io issue.


version: FreeNAS-11.0-U4
Disk WD 4 TD Red
Config:

Code:

		NAME											STATE	 READ WRITE CKSUM
		LV0											 ONLINE	   0	 0	 0
		  raidz2-0									  ONLINE	   0	 0	 0
			gptid/2dae4533-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0
			gptid/2eb2baad-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0
			gptid/2fe64965-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0
			gptid/30b1ebf4-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0
			gptid/319056db-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0
		  raidz2-1									  ONLINE	   0	 0	 0
			gptid/42ebbddc-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0
			gptid/44243d1c-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0
			gptid/455c513f-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0
			gptid/46b7dc59-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0
			gptid/481b7fad-c447-11e7-9467-d05099799d45  ONLINE	   0	 0	 0

 

[wblock]

We really need to know the details of your hardware to be able to help.

 

[Anthony Segelhorst]

Here is some info on the HW. Let me know if you need more information.

CPU - Intel Xeon E5-2609 v2 Ivy Bridge-EP 2.5 GHz 10MB L3 Cache LGA 2011 80W BX80635E52609V2 Server Processor
Mem - 32 GB (2 x 16) - 40-Pin DDR3 SDRAM ECC
MotherBoard - ASRock EP2C602-4L/D16 SSI EEB Server Motherboard Dual LGA 2011 Intel C602
Disks - WD Red 4TB NAS Hard Disk Drive - 5400 RPM Class SATA 6 GB/S 64 MB Cache 3.5-Inch (WD40EFRX)
Num of Disks - 10
Raid - raidz2
Network: 10 GB Chelsio card.

NOTE: I have autotune enabled.

Code:

#This was some disk io benchmarks we took after building the freenas server.
#Disk Speed Compression testing locally on FreeNAS server.
zfs set compression=off LV0
dd if=/dev/zero of=/mnt/LV0/trash/test1.dat bs=2048k count=10000
20971520000 bytes transferred in 32.032059 secs (654704092 bytes/sec)
	
dd of=/dev/zero if=/mnt/LV0/trash/test1.dat bs=2048k count=10000
20971520000 bytes transferred in 25.621227 secs (818521313 bytes/sec)
		
dd if=/dev/zero of=/mnt/LV0/trash/test2.dat bs=1M count=10000
10485760000 bytes transferred in 14.527714 secs (721776310 bytes/sec)
	
dd of=/dev/zero if=/mnt/LV0/trash/test2.dat bs=1M count=10000
0485760000 bytes transferred in 1.492889 secs (7023804553 bytes/sec)

 

[Anthony Segelhorst]

Sorry to ask, but does anyone have any thoughts on where to start troubleshooting this issue?

 

[dlavigne]

It might be resolved with 11.1 as there were many speed improvements. If not, what is the capacity of the pool? Is there a pattern in the number of connections when the slow down occurs? Anything in /var/log/messages when things slow down?