Freenas gets slow after couple of hours

[maj0rmil4d]

Hello , I hope you doing well.

I'm facing a weird problem , my freenas has a pool with capacity of 23 TB ( only filled with 5 tb of data ) and it gets slow after couple of hours .


Here is the hardware details :




dl 380 g9
ram => 128 (2133 - ddr4)
cpu => Xeon(R) CPU E5-2609 v3 @ 1.90GHz
storage =>
hdd : 9 * 5.5 T nlSAS ( 7200 rpm )
SSD : LOG , CACHE : ATA MK0400GEYKD ( write intensive )


% zpool list -v
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
freenas-boot 14.5G 1.00G 13.5G - - 0% 6% 1.00x ONLINE -
da11p2 14.5G 1.00G 13.5G - - 0% 6%
mainpool 49T 10.1T 38.9T - - 2% 20% 1.00x ONLINE /mnt
raidz3 49T 10.1T 38.9T - - 2% 20%
gptid/e6db44ec-b5e0-11ea-b541-2c768a4e62c0 - - - - - - -
gptid/e8e446c5-b5e0-11ea-b541-2c768a4e62c0 - - - - - - -
gptid/e8c61235-b5e0-11ea-b541-2c768a4e62c0 - - - - - - -
gptid/e90481c1-b5e0-11ea-b541-2c768a4e62c0 - - - - - - -
gptid/e95b43da-b5e0-11ea-b541-2c768a4e62c0 - - - - - - -
gptid/e93e1eab-b5e0-11ea-b541-2c768a4e62c0 - - - - - - -
gptid/e94b9b92-b5e0-11ea-b541-2c768a4e62c0 - - - - - - -
gptid/e99066a5-b5e0-11ea-b541-2c768a4e62c0 - - - - - - -
gptid/e9217b83-b5e0-11ea-b541-2c768a4e62c0 - - - - - - -
log - - - - - -
gptid/e461bf92-b5e0-11ea-b541-2c768a4e62c0 372G 60K 372G - - 0% 0%
cache - - - - - -
gptid/e46bc8ef-b5e0-11ea-b541-2c768a4e62c0 373G 369G 3.68G - - 0% 99%
%
I thought it's about the cache ( it will completely filled and only gets free when I reboot ) so I just removed the cache from the pool but problems still exists .

My hdd is NLSAS seagate ( probably it should gives me 70 mn/s just like the first hours after restart ) but after couple of hours it only reachs 35 mn/s

What is the problem ?
I just don't know what is wrong
My network is 10 GB/s , my hdd is healthy , there is no cpu load , and zfs cache is about 120 gb