TrueNAS Scale v22.02.1 RAM Leak

[DevilNAS]

Hi everyone,

I have been struggling with Memory leak to Middlewared that is eating like 70% of my RAM Memory.
Version 22.02.1 release notes have a fix to the memory leak [NAS-114013] but it looks that for me it was not solved.

The only way that I used to solve this was to create a Cron Job to restart the service Middlewared for every two days.

This is a known issue or there is something wrong with my system? What makes Middlewared to increase the need for more RAM?

Thank you in advance!

 

[neofusion]

There are multiple causes, at least two are adressed in 22.02.2.
Time will tell if they found them all.

 

[nihues]

I'm having this behavior of SWAP usage in 2 days from reboot it grows to full swap usage even if ram is ~50%.
After it hits full swap, all server web page is slow or don't work anymore (even login page don't show user/pass), only ssh works, services/docker are slower but works.

Today I tested forcing/reducing zfs_arc_max and seems to have worked.
The command below limit to 5gb of ARC cache, try for testing if you have too many files or users it could reduce performance. The default is 50% of total ram, here was 15gb (for 32 total ram, so I reduced to 1/3 of original size). The numbers are in bytes.

Code:

echo 5368709120 > /sys/module/zfs/parameters/zfs_arc_max

I was having this errors from middleware - related to cputemp crash and he said limiting zfs_arc_max get good results, so I tested and 6h later I's still snappy to load truenas pages.

https://jira.ixsystems.com/browse/NAS-115692?jql=project%20%3D%20NAS%20AND%20text%20~%20swap%20ORDER%20BY%20created%20DESC