Strange High Memory Usage by Services

[Seani]

I had planned to wait for TrueNAS 13 U2 but this is becoming too annoying to ignore. If I don't restart the system every week it ends up shutting down jails when it runs out of memory. I'll try my luck with 13 U1.

 

[GeorgePatches]

I've been poking at this a little and I discovered something. I was doing some backup job testing (run full backup, delete, repeat) and noticed that when I'd written a ton to my TrueNAS and then deleted files that were likely cached in the ARC, the ARC would report that it was empty, but the OS Wired memory stayed the same. So for some reason the OS is not realizing that the ARC is releasing the memory. But the memory is free and when I run another backup job it immediately starts rebuilding the ARC and the Services usage decreases accordingly.

I guess that's more of a FreeBSD problem? I think the UI widget is just pulling info from top. WIRED - ARC = SERVICES is what appears to be going on.

 

[homer27081990]

Adding to the pile...

 

[homer27081990]

Adding to the pile...

I am not convinced it is the same issue, though, after a reboot the numbers go way down with everything up and running.

 

[homer27081990]

After reboot (upgraded to 13.0 U2) services memory is increasing by 0.2MB/min

 

[anodos]

In your top output it looks like most of your wired memory is related to bhyve and / or jailed processes. c.f RES.

 

[themaharshpatel]

Few months back I was also having the same issue where my RAM usage in dashboard was very high, and memory usage by services shown in output of

Code:

top -o res

or

Code:

htop

was not matched with dashboard's values, only ZFS cache was matching and it was around 1GB.


Solution which worked for me was to remove all jails(I was having one for Gitea(plugin of community section) ) and I moved that to my VM of TrueNAS. Now I am getting around 75% of my RAM for ZFS cache and haven't faced any issue since then.

 

[homer27081990]

In your top output it looks like most of your wired memory is related to bhyve and / or jailed processes. c.f RES.

Thank you for taking the time to respond. The "normal" use of RAM by bhyve and jails is about half that shown in the UI. Even if I deactivate my VM (pfsense - 2GB RAM 4 cores) and shutdown my jails (plex, sonarr, radarr, bazarr, prowlarr, nextcloud, qbittorrent, vault, unifi, guacamole), service RAM usage falls by about 3,5-4GB and after a while starts to grow again with a moderate pace. Last time I did that, it grew to the point where I could not activate the VM or any jail, UI froze and I (panicked) rebooted through shell by IPMI.