I'm not sure how it works with Unix but with Linux there is page file caching that always eats up all the free RAM (what is fine). I would assume Unix does it the same way and that would be counted as "services" too? So maybe something is causing alot of unix (not ZFS) file caching and that caching is maybe higher priorized than the ARC and will force the ARC to shrink?
Really looks like some kind of caching to me because it uses up all the RAM but never kills any processes and will free up the RAM again later. And with Linux page file caching is also not monitored by top/htop.
Maybe I should set the ARC minimum limit to something like 12GB so ARC can't be forced to shink below 12GB. If it is some kind of caching the "services" should only grow from 10-12GB to 20GB.
