TrueNAS uses ZFS, and ZFS will use memory for a type of read cache call ARC, (Adaptive Replacement Cache). For TrueNAS SCALE, that is maxed at 50% of memory. Then their is the actual code and other data installed into memory.
Reducing the ARC means more reads from disks, and potentially slower performance.
Neither TrueNAS nor ZFS were designed to be virtualized. Lots of people do it, and their are now guides to make it reasonably reliable. But, wanting a smaller memory foot print implies you want less ZFS ARC.
Next, 8GB is very low for TrueNAS, some might even consider it too low. Today, many advocate for 16GBs as the recommended minimum. (I don't know the "official" required minimum... you should be able to get that off the documentation.)