SOLVED Network shares are eating all my RAM

[Vitz]

Hi there,

So I'm not sure what's going on here. I'm running TrueNAS Core 12.0-U5.1 in a virtualised environment on Proxmox. Everything is functional but there is one rather peculiar problem and I don't even know where to start in order to fix it.

Seeing as Proxmox does not offer robust network sharing options, I opted for TrueNAS to handle that part of the setup. I gave the VM 48GB of memory, which should be more than plenty for the ~30TB of hard drives connected to it through an ASUS PIKE2008 SAS controller (in IT mode) that TrueNAS has direct access to using PCI passthrough.

Another VM is running Jellyfin on Ubuntu Server 20.04 LTS, which accesses my data through an SMB share set up in TrueNAS. The problem is that whenever I play a video or refresh the library, RAM usage in TrueNAS skyrockets and fills it up over the space of about an hour (see screenshot). Once it fills up, everything still works, but performance isn't great. Even if it was, I'd still be making this thread because it makes no sense to me. The weird thing is that it was even worse when I tried NFS. The same behaviour occurred with one big difference: When memory was filled up, it flat-out crashed the TrueNAS VM. With SMB, it seems to release some memory when it needs more. NFS just didn't care and took it all so the OS couldn't even function. What is also weird is that none of these issues were present when I moved terabytes of data from one location to another through Windows 10, which of course also generates a ton of SMB traffic. It's like Jellyfin is the culprit here but why would TrueNAS allow an entirely separate machine to utilise all of its RAM?

All of this must be completely unusual behaviour. I'm more of a hardware guy and not terribly experienced with TrueNAS and networking in general. I just don't understand why TrueNAS allows a sharing service to nab all its available memory and only leave a sliver of it for the core system and ZFS cache. The install is basically fresh and I've pretty much only set up shares and permissions through ACL on it.

Is there something obvious I'm overlooking? Could I somehow diagnose this? I'd love some input and would gladly provide more details if requested.

Thank you in advance.

 

[Heracles]

I'm running TrueNAS Core 12.0-U5.1 in a virtualised environment on Proxmox.

Not recommended...

https://www.truenas.com/community/t...nas-in-production-as-a-virtual-machine.12484/

an ASUS PIKE2008 SAS controller (in IT mode) that TrueNAS has direct access to using PCI passthrough.

Still, Proxmox as an hypervisor for TrueNAS and such a large storage are not recommended...

whenever I play a video or refresh the library, RAM usage in TrueNAS skyrockets and fills it up over the space of about an hour

Normal : Unused RAM is wasted RAM. As such, TrueNAS will fill it all with the data it has to handle. This is just basic caching.

Once it fills up, everything still works, but performance isn't great.

Normal : nothing can compare to RAM when talking about speed (except CPU but they store basically nothing).

it flat-out crashed the TrueNAS VM.

And that is a very example why TrueNAS on Proxmox is not recommended at all. The only hypervisor with a minimum amount of experience is VMWare. ESXi can be used for free but still, even more for such a large storage, to go bare metal would be the way.

All of this must be completely unusual behaviour.

Nope. Everything is the most standard and as expected possible.

 

[Vitz]

Well, that's a bummer. Is there any chance the resource you linked to is outdated? After all, the last post in that thread is from 2015 and technology moves fast.

 

[danb35]

Well, there are at least two issues with Proxmox as a VM host for Free/TrueNAS. The first is simply that it simply doesn't have the testing that ESXi does. The second is that the qemu guest agent isn't easily available to run there, so there's minimal communication between the host and the guest OS. The latter issue appears to be addressed here:

Resource - QEMU Guest Agent

DISCLAIMER: Under no circumstances will I be held responsible or liable in any way for any claims, damages, losses, expenses, costs, or liabilities whatsoever (including, without limitation, any direct or indirect damages for loss of profits...

www.truenas.com

I'm not aware of any known incompatibilities or conflicts with Proxmox/KVM/QEMU and Free/TrueNAS, but again, it doesn't have near the testing behind it that ESXi does (AFAIK).

 

[Vitz]

To follow up on this: I believe I've fixed this issue by simply switching over to TrueNAS SCALE (yes, I know it's still in beta). It virtualises beautifully now that it's running on Linux instead of BSD.