How to stop FreeNAS from using swap

[bodriye]

Hi,

With the recent 11.1 release I am seeing FreeNAS memory using swap all the time. I have maxed out my ram with 32G
I have a raidz2 with 6x4TB disks.

I have a rancher OS VM running with 8G of ram being used.
I also have autotune enabled and it created following tunables:

I never had issue before but I think this might be a issue with the VM using up 8G of ram. But that is needed for my 7 docker containers, including plex and nextcloud.

 

[SweetAndLow]

Hardware specs?

But remove all the tunables because those are making things worse. And if your VM is using all your memory then that's why you are using swap.

 

[Chris Moore]

Hardware specs?

But remove all the tunables because those are making things worse. And if your VM is using all your memory then that's why you are using swap.

I know that the autotune used to be bad but has it not improved over the years to actually be some benefit? If not, why do they still have it?

I think the swap utilization is all down to the VM utilization.
Counting Rancher, that is 8 Virtual Machines and you are worried about less than a single GB of swap usage?
Does it really matter that much?

 

[bodriye]

Xeon E3-1241 v3 @ 3.50GHz
Supermicro X10SL7-F
Noctua NH-D15 CPU Cooler (I know overkill, but had it laying around)
4 x Crucial 8GB DDR3-1600 ECC CT102472BD160B
6 x 4TB WD RED WD40EFRX
SanDisk 16GB CZ43 Ultra Fit Series USB 3.0 Flash Drive (for boot)
SeaSonic Platinum Series SS-400FL2 400 Watt 80 PLUS Platinum
Fractal Design Define R4 ATX Tower (2 x intake fan, 1 x out fan)

 

[Chris Moore]

With the recent 11.1 release I am seeing freenas memory using swap all the time. I have maxed out my ram with 32G

How were you doing this before, because it wasn't possible to use docker containers in 9.x

 

[SweetAndLow]

There is a bug here I think. Something is not releasing data in arc fast enough.

Disable the tunables and see what happens.

 

[bodriye]

How were you doing this before, because it wasn't possible to use docker containers in 9.x

Before I was using 5gb of ram with 2 VMs. 4GB for plex on debain and another 1GB for transmission + openvpn + flexget on debian. Instead of nextcloud, I was using cifs for saving my files, but with rancher making deployment of containers so easy, I have migrated all my personal files to nextcloud container and even made offsite backup to google drive with duplicati container.

Edit:
And before that I was using jails.

 

[bodriye]

Disable the tunables and see what happens.

Done and turned off autotune. Will report back if I see swapping.

 

[Chris Moore]

Before I was using 5gb of ram with 2 VMs. 4GB for plex on debain and another 1GB for transmission + openvpn + flexget on debian. Instead of nextcloud, I was using cifs for saving my files, but with rancher making deployment of containers so easy, I have migrated all my personal files to nextcloud container and even made offsite backup to google drive with duplicati container.

Edit:
And before that I was using jails.

When you were using Debian, were you using ZFS?
Did you use any swap when you were running these VMs on Debian?

 

[Chris Moore]

I have never seen FreeNAS release swap space once it starts using it. I use the script that @Stux made to flush my swap if I feel like I need to, but he has cron run the script every hour if I recall. My swap use has been sitting at 10MB for a week or more but I never used any when I was on version 9.3...

 

[bodriye]

When you were using Debian, were you using ZFS?
Did you use any swap when you were running these VMs on Debian?

I didn't pay much attention to the filesystem and did auto partitioning when I installed debian VM, my guess is that it just used ext4 or something like that. It probably even created its own swap in the vm.

 

[0x6a6576]

I have been having this exact same issue every since I have upgraded to 11.1 and it is driving me nuts! I can't figure out what is using the swap space when there is a lot of free RAM available. I haven't had this issue on the previous versions. Same jails and VM. The only solution is to reboot the nas to free up the swap space but it will eventually max out again as per the screenshot below.

I also see a lot of this every now and then:

 

[fracai]

These swap issues could be this: https://redmine.ixsystems.com/issues/27270

 

[Chris Moore]

I have been running FreeNAS 11.1 since the day it was released and have not had this problem, but I am only running one 'VM' which is a Plex Plugin in the old style Jail as I have not made any effort to change over to the new virtualization system.
Are all of you using virtualization? If you are, I would say that is the problem, but I am not sure why the use of swap is such an issue.
What problem (malfunction) is the use of swap space causing?

 

[bodriye]

So its been like 14 hours and so far no swap usage after I turned off autotune.

But you will see to try and make the ram usage go up I played a 4K Planet Earth II for couple of hours on plex (80% CPU usage).
The "active" ram went up to like 8GB and "wired" ram has since stuck to 21-22GB. The inactive ram is just difference between max active ram of like ~8GB and current active ram.
I played 4k video again (this time it was not smooth playback) and noticed inactive ram usage kept going up and active usage kept going down (very odd) but never used swap.

I will report back if swap ever gets used

 

[Stux]

but he has cron run the script every hour if I recall

I actually run it every 10 minutes on some systems. The script is a no-op if there is no swap in use.

 

[bodriye]

I actually run it every 10 minutes on some systems. The script is a no-op if there is no swap in use.

Can I run that script on my system, will it make any difference when using VMs?

 

[Chris Moore]

Can I run that script on my system, will it make any difference when using VMs?

Here is a link to the discussion about the script that @Stux wrote:
https://forums.freenas.org/index.ph...any-used-swap-to-prevent-kernel-crashes.46206
You can give it a try. I only run it occasionally because I hardly ever have any swap in use.

Is the swap that your system is using causing some kind of malfunction?

 

[Chris Moore]

So its been like 14 hours and so far no swap usage after I turned off autotune.
View attachment 22114
But you will see to try and make the ram usage go up I played a 4K Planet Earth II for couple of hours on plex (80% CPU usage).
The "active" ram went up to like 8GB and "wired" ram has since stuck to 21-22GB. The inactive ram is just difference between max active ram of like ~8GB and current active ram.
I played 4k video again (this time it was not smooth playback) and noticed inactive ram usage kept going up and active usage kept going down (very odd) but never used swap.

I will report back if swap ever gets used

My NAS has been up for 15 days and I have only 10MB of swap usage and it isn't changing. Is the swap utilization causing some sort of malfunction?

 

[bodriye]

Is the swap that your system is using causing some kind of malfunction?

My NAS has been up for 15 days and I have only 10MB of swap usage and it isn't changing. Is the swap utilization causing some sort of malfunction?

It is just that I didn't see swap usage before 11.1 and now I am seeing the usage spike. From what I know zfs relies on RAM to do lots of things and that is why ECC ram is recommended but if swap that is spread across my 6 drives and some of the drives start to malfunction, there is a slight chance of loosing my data or having it in unrecoverable state as swap might get corrupted too.