Advantage/Disadvantage to using FreeNAS OS disk for Swap

[siconic]

During a few of my other posts, people had mentioned moving the SWAP to another disk. I liked the idea, so I did some research, and made the move to my main OS disk, and made it 32GB.

I immediately noticed my disk busy time on my 11 disk main Zpool has gone down from 15% to 5%. So thats good, less disk activity = longer life. Also, this should allow disk failures and hot swapping to keep the system running along. Also utilizes the most underutilized disk on the system. Utilizing the main disk is both good and bad, since using it more = more chance of failure, but sitting there idle 90% of its life is really no good either. All seem good to me!

But is there any more advantages and what are some disadvantages to moving SWAP? I feel like the reason it was built like that in the first place was to reduce writes for USB and SSD use. Since I am running a high reliability and fast 15k SAS, I am not sure any of the applies, so is there a disadvantage for me?

Thanks!

 

[Chris Moore]

My systems hardly ever tell me that they are using any swap, so I don't know how much benefit there is. I moved my 'system dataset' to the boot pool and that caused increased activity on the drives. I run a mirrored pair of 40GB spinning disks for my boot pool and I monitor their health with SMART tests and status reports and they appear to be handling it. I am seriously considering moving the swap also, I just have not gotten around to it yet.
The TrueNAS system that iXsystems sells is the ultimate beneficiary of the development on FreeNAS and it uses SATA DOMs for the boot pool. I am sure the developers want to keep the wear on those to a minimum because the Disk on Module devices are not made for heavy use. I don't think the SATA DOMs are much better than USB sticks. So, with a reasonably durable SSD or hard drive, I think the boot pool can successfully do more but the design is to keep activity to a bare minimum.
Just my thought on it. No warranty, no return.

 

[SweetAndLow]

How did you move the swap? That is not a setting in the GUI.

There should be zero write or reads happening to the swap on zfs so I have no clue why you think your disk busy went down because of it.

I'm pretty sure you are barking down the wrong path and have not accomplished anything.

 

[siconic]

How did you move the swap? That is not a setting in the GUI.

https://forums.freenas.org/index.php?resources/how-to-relocate-swap-to-the-boot-drive.69/

There should be zero write or reads happening to the swap on zfs so I have no clue why you think your disk busy went down because of it.

I do have swap use. I am running quite a few VM`s, and probably should be running closer to 96GB of RAM, so maybe thats why. But yesterday when I did swapinfo I had 30% utilization. Today:

So if thats a problem not related to RAM, I am all ears for a fix. But I know I tax my system a bit, mostly because it takes it like a champ. I do plan on upgrading RAM in the near future, to the MAX of 128GB.

Until the upgrade, I am moving SWAP to the fastest, and least used disk, to hopefully keep things stable.

I'm pretty sure you are barking down the wrong path and have not accomplished anything.

You may be right, but I can`t imagine that moving SWAP off my slower RAID, 7200RPM disks to a fast 15k SAS disk would do nothing, especially when it comes to SWAP being used regularly.

 

[danb35]

There should be zero write or reads happening to the swap on zfs

Tell that to my 128 GB system. It's been some time since this was the case.

 

[SweetAndLow]

If this is all you then there needs to be a FreeNAS bug filed to move swap from being on all disks to just one disk or possibly not destroy things when a disk dies that has swap on it.

My system has never seen swap usage in 5y. I don't run vms though. Only jails.

 

[danb35]

I don't run VMs either, and my system regularly uses a bit of swap. Not a lot (currently 4.9M), but a bit. And there's already a bug (due with 11.1, I think) to implement mirrored swap, which would have a pretty similar effect in terms of a dead disk not killing the system.

 

[Ericloewe]

Tell that to my 128 GB system. It's been some time since this was the case.

Sure, but you don't see 30% usage, which sounds crazy high to me. You get a bit here and there from the memory manager's somewhat excessive latency, right?

Edit: Answered above

 

[gpsguy]

... or possibly not destroy things when a disk dies that has swap on it.
.

11.1 will mirror the swap file.

 

[siconic]

Sure, but you don't see 30% usage, which sounds crazy high to me.

So 30% plus is high then. If I am over utilizing RAM, this very likely could be the reason my SWAP is so high, correct?

 

[Ericloewe]

Yes, my first guess is too much RAM dedicated to VMs in relation to physical memory size. Your options are:

1. Tune it and eat the hit in ARC size
2. Add more RAM
3. Consolidate VMs and/or assign less RAM

 

[Chris Moore]

You could just buy a new system with 512GB of RAM.

 

[danb35]

You could just buy a new system with 512GB of RAM.

Why stop there?

 

[siconic]

Lets shoot for 1TB while we are at it. With 11 12TB helium disks....

 

[Chris Moore]

Lets shoot for 1TB while we are at it. With 11 12TB helium disks....

Best to stay clear of the latest technology. Give it some time for them to iron out the bugs. Go with 8TB drives.