11.1-U5 Laundry Memory over 10G

[wgreenway]

I thought I had the issue with swap growth licked after making some adjustments but now the main processes seem to be eating tons of memory for no reason (it seems to be happening when the system is idle).

Check this:

Why is the GUI webserver eating so much swap?

And 10Gigs of Laundry? Is this normal? I'm not doing anything that weird on this machine... running plex and resilio sync to backup my stuff.

 

[joeschmuck]

First of all, are you experiencing any problems running FreeNAS? I don't see anything wrong with your screen shot. You may desire to add more RAM so you stop using SWAP space.

Second, trying to understand how memory is utilized in FreeBSD is a difficult thing and there is a lot of faith if you don't want to dive into how FreeBSD operates. Have you looked up what Laundry Memory is? why are you concerned about it? It's just dirty RAM that needs to be cleaned up (erased or zeroed I believe) before it can we used again. Why they used "Laundry" as it's name, who knows. I'm just surprised this is even tracked.

When you ask questions like these you need to provide your system makeup and specs per the forum rules. Stuff like the version of FreeNAS, how much RAM you have, your pool/vdev info, etc... However with respect to thsi specific question type, you should be doing some searching in the forums and FreeBSD forums because trying to understand how memory works is very confusing. I'm not going to pretend I know how it works, I have to investigate it each time I want to know something and then hope I can understand it.

Sorry if I sound a bit negative, not my intention. My intention is that you should do a little searching for "freebsd laundry memory" and read up on it and try to understand it. And that trying to understand how the memory stuff is managed is very complex and you will need a lot of research to understand it.

 

[wgreenway]

I did search on this. My issue is before I increased the swap, the machine was going into swap death... the defaults when the machine was configured by the installer only allocated 4GB... At the point it is now, it would have died if I hadn't increased the allocation. Regardless of config, I didn't think the question was so unreasonable... i.e. is using 65% of the available ram in a box for "cleanup of page blocks" a normal behavior? If the machine will function normally without fussing with it... I'm good. I have unix boxes at work that run heavy loads with YEARS of uptime without being jacked with. This box was a vanilla installation, and it's crashed hard several times with it just sitting and idling with light fileserver use.

It's already got 16GB of ECC memory in it... shouldn't that be enough for a light load? I don't want to spend another $150 on ram if this just a tuning setting.

 

[joeschmuck]

It's already got 16GB of ECC memory in it... shouldn't that be enough for a light load?

Light load is a subjective term where I'd associate it with CPU resourses however RAM utilization is a different resource. You could install 20 jails that have a light load but eat up your RAM. So you currently have 3 jails running, Plex, Rsync (or similar), squid ? If i'm thinking about the same person, sorry if I got this wrong. Whatever the situation while your jails may not need much CPU horsepower they do need RAM.

Regardless of config, I didn't think the question was so unreasonable...

Nope, not unreasonable at all. I just wanted you to know that Laundry is not a typical item to be concerned about, it's a housekeeping chore. However knowing your configuration helps us provide specific details in our answers such as if you had only 8GB of RAM we could say you have a shortage or if we know you have 64GB RAM we could say that the system isn't recognizing all your RAM or maybe that it's looking great.

I don't want to spend another $150 on ram if this just a tuning setting.

I fully understand that, I'd do the same thing and try tuning first.

So my opinion is that the Laundry value is fine, it's a housecleaning function. And I'm glad you are taking an interest in the finer details of your system and how it operates. Knowledge is power.

And just so that you know, I do my best not to talk down to people, that is not very helpful so if I come off like that, please understand that it is unintentional.

I hope your system remains stable.

 

[wgreenway]

@joeschmuck Yes I am the person with 3 jails, and I completely disabled squid... which was a big hog... but that's why when the swap utilization continued to climb I am just trying to get my head wrapped around what's going on. So, yesterday... laundry grew to 12G and ate 9 gigs of swap. Now, it's down to 5G... but it's using 10gig of swap. If anything, I've pared back on things using memory and disabled unused or hoggy services (like the nginx + php that was running in the one jail along with squid. I was using webmin to configure squid and other stuff in there.)

I've ordered the 16G stick... but I really don't think it should NEED it. I hate throwing $$$ at a problem...

Here's the system break down:
Base Board Information
Manufacturer: ASUSTeK COMPUTER INC.
Product Name: PRIME X470-PRO
Handle 0x002F, DMI type 19, 31 bytes
Memory Array Mapped Address
Starting Address: 0x00000000000
Ending Address: 0x003FFFFFFFF
Range Size: 16 GB
Physical Array Handle: 0x002E
Partition Width: 1

Handle 0x0033, DMI type 4, 48 bytes
Processor Information
Socket Designation: AM4
Type: Central Processor
Family: Zen
Manufacturer: Advanced Micro Devices, Inc.
ID: 82 0F 80 00 FF FB 8B 17
Signature: Family 23, Model 8, Stepping 2
Version: AMD Ryzen 5 2600 Six-Core Processor
Voltage: 1.1 V
External Clock: 100 MHz
Max Speed: 3900 MHz
Current Speed: 3400 MHz
Status: Populated, Enabled
Upgrade: Socket AM4


Handle 0x003B, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x002E
Error Information Handle: 0x003A
Total Width: 128 bits
Data Width: 64 bits
Size: 16384 MB
Form Factor: DIMM
Set: None
Locator: DIMM_B2
Bank Locator: BANK 3
Type: DDR4
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 2400 MT/s
Manufacturer: Kingston
Serial Number: EB0868FC
Asset Tag: Not Specified
Part Number: 9965669-027.A00G
Rank: 2
Configured Clock Speed: 1200 MT/s
Minimum Voltage: 1.2 V
Maximum Voltage: 1.2 V
Configured Voltage: 1.2 V

# swapinfo
Device 1K-blocks Used Avail Capacity
/dev/md99 33554432 10046984 23507448 30%

last pid: 64294; load averages: 1.49, 1.14, 1.04 up 2+04:08:36 12:31:31
77 processes: 1 running, 76 sleeping
CPU: 2.2% user, 0.0% nice, 3.1% system, 0.1% interrupt, 94.7% idle
Mem: 1864M Active, 4214M Inact, 5462M Laundry, 4032M Wired, 273M Free
ARC: 1785M Total, 1154M MFU, 321M MRU, 19M Anon, 24M Header, 268M Other
339M Compressed, 1324M Uncompressed, 3.91:1 Ratio
Swap: 32G Total, 9811M Used, 22G Free, 29% Inuse

PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME WCPU COMMAND
12811 root 1 20 0 12296K 0K 5604K pause 9 0:00 0.00% <nginx>
5337 btsync 23 24 0 19355M 10183M 0K uwait 0 35.0H 47.64% rslsync
5369 972 13 52 15 131M 48916K 0K piperd 3 2:49 0.08% Plex Script Host
64269 root 1 20 0 7940K 3948K 0K CPU0 0 0:00 0.05% top
5374 972 15 21 0 135M 29020K 0K usem 3 1:13 0.04% Plex DLNA Server
5113 972 27 20 0 408M 171M 0K uwait 3 8:42 0.03% Plex Media Server
63766 972 10 52 0 75176K 44192K 0K piperd 3 0:01 0.03% Plex Script Host
63727 972 10 52 0 73640K 43800K 0K piperd 8 0:01 0.03% Plex Script Host
63722 972 10 52 0 67880K 36548K 0K piperd 2 0:01 0.03% Plex Script Host
63661 972 10 52 0 78760K 50052K 0K piperd 5 0:01 0.03% Plex Script Host
63690 972 10 52 0 81832K 46264K 0K piperd 0 0:02 0.02% Plex Script Host
63709 972 10 52 0 83368K 55860K 0K piperd 6 0:03 0.02% Plex Script Host
5375 972 11 20 0 31764K 6140K 0K usem 10 0:03 0.00% Plex Tuner Service
3155 root 8 20 0 44092K 6576K 0K select 3 2:15 0.00% rrdcached
3116 root 1 20 0 70552K 46680K 0K select 10 0:34 0.00% nmbd

# iocage df
+--------------------------+-------+------+------+-------+-------+
| NAME | CRT | RES | QTA | USE | AVA |
+==========================+=======+======+======+=======+=======+
| DNSMasq | 2.26x | none | none | 4.89G | 12.9T |
+--------------------------+-------+------+------+-------+-------+
| plexmediaserver-plexpass | 1.05x | none | none | 52.2G | 12.9T |
+--------------------------+-------+------+------+-------+-------+
| rslsync | 1.32x | none | none | 2.13G | 12.9T |
+--------------------------+-------+------+------+-------+-------+

POOL INFO

DISK CONFIG
=> 40 488397088 nvd0 GPT (233G)
40 1024 1 freebsd-boot (512K)
1064 488396056 2 freebsd-zfs (233G)
488397120 8 - free - (4.0K)

=> 40 7814037088 da0 GPT (3.6T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 7809842688 2 freebsd-zfs (3.6T)
7814037120 8 - free - (4.0K)

=> 40 7814037088 da1 GPT (3.6T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 7809842688 2 freebsd-zfs (3.6T)
7814037120 8 - free - (4.0K)

=> 40 7814037088 da2 GPT (3.6T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 7809842688 2 freebsd-zfs (3.6T)
7814037120 8 - free - (4.0K)

=> 40 7814037088 da3 GPT (3.6T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 7809842688 2 freebsd-zfs (3.6T)
7814037120 8 - free - (4.0K)

=> 40 7814037088 da4 GPT (3.6T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 7809842688 2 freebsd-zfs (3.6T)
7814037120 8 - free - (4.0K)

=> 40 7814037088 da5 GPT (3.6T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 7809842688 2 freebsd-zfs (3.6T)
7814037120 8 - free - (4.0K)

=> 40 7814037088 da6 GPT (3.6T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 7809842688 2 freebsd-zfs (3.6T)
7814037120 8 - free - (4.0K)

=> 40 7814037088 da7 GPT (3.6T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 7809842688 2 freebsd-zfs (3.6T)
7814037120 8 - free - (4.0K)

 

[wgreenway]

@joeschmuck maybe just a coincidence, I saw something in the forums about "autotune" causing bad behavior. I unclicked the box and restarted the machine. Now, the machine has rrun 10 hours since, laundry is 0kb ... it hasn't used one fricken byte of swap all day... seriously? It's good... but... damn.

 

[joeschmuck]

Oh, you had autotune turned on? Yea, that feature was good in older versions of FreeNAS but in the more current versions it seems to cause more problems than solutions. I didn't even think to ask you about it just becasue most people who turn it on tell you that it in conversation and expecting super performance. Keep in mind that Laundry being used isn't a bad thing however if you are using a lot of Swap then that is a problem. I really hope it all works out and I'd still add that extra 16GB stick and then enable the squid jail, that would really help squid out quite a bit. So if you do desire to tune your system, read up on that feature before using it and how to customize your settings. I'd only do one or two changes at a time so you can catch anything that causes a problem.

Good luck!