Jail memory allocation?

[Daisuke]

Can someone explain briefly how jail memory allocation works? Is there a way to allocate say 4GB to a jail? How do I see the memory consumption for a jail? In FreeBSD 12 we have rctl but is disabled into TrueNAS:

Code:

root@nas[~]# rctl
rctl: RACCT/RCTL present, but disabled; enable using kern.racct.enable=1 tunable

Not sure if this is recommended to enable it with a tunable, I rather follow your advice what is the proper way to allocate memory to a jail. Thank you for your help.

 

[Patrick M. Hausen]

You can. Sort of.

Resource control for jails does not work like you might expect from a hypervisor, because it's just normal processes on a single kernel. But all the knobs are there in iocage and the limits are enforced with a certain amount of elasticity.
When a process in a jail tries to allocate more memory than assigned to the jail, the request will not be denied but served from swap instead of main memory. But then you can limit swap, too. So it probably needs a bit of experimentation to find out what works.

The official documentation claims these features are not implemented. Which is wrong. They are there and they work.

E.g. to limit the memory use to 4G you can use iocage set memoryuse=deny=4G <jailname>.

You need the tunable in loader.conf. I really don't know if this has any negative impact on the TrueNAS system as a whole. We use it on stock FreeBSD in our hosting environment.

What precisely are you trying to achieve? If you are struggling with e.g. tomcat, redis or elastic going rogue, it might be better to limit these services via their own configuration.

 

[Daisuke]

@Patrick M. Hausen I have a media jail where files writing and memory usage spikes on a regular basis. The disk IO is not an issue, I'm running the jail pool on SSD. I want to make sure the jail has at least 2GB of reserved memory and be allowed to go up to 16GB if needed. I have 192GB of RAM installed on my TrueNAS 12.

 

[Patrick M. Hausen]

Reservation is not possible, only limitation. Try iocage set memoryuse=deny=16G <jailname>, then.

 

[Daisuke]

@Patrick M. Hausen In theory, the jail will get allocated the least amount of memory and if memory usage spikes, more will be allocated? I don't care if it uses more than 16GB, I just want to make sure by default it is not forced to max 1G, for example.

 

[Patrick M. Hausen]

My statement above limits the memory for the jail to 16G in total. I doubt any more "elastic" control is possible. If you don't set a limit the processes will allocate as much as they are allowed on an individual basis. Jails are all symmetric multiprocessing. You can set an upper limit and that is that as far as I know.

 

[Daisuke]

Related to limit, I set it directly into loader.conf?

I don't see a way to define the jail into TrueNAS menu:

If you don't set a limit the processes will allocate as much as they are allowed on an individual basis.

That is exactly the part I'm interested, what is the allowed value?

 

[Patrick M. Hausen]

That - again - depends. But first things first.

To set that value simply enter as a command on your TrueNAS:
iocage set memoryuse=deny=16G <jailname>

That sets and persists the limit. There is no GUI for that. You can see all values for your jail with:
iocage get all <jailname>

How much memory is available to a single process is dependent on certain kernel parameters set at process creation by the parent process. You can check the maximum values (for root) inside your jail by invoking a shell inside your jail via ssh or iocage console <jailname> and then:

Code:

root@freenas[~]# iocage console cloud
[...]
root@cloud:~ # limit
cputime      unlimited
filesize     unlimited
datasize     33554432 kbytes
stacksize    524288 kbytes
coredumpsize unlimited
memoryuse    unlimited
vmemoryuse   unlimited
descriptors  1883043
pseudoterminaunlimited
kqueues      unlimited
memorylocked unlimited
maxproc      63694
sbsize       unlimited
swapsize     unlimited

A process can limit itself to smaller values and this is frequently done at service startup. The startup process limits itself, then "forks" a child process that inherits these limits and cannot raise them again. The child runs in the background serving e.g. HTTP requests or whatever.
For example you can set an upper memory limit of 2G in the Elasticsearch config file. The Elasticsearch startup routine will then limit the memory to said 2G before creating the child process that does "Elasticsearch things".

To set anything smaller than "unlimited" on a system or user basis you can edit (inside the jail!) the file /etc/login.conf. After every change to that file you need to regenerate the database that is used by the system instead of the text file via: cap_mkdb /etc/login.conf.

Puzzled?

Yes, on a standard Unix system every process can gobble up all of the system's memory and there is nothing in place to prevent it from doing that. It's all about everyone playing nice. You can write a five-line program in C, start it, and your memory is gone ...