Not using all available free RAM

[The Gecko]

My FreeNAS server has 96GiB RAM. This image (above) shows the current steady-state with auto-tune disabled and zero Tunables. With auto-tune enabled, the average memory level at a steady-state for Wired memory is around 62 GiB, possibly lower. What Tunables must I create to allow the ARC & Wired memory to grow past 90GiB?

 

[mav@]

You should be careful going to the max. ZFS sometimes tend to allocate more then you say, that may cause swapping or allocation errors.

 

[zambanini]

what kind of files do you store on your pool?

 

[joeschmuck]

What version of FreeNAS are you running?

First, disable tunables and deleted any tunable variables you have. Then turn on tunables and reboot.

If you have enabled tunables and rebooted then you should have a variable called "vfs.zfs.arc_max" that is now current. Of course you can disable Autotune and then manually set this variable as well.

Be careful what you do and backup your current settings before proceeding. You could end up with an unstable system and be forced to default settings to recover, then have to load your last saved configuration data.

 

[The Gecko]

You should be careful going to the max. ZFS sometimes tend to allocate more then you say, that may cause swapping or allocation errors.

I'd like to see the utilization rise to 90GiB.

what kind of files do you store on your pool?

Several TB of ISO files ranging from 4GB to 40GB, application installation files, various Office documents, and a slew of virtual machine hard drives on an iSCSI LUN that is accessed by Hyper-V.

What version of FreeNAS are you running?

Build: FreeNAS-9.2.1.5-RELEASE-x64 (80c1d35)

First, disable tunables and deleted any tunable variables you have. Then turn on tunables and reboot.

This is what I first tried. After 24 hours, the amount of used RAM wouldn't rise much above 40GiB. I disabled autotune, removed all tunables, and rebooted. After 24 hours of banging on the system with autotune disabled and no tunables, RAM utilization ran upwards fast, but began to sawtooth between 70GiB and 72GiB, but no higher, with 20+GiB free. This is where I am now.

If you have enabled tunables and rebooted then you should have a variable called "vfs.zfs.arc_max" that is now current. Of course you can disable Autotune and then manually set this variable as well.

Currently, vfs.zfs.arc_max = 98940997632, which roughly equates to 92.15GiB.

 

[cyberjock]

You will *never* be able to use more than 7/8s of your RAM for ZFS. It's hard coded to prevent crashing your system.

Honestly, I think you are just barking up the wrong tree. The system will use it as appropriate and it is possible that some plugin or other service is doing something to end up where you are. It's also possible that you just don't have a need for it so ZFS won't allocate it. I think you should leave well enough alone and just use the box as-is. That's what I would do.

 

[zambanini]

@The Gecko

I took a look at one of our freenas 9.2.16 boxes with 96gb ram:

tunables are almost default,
vfs.zfs.arc_max is set to 89120571392