/var filled up, Samba to blame

[jasonlitka]

I just had my primary FreeNAS box at work send me a bunch of error emails about not being able to generate traffic graphs or rotate logs. I logged in via SSH and saw that /var was full. Around 50% of the space used was from Samba tdb files.

brlock.tdb - 13MB
locking.tdb - 47MB
messages.tdb - 14MB

Any way to prevent this?

 

[jasonlitka]

I just had this happen again. This time it was just brlock.tdb & locking.tsb in samba, but I'm noticing that the rrd graphs are also burning 40% of /var.

It comes down to this, they're simply storing too much data on the tiny /var partition that comes with FreeNAS.

 

[jasonlitka]

... and full again.

Can anyone tell me if it's possible to move /var to a new volume? This is ridiculous.

 

[jasonlitka]

I moved the rrd data from /var/db/collectd/rrd to a folder in a ZFS pool and added in a symlink. That bought me back ~50MB. I guess I should try to move the samba tdb files as well since those are the next largest offender... Maybe I'll do that tonight.

 

[jasonlitka]

Alright, I moved the TDB files before I left the office yesterday and once again, by this morning, /var was full, this time due to /var/log. I've now moved /var/log, /var/db/collectd/rrd/localhost, and /var/db/samba to a zfs pool where they can't cause any damage.

The devs really need to rethink the tiny /var volume. It's not suitable for a system with a bunch of CPUs/network interfaces (each one burns 300-800KB for the RRD) or a large user base (where the samba tdb files grow rapidly). If they don't want to make it larger to avoid potentially breaking systems by resizing partitions and file systems then they should at least offer a built in option for "busy" systems where the data can be moved elsewhere, either to a storage device or to spare space on the boot volume.

 

[paleoN]

If they don't want to make it larger to avoid potentially breaking systems by resizing partitions and file systems then they should at least offer a built in option for "busy" systems where the data can be moved elsewhere, either to a storage device or to spare space on the boot volume.

Well /var is a md backed ramdisk. If you have available RAM it's simple enough to enlarge it. In your case it sounds like you might be better off with a persistent /var. This requires a separate UFS formatted drive, but works around the other issues.

 

[jasonlitka]

Ok, could you explain how to do that then? I'm a linux guy so md to me means software RAID and I assumed, without looking into it at all, that it was a device made to allow for expansion to a second boot device in a future version (since mention of this was made a long time ago).

 

[paleoN]

Ok, could you explain how to do that then?

Code:

mountrw /

vi /conf/base/var/md_size

shutdown -r now

 

[jasonlitka]

I think that will do it. I just tested out doubling that value on my backup box and it came up twice the size after a reboot.

I'll do the same think on my production box this weekend. Thanks for your help.

EDIT: Any idea if this will persist after a version update?

 

[titan_rw]

No, any changes made to files in /conf will not survive version upgrades. You'll have to 'tweak' the new version after applying the upgrade.

 

[jasonlitka]

Ok, so an upgrade then will require two reboots, one for the upgrade and one for the config change. That sucks. My main box takes 30 minutes to boot.