tmpfs /var/db/collectd/rrd 100% full -- throwing errors

[Tekz]

Hi all,

I noticed that the console on one of my FreeNAS servers has started spitting out errors "No space left on device". Pretty clear that it's because the tmpfs /var/db/collectd/rrd filesystem is 100% full.


root@hostname:~ # df -h
Filesystem Size Used Avail Capacity Mounted on
freenas-boot/ROOT/11.1-U2 26G 888M 25G 3% /
devfs 1.0K 1.0K 0B 100% /dev
tmpfs 32M 10M 22M 33% /etc
tmpfs 4.0M 8.0K 4.0M 0% /mnt
tmpfs 21G 121M 21G 1% /var
freenas-boot/grub 25G 6.8M 25G 0% /boot/grub
fdescfs 1.0K 1.0K 0B 100% /dev/fd
..
dmvol/.system 7.3T 509M 7.3T 0% /var/db/system
dmvol/.system/cores 7.3T 23M 7.3T 0% /var/db/system/cores
dmvol/.system/samba4 7.3T 11M 7.3T 0% /var/db/system/samba4
dmvol/.system/syslog-a75e42d926e4418a8e21357ce29c963c 7.3T 156M 7.3T 0% /var/db/system/syslog-a75e42d926e4418a8e21357ce29c963c
dmvol/.system/rrd-a75e42d926e4418a8e21357ce29c963c 7.3T 96K 7.3T 0% /var/db/system/rrd-a75e42d926e4418a8e21357ce29c963c
dmvol/.system/configs-a75e42d926e4418a8e21357ce29c963c 7.3T 53M 7.3T 0% /var/db/system/configs-a75e42d926e4418a8e21357ce29c963c
tmpfs 600M 600M 0B 100% /var/db/collectd/rrd
root@conf-dm-free1:~ # zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
dmvol 10.9T 3.26T 7.62T - 18% 29% 1.00x ONLINE /mnt
freenas-boot 29.8G 4.02G 25.7G - - 13% 1.00x ONLINE -
root@hostname:~ #


root@hostname:~ # tail /var/log/messages
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180612.2100-2w): No space left on device
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180612.2000-2w): No space left on device
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180612.2000-2w): No space left on device
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180612.1900-2w): No space left on device
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180612.1900-2w): No space left on device
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180612.1900-2w): No space left on device
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180612.1800-2w): No space left on device
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180612.1800-2w): No space left on device
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180612.1800-2w): No space left on device
Jun 13 16:06:48 hostname collectd[5237]: check_create_dir: mkdir (/var/db/collectd/rrd/hostname.my.domain.com/df-mnt-dmvol-dmdata-.zfs-snapshot-auto-20180613.1300-2w): No space left on device
root@hostname:~ #

Server has plenty of space left, as you can see. Any recommendations for clearing this error?

 

[dlavigne]

Have it record to the system dataset instead: http://doc.freenas.org/11/system.html#system-dataset.

 

[Tekz]

Ah, I see. So the tempfs is just a RAM disk holding the reporting database and it filled up. I hit the checkbox on the reporting database, saved, and now no more errors. Thanks!