rrdcached - Could not read RRD file

[mrspudsalot]

Hey everyone, so I just updated from FreeNAS>9.2 to current FreeNAS (FreeNAS-11.2-U2.1 ), and everything in the update went smoothly except for one weird thing that is now spamming my terminal:

Mar 16 17:32:50 freenas rrdcached[2635]: handle_request_update: Could not read RRD file.

This has completely taken over my terminal, as it just spams the message non-stop. The system seems to work as normal. Aside from the spam, there doesn't seem to be any problems. I did a good amount of searching where I could and couldn't find anything. After some snooping I found the rrdcached folder in the `/var/db/collectd/rrd/freenas.local/rrdcached/` path, and it contains the following files:

• counter-journal-bytes.rrd
• counter-journal-rotates.rrd
• gauge-tree_depth.rrd
• gauge-tree_nodes.rrd
• operations-receive-flush.rrd
• operations-receive-update.rrd
• operations-write-data_sets.rrd
• operations-write-updates.rrd
• queue_length.rrd

Am I perhaps missing one? Should I replace one?

Hardware specs:

Processor:
Intel(R) Core(TM) i5-4460 CPU @ 3.20GHz (4 cores)

Memory:
16 GB

Motherboard:
Gigabyte GA-Z97X-UD3H-BK

 

[dlavigne]

Was that done in one update or several (eg 9.2 -> 9.3 -> 11.0 -> 11.1 -> 11.2)? If it was one, a change in RRD didn't get migrated.

What is the output of ls -l /var/db/collectd/rrd/freenas.local/rrdcached/ ?

Also, what are your settings for System -> System Dataset?

 

[mrspudsalot]

It was done in one update.

The output of this command is:

Code:

# ls -l /var/db/collectd/rrd/freenas.local/rrdcached/
total 1332
----rw-rw-  1 root  wheel  148648 Mar 19 20:29 counter-journal-bytes.rrd
----rw-rw-  1 root  wheel  148648 Mar 19 20:29 counter-journal-rotates.rrd
----rw-rw-  1 root  wheel  148648 Mar 19 20:29 gauge-tree_depth.rrd
----rw-rw-  1 root  wheel  148648 Mar 19 20:29 gauge-tree_nodes.rrd
----rw-rw-  1 root  wheel  148648 Mar 19 20:29 operations-receive-flush.rrd
----rw-rw-  1 root  wheel  148648 Mar 19 20:29 operations-receive-update.rrd
----rw-rw-  1 root  wheel  148648 Mar 19 20:29 operations-write-data_sets.rrd
----rw-rw-  1 root  wheel  148648 Mar 19 20:29 operations-write-updates.rrd
----rw-rw-  1 root  wheel  148648 Mar 19 20:29 queue_length.rrd

The settings of Settings -> System Dataset are:

1. My chosen System Dataset Pool
2. Syslog is checked
3. Reporting Database is not checked.

Thank you so much for your help. I really appreciate it.

 

[onryo]

I too have this issue.
Been running my FreeNAS for the past five years without incident. After upgrading to 11.2 I too have been getting spammed with

Code:

Mar 29 16:40:10 freenas rrdcached[2779]: handle_request_update: Could not read RRD file.
Mar 29 16:40:21 freenas rrdcached[2779]: handle_request_update: Could not read RRD file.
Mar 29 16:40:21 freenas rrdcached[2779]: handle_request_update: Could not read RRD file.
Mar 29 16:40:31 freenas rrdcached[2779]: handle_request_update: Could not read RRD file.
Mar 29 16:40:31 freenas rrdcached[2779]: handle_request_update: Could not read RRD file.
Mar 29 16:40:40 freenas rrdcached[2779]: handle_request_update: Could not read RRD file.
Mar 29 16:40:40 freenas rrdcached[2779]: handle_request_update: Could not read RRD file.

Everything seems to work fine but the spam is really annoying.

I don't have the exact path with "freenas.local" but I do have these two.

Code:

ls -l /var/db/collectd/rrd/localhost/rrdcached

counter-journal-bytes.rrd    gauge-tree_depth.rrd        operations-receive-flush.rrd    operations-write-data_sets.rrd    queue_length.rrd
counter-journal-rotates.rrd    gauge-tree_nodes.rrd        operations-receive-update.rrd    operations-write-updates.rrd

and

Code:

ls -l /var/db/collectd/rrd/freenas.com/rrdcached # ls
counter-journal-bytes.rrd    gauge-tree_depth.rrd        operations-receive-flush.rrd    operations-write-data_sets.rrd    queue_length.rrd
counter-journal-rotates.rrd    gauge-tree_nodes.rrd        operations-receive-update.rrd    operations-write-updates.rrd

Syslog has a Check
Reporting Database Not checked

Been updating the system at every stable release for the past five years. No gaps in the upgrades.


The system is running on E5 Xeon 2620v3 CPUs. x6 WD Reds in RAIDZ-2.

 

[Bashern]

Same issue here, updated from 11.2-U2.1 to 11.2-U3

 

[statik]

I get that same error spammed when I go to shut down. I have nothing past /var/db/collectd/

I upgraded from 11.1 to 11.2 U3.

What does this mean for me?

 

[cafmoms]

I had the same issue after upgrading from 9.3 to the current 11.2-U4.1.

Found a bug ticket here that they “couldn’t reproduce”, https://redmine.ixsystems.com/issues/82827

There is a work around in the ticket referencing /var/log... that didn’t seem right with the files and folders I was looking at, so I changed the work around to /var/db... and everything seems to be happy now. No longer getting spammed with errors in the console. This is what I did,

# service collectd stop
# service rrdcached stop
# rm -rf /var/db/collectd/rrd/*
# cd /var/db/collectd/rrd/
# mkdir localhost
# ln -s localhost freenas.local
# ln -s localhost [FQDN]
# service rrdcached start
# service collectd start

I hope this helps.

 

[PhiloEpisteme]

I'm seeing an issue similar to this where my system cannot completely boot; the UI is completely unaccesable. I was seeing errors as above and when I try the fix provided by @cafmoms and try to restart rrdcached I get

Code:

$ service rrdcached start
Starting rrdcached.
read_options: couldn't map "www" to a group, Sorry
/usr/local/etc/rc.d/rrdcached: WARNING: failed to start rrdcached

I have a scrub ongoing in case my boot pool has an issue that hasn't been caught yet; I need this system up soon so I may try reinstalling the OS which will preclude too much troubleshooting here.

I'm running 11.2-U6