snmp weirdness as of FreeNAS-9.3-STABLE-201602031011

[estrabd]

I update a couple of days ago, and I am getting some weird stoppages in snmp reporting. It's happening at the same time both inside of the FreeNAS dashboard's own reporting tool (first 2) and the last through an outside observium server that I have keeping an eye on things. It was working unfailingly until the recent update.

I should note the functioning of the server is fine during this time. It's only consumers of snmp that are showing the disruption.


TIA

 

[dlavigne]

Anything in /var/log/messages during the time period the blip occurs?

 

[estrabd]

Good suggestion. Here you go.

 

[dlavigne]

Looks like a result of https://bugs.freenas.org/issues/8109. Note that there is another bug regarding the suggested workaround: https://bugs.freenas.org/issues/13551.

 

[estrabd]

Do these workaround flags in https://bugs.freenas.org/issues/13551 just quiet the log spam or fix the disruption in snmp data? It seems like https://bugs.freenas.org/issues/13551 is a report that simply resetting a tunable could result in deplicate entries in /etc/rc.conf.freenas ... which is not my current issue.

 

[dlavigne]

Do these workaround flags in https://bugs.freenas.org/issues/13551 just quiet the log spam or fix the disruption in snmp data?

Possibly, if the blip is due to the SPAM of error messages. I posted the second bug as the suggested workaround in the first bug (setting that tunable) results in two tunables meaning that the workaround doesn't quite work.

 

[estrabd]

Thank you. I have set the flags and restarted snmpd. We shall see! Thank you.

 

[estrabd]

The issue persists and I am still getting the log spam; but nothing is pointing to why snmpd is dying.

 

[estrabd]

Some more details:

1) the internal reporting feature seems to not be disrupted now
2) external reporting is still disrupted

One last note, I have auxiliary parameters set, as shown in the attachment.

 

[estrabd]

I have rolled back to 201601181840 and so far all seems right. I also noticed that with 201602031011, the webgui was much less responsive. I hope that I am not stuck on 201601181840 because of this.

 

[estrabd]

After combing the github repo, the only commits between 201601181840 and 201602031011 I can see that have to do with any kind of logging are the version bump and subsequent version bump of syslog-ng:

(update of syslog-ng) https://github.com/freenas/freenas/commit/d2ad8087cc04a982ab11569412532bec63c633f9
(revert of update of syslog-ng) https://github.com/freenas/freenas/commit/f028b11012142959ee0a64d3d4a4408f630db691

Would the upgrade, then downgrade of syslog-ng between 201601181840 and 201602031011 have affected anything?

Furthermore, I can confirm that as I am on 201601181840, syslog-ng is sitting at 3.6.4.

$ syslog-ng --version
syslog-ng 3.6.4
Installer-Version: 3.6.4
Revision:
Compile-Date: Jan 18 2016 19:10:52
Available-Modules: linux-kmsg-format,system-source,dbparser,syslogformat,afsocket,afprog,afuser,graphite,csvparser,basicfuncs,affile,json-plugin,cryptofuncs,afsocket-tls,afsocket-notls,pseudofile,confgen,afmongodb,afamqp,afstomp
Enable-Debug: on
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: off
Enable-TCP-Wrapper: off
Enable-Linux-Caps: off

 

[ovizii]

Just subscribing to this as I have added my freeNAS to my Observium server today and was surprised to see it going down every 20 minutes or so :-(
Btw. does data collection via SNMP also stop for the period that its marked as down?