A new message with 11.1-RC1 and now 11.1, 11.1-U1, 11.1-U2, too

[Redcoat]

I haven't seen this one before:
"dellfreenas daemon[3282]: 2017/11/01 10:17:08 [WARN] agent: Check 'service:nas-health' is now warning"
Anyone have any ideas? I searched but didn't turn up anything.

 

[Alecmascot]

me too.....
Repeats every 2 minutes or so....

After RTFM I found this :

"A nas-health service is registered with Consul. This service runs /usr/local/etc/consul-checks/freenas_health.sh periodically, currently every two minutes. If an alert marked WARNING or CRITICAL is found, the nas-health service is marked as “unhealthy”, triggering consul-alerts to notify configured alert services."

Looks like it is not setup correctly :

2017/11/01 15:11:33 [WARN] agent: check "service:nas-health" has the 'script' field, which has been deprecated and replaced with the 'args' field.

 

[Redcoat]

2017/11/01 15:11:33 [WARN] agent: check "service:nas-health" has the 'script' field, which has been deprecated and replaced with the 'args' field.

Thanks. Which FM BTW - FreeBSD?

Now I did find a bug report that relates to the note above but I did not connect them as it looks like I should have - https://bugs.freenas.org/issues/26355
https://bugs.freenas.org/issues/26355
I also now see the warning you pasted as the first warning in the logs after the reboot.

Thanks for the helpful comments.

 

[Alecmascot]

Thanks. Which FM BTW - FreeBSD?

Now I did find a bug report that relates to the note above but I did not connect them as it looks like I should have - https://bugs.freenas.org/issues/26355
I also now see the warning you pasted as the first warning in the logs after the reboot.

Thanks for the helpful comments.

It's actually in the FreeNAS manual

 

[Redcoat]

It's actually in the FreeNAS manual :)

Thx. I finally found it - I had used "search" for "nas-health" initially and found nothing, forgetting that the search feature in the manual may be a bit "weak".

 

[Architct]

I want to put it out there that I was able to upgrade (via GUI) from 9.10.2-U6 to 11.1-RC1 without a hitch. But I also have this wonderful non-warning warning in /var/log/messages ... every two minutes. Otherwise, brilliant that the upgrade worked and looking forward to the real release Release and subsequent 11.2 with iocage via GUI.

 

[Keno5net]

Could this be because I haven't upgraded ZFS to the new feature flag level?

 

[Redcoat]

No, it's a bug and to be fixed in the next release, see message #3 above.

 

[AlexF]

(As per 5.10. Alert Services) doesn't it make sense that "nas-health" service only run if I have added at least one Alert Service and not at all if I have none?

 

[rogerh]

(As per 5.10. Alert Services) doesn't it make sense that "nas-health" service only run if I have added at least one Alert Service and not at all if I have none?

It makes reasonable sense to me, but 5.10 doesn't actually say that the service is only enabled when you use it for an alert service. Maybe the developers prefer it to run anyway, for some reason. Could it be part of the alert service in either the old or the new GUI? I haven't looked into this myself.

 

[Mamdoh]

I started to see the same message after upgrading from 11.1 to 11.1-U1. Every two minutes.

Code:

Jan 19 01:32:25 NAS daemon[3288]:	 2018/01/19 01:32:25 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 01:34:26 NAS daemon[3288]:	 2018/01/19 01:34:26 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 01:36:26 NAS daemon[3288]:	 2018/01/19 01:36:26 [WARN] agent: Check 'freenas_health' is now warning

I tried the following:

1)

Code:

service consul stop
rm -rf /var/db/system/consul
service consul start

2)
Restarted the system a couple of times.

I saw something about updating /usr/local/etc/consul.d/freenas.json. Mine currently looks like this (no modifications):

Code:

{																			 
  "disable_update_check": true,												
  "service": {																
	"name": "nas-health",													 
	"tags": ["primary"],														
	"address": "",															
	"port": 80,																
	"enableTagOverride": false,												
	"check": [																
	  {																		
		"id": "freenas_health",												
		"name": "FreeNAS health check",										
		"args": [ "/usr/local/etc/consul-checks/freenas_health.sh" ],		 
		"interval": "120s"													
	  }																		
	]																		 
  }																			
}

I opted not to do anything about that, for now, since I have the latest stable version of FreeNAS and they look almost identical to me, apart from this extra segment.

Code:

"disable_update_check": true,

Any suggestions/fixes, apart from stopping the consul service altogether? Thanks in advance!

 

[Redcoat]

Yes, I noticed that it hasn't yet been fixed. I have no suggestion other than to stop the consul service.

 

[hervon]

Same bug appeared going from 11.1 to 11.1U1. For me 11.1 did not have this bug.

 

[Mamdoh]

Yes, I noticed that it hasn't yet been fixed. I have no suggestion other than to stop the consul service.

@hervon: Thanks for confirming the emergence of this issue with 11.1-U1.
@Redcoat: I see. Somehow stopping a service completely, even if it's for a few weeks, makes me a bit nervous; especially that I don't know the inners of the software > Regular user! I'm not even sure if it's a big issue to have this message appear every two minutes in my log or if I should worry about it. Input from the FreeNAS experts would be much appreciated.

However, I kinda chose a halfway solution for now. I edited /usr/local/etc/consul.d/freenas.json and changed the Interval field from 120s to 1800s.

Code:

{																			
  "disable_update_check": true,												
  "service": {																
	"name": "nas-health",													
	"tags": ["primary"],														
	"address": "",															
	"port": 80,																
	"enableTagOverride": false,												
	"check": [																
	  {																		
		"id": "freenas_health",												
		"name": "FreeNAS health check",										
		"args": [ "/usr/local/etc/consul-checks/freenas_health.sh" ],		
		"interval": "1800s"													
	  }																		
	]																		
  }																			
}

Then executed the following:

Code:

service consul stop
rm -rf /var/db/system/consul
service consul start

The result is as follows:

Code:

Jan 19 05:27:07 NAS daemon[13459]:	 2018/01/19 05:27:07 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 05:57:07 NAS daemon[13459]:	 2018/01/19 05:57:07 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 06:27:08 NAS daemon[13459]:	 2018/01/19 06:27:08 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 06:57:09 NAS daemon[13459]:	 2018/01/19 06:57:09 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 07:27:10 NAS daemon[13459]:	 2018/01/19 07:27:10 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 07:57:10 NAS daemon[13459]:	 2018/01/19 07:57:10 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 08:27:11 NAS daemon[13459]:	 2018/01/19 08:27:11 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 08:57:12 NAS daemon[13459]:	 2018/01/19 08:57:12 [WARN] agent: Check 'freenas_health' is now warning
Jan 19 09:27:12 NAS daemon[13459]:	 2018/01/19 09:27:12 [WARN] agent: Check 'freenas_health' is now warning

This solution doesn't survive a reboot though.

 

[Redcoat]

It's interesting that @dlavigne changed the status on https://redmine.ixsystems.com/issues/26355 to "Resolved" a few hours ago.

 

[GreyMatters]

11.1-U1 just upgraded here. Log spammed with "[WARN] agent: Check 'freenas_health' is now warning" .

Do this means that every 11.1-U1 installation have this problem? How it's even possible?

 

[cygnus]

I just upgraded and have this also! So annoying has anyone tried rolling back?

 

[wackymole]

"Check 'freenas_health' is now warning". AMD here... spamming my logs

 

[Redcoat]

Yes - I rolled one machine back to 11.0-U4.

 

[cygnus]

Yes - I rolled one machine back to 11.0-U4.

I did to and the messages seemed to have stopped.