Nagios notification when no message is received within 48 hours - nagios

Question

In Nagios it is easy to check that a LogMessage happened in the last 48 hours and sound an alarm. What I would like, though, is to instead configure Nagios to sound an alarm when a specific message did not occur within 48 hours.
Can anyone point me in the right direction?
I am using the "Check WMI Plus" plugin (no agent required) in order to check the event log on a windows box.

Answer 1

Without knowing what your exact "specific message" is, it's hard to give a specific answer, but we can do this:
I'm going to raise a CRITICAL event when I haven't seen a "processing of windows Group Policy failed" error or warning event in the last 48 hours.
You use the -w and -c options to define criteria for WARNING and CRITICAL events in check_wmi_plus.
From check_wmi_plus.pl --help | less -i we get the help and we can find the checkeventlog options.
There are two tricks:
checkeventlog only has one field _ItemCount, so you don't need to specify it
You want to specify a
range of values, that includes only 0 - so use #0:0
First, define a specific section in the events.ini file. Mine is: /opt/nagios/bin/plugins/check_wmi_plus.d/events.ini
I added this:
[eventSpecial]
im=Group Policy failed
I added that just below the [eventdefault] section.
Basically, the im= means 'include message' - if it's not specified everything is included, so by specifying it, I've said "only include messages that match this regular expression."
Then you need the command for checkeventlog
I use:
/opt/nagios/bin/plugins/check_wmi_plus.pl -H HOST -u USER -p PASS -m checkeventlog -a % -o 2 -3 48 -4 eventSpecial -c #0:0
So for the optional arguments (again with the --help option):
-a % == search all event logs
-o 2 == Warning and error severity only
-3 48 == last 48 hours
-4 eventSpecial == refer to the section in events.ini that I just created
-c #0:0 == raise a CRITICAL if there are exactly 0 occurances
With this command, if there ARE messages during the period, I get:
OK - 3 event(s) of Severity Level: "Error,Warning", were recorded in
the last 48 hours from the % Event Log. (List is on next line. Fields
shown are -
Logfile:TimeGenerated:SeverityLevel:EventId:Type:SourceName:Message)|'Event
Count'=3;0;
System:20130604195600.378642-000|Error:1129:0:Microsoft-Windows-GroupPolicy:The processing of Group Policy failed because of lack of network
connectivity to a domain controller. This may be a transient
condition. A success message would be generated once the machine gets
connected to the domain controller and Group Policy has succesfully
processed. If you do not see a success message for several hours, then
contact your administrator.
System:20130604055521.084809-000|Error:1129:0:Microsoft-Windows-GroupPolicy:The processing of Group Policy failed because of lack of network
connectivity to a domain controller. This may be a transient
condition. A success message would be generated once the machine gets
connected to the domain controller and Group Policy has succesfully
processed. If you do not see a success message for several hours, then
contact your administrator.
System:20130603220259.894040-000|Error:1055:0:Microsoft-Windows-GroupPolicy:The processing of Group Policy failed. Windows could not resolve the
computer name. This could be caused by one of more of the following:
a) Name Resolution failure on the current domain controller. b)
Active Directory Replication Latency (an account created on another
domain controller has not replicated to the current domain
controller).
Which does not include a critical event.
If there are none, I get this:
CRITICAL - [Triggered by _ItemCount in the range 0:0] - 0 event(s) of
Severity Level: "Error,Warning", were recorded in the last 4 hours
from the % Event Log.|'Event Count'=0;0;
Which does include the critical event, because there were no entries in the log file to match my criteria.
And you can just define a standard Nagios command using the appropriate $USER8$ macros to include it in your configuration.

Answer 2

You should try this and create a simple DOS script that is kicked off every hour to monitor nagios and restart it when it see 2 nagios.exe. Here is the DOS script to kill the nagios.exe service and restart it.
-------- CheckNagios.bat --------
#echo off
set mypgm=nagios.exe
REM GET date/time stamp
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set mydate=%%c-%%a-%%b)
For /f "tokens=1-2 delims=/:" %%a in ('time /t') do (set mytime=%%a%%b)
:checkNagios
rem get number of nagios processes
for /f %%i in ('c:\windows\system32\tasklist.exe ^| find /i /c "%mypgm%"') do set /a numProc=%%i
echo Last Check: %mydate%_%mytime%
ECHO # of processes = %numProc%
if %numProc% GTR 1. (goto kill) else goto end
:kill
c:\windows\system32\taskkill.exe /f /IM %mypgm%
REM restart nagios
net start Nagwin_Nagios
REM restart other nagios processes
rem for /f %%x in ('net start ^| findstr /i "nagwin_"') do net stop %%x
:end
echo Exiting program.
echo =================
rem SCHEDULE TASK TO RUN EVERY HOUR and pipe to a logfile
rem SCHTASKS /create /TN "Check Nagios" /TR "c:\icw\bin\checkNagios.bat >> c:\checknagios.log 2>&1" /SC HOURLY /ST 16:00 /MO 1 /RU DOMAIN\USERNAME /RP PASSWORD
REM store last check that will be used by emailNagios.bat using blat.exe
set LAST_NAGIOS_CHECK=%%mydate%%_%%mytime%%