Notifications stopped

[Andreas Ericsson]

Chris Wilson wrote:
> Hi all,
> 
> I have been using Nagios very successfully for a number of years.
> However, one instance has started behaving strangely in the last few
> weeks.
> 
> I first noticed it because notifications had stopped without any
> explanation. Restarting Nagios caused them to come back, and I thought
> little more of it until a week later, when it happened again.
> 
> I configured Nagios to log all output of the notify commands to a log
> file, for example:
> 
> define command{
>         command_name    notify-by-email
>         command_line    /usr/bin/printf "%b" 'This is an automated alert
> from the AidWorld website monitoring system\n(aka Nagios). The
> $SERVICEDESC$ on $HOSTALIAS$ is now $SERVICESTATE$.\n\nNotification
> Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState:
> $SERVICESTATE$\nAddress: $HOSTADDRESS$\nDate/Time:
> $DATETIME$\nAdditional Info:\n\n$OUTPUT$' | /bin/mail -s '[NAGIOS]
> Alert: $HOSTALIAS$ $SERVICEDESC$ is $SERVICESTATE$' $CONTACTEMAIL$
> 
>>>/usr/local/nagios/var/notify.out 2>&1
> 
>         }
> 
> The file notify.out was created, but nothing written to it.

The mail command doesn't print anything to stdout normally, and if 
nothing goes wrong it won't print to stderr either, so it's supposed to 
be empty.

> So I left
> nagios running... and again notifications stopped, about a week later.
> 

grep -i 'shutting down' nagios.cfg
It might have not dumped core and just shut down gracefully. If you're 
using CVS code a month or so old, that was the default behaviour.

> I couldn't find any errors in the notify.out file. nagios.log says that
> a notification was sent to each contact, but the system mail logs show
> no record of it. Mail configuration works, I can send email using
> /bin/mail and it does appear in /var/log/maillog.
> 
> How can I go about diagnosing this problem further? Could Nagios itself
> capture the standard output and error from notification plugins and log
> it to syslog or the Nagios log file?

It does that by default if the notify command returns non-zero.

> That would make diagnosing problems
> like this much easier, in my opinion.
> 
> Thank you all in advance for your help.
> 
> Cheers, Chris.

-- 
Andreas Ericsson                   andreas.ericsson at op5.se
OP5 AB                             www.op5.se
Lead Developer


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null