Not getting email alerts

Joe Rhett jrhett at meer.net
Wed Aug 25 05:47:55 CEST 2004


First point of review -- you won't get any more service alerts after the
host is down.  That makes sense.  So the problem lies in the host
notification.

On Tue, Aug 24, 2004 at 07:50:01PM -0700, Quanah Gibson-Mount wrote:
> --On Tuesday, August 24, 2004 6:55 PM -0700 Joe Rhett <jrhett at meer.net> 
> wrote:
> 
> >>It is willing and able to be notified 24x7.  This works as expected with
> >>all service notifications.
> >
> >Yes, but the host or service notification option was set to businesshours.
> >Why don't you try making them the same across the board?
> 
> Done, and Nagios restarted (I even restarted the entire system, just to 
> make sure there were no renegade nagios processes lurking about).  I'll 
> note that is how all of our production systems are configured, and they all 
> have this same issue.  I also brought the host back up prior to restarting 
> Nagios, so everything would be in an OK state:
> 
> [1093399400] HOST ALERT: ldap-dev1;UP;HARD;1;PING OK - Packet loss = 0%, 
> RTA = 2.00 ms
> [1093399400] HOST NOTIFICATION: 
> test-service;ldap-dev1;UP;notify-by-email;PING OK - Packet loss = 0%, RTA = 
> 2.00 ms
> [1093399401] GLOBAL HOST EVENT HANDLER: ldap-dev1;UP;HARD;1;notify-smarts
> [1093399402] SERVICE ALERT: ldap-dev1;Disk;UNKNOWN;SOFT;1;(No output!)
> [1093399402] GLOBAL SERVICE EVENT HANDLER: 
> ldap-dev1;Disk;UNKNOWN;SOFT;1;notify-smarts
> [1093399410] SERVICE ALERT: ldap-dev1;Disk;OK;SOFT;2;DISK OK [3875661 kB 
> (94%) free on /dev/dsk/c0t0d0s0] [0 kB (100%) free on fd] [0 kB (100%) free 
> on mnttab] [26776028 kB (99%) free on /dev/dsk/c0t0d0s4] [3965584 kB (99%) 
> free on swap] [34623920 kB (99%) free on /dev/dsk/c0t1d0s1] [953992 kB 
> (99%) free on /dev/dsk/c0t0d0s7] [1941152 kB (97%) free on 
> /dev/dsk/c0t0d0s6] [9000000 kB (100%) free on AFS
> [1093399410] GLOBAL SERVICE EVENT HANDLER: 
> ldap-dev1;Disk;OK;SOFT;2;notify-smarts
> [1093400129] HOST ALERT: ldap-dev1;UP;HARD;1;(Host assumed to be up)
> [1093400129] SERVICE ALERT: ldap-dev1;Disk;OK;HARD;1;DISK OK [3875661 kB 
> (94%) free on /dev/dsk/c0t0d0s0] [0 kB (100%) free on fd] [0 kB (100%) free 
> on mnttab] [26776012 kB (99%) free on /dev/dsk/c0t0d0s4] [3965824 kB (99%) 
> free on swap] [34623920 kB (99%) free on /dev/dsk/c0t1d0s1] [953992 kB 
> (99%) free on /dev/dsk/c0t0d0s7] [1941152 kB (97%) free on 
> /dev/dsk/c0t0d0s6] [9000000 kB (100%) free on AFS
> 
> 
> The host was then shut down.  I correctly get the first notification:
> 
> Notification Type: PROBLEM
> 
> Service:
> Service State:
> Host: ldap-dev1
> Host State: DOWN
> 
> Date/Time: Tue Aug 24 19:23:36 PDT 2004
> 
> Additional Info:
> 
> CRITICAL - Plugin timed out after 10 seconds
> 
> Logs show:
> 
> [1093400604] HOST ALERT: ldap-dev1;DOWN;SOFT;1;CRITICAL - Plugin timed out 
> after 10 seconds
> [1093400604] GLOBAL HOST EVENT HANDLER: ldap-dev1;DOWN;SOFT;1;notify-smarts
> [1093400616] HOST ALERT: ldap-dev1;DOWN;HARD;2;CRITICAL - Plugin timed out 
> after 10 seconds
> [1093400616] HOST NOTIFICATION: 
> test-service;ldap-dev1;DOWN;notify-by-email;CRITICAL - Plugin timed out 
> after 10 seconds
> [1093400616] GLOBAL HOST EVENT HANDLER: ldap-dev1;DOWN;HARD;2;notify-smarts
> [1093400617] SERVICE ALERT: ldap-dev1;Disk;UNKNOWN;HARD;1;(No output!)
> 
> 
> Now more than 20 minutes have passed.  No further alerts have been 
> received.  Nothing more has been entered in nagios.log.
> status.log shows that checks are happening regularly:
> 
> [1093402139] 
> HOST;ldap-dev1;DOWN;1093402104;1093400616;0;408976;201490;0;1093400616;1;1;1;1;1;0;0.00;0;1;1;CRITICAL 
> - Plugin timed out after 10 seconds
> [1093402139] 
> SERVICE;ldap-dev1;Disk;UNKNOWN;1/3;HARD;1093401865;1093402094;ACTIVE;1;1;1;1093400617;0;UNKNOWN;408653;201561;0;0;0;0;1;21;225;1;0;0.00;0;1;1;1;(No 
> output!)
> 
> 
> [1093402154] 
> HOST;ldap-dev1;DOWN;1093402104;1093400616;0;408976;201490;0;1093400616;1;1;1;1;1;0;0.00;0;1;1;CRITICAL 
> - Plugin timed out after 10 seconds
> [1093402154] 
> SERVICE;ldap-dev1;Disk;UNKNOWN;1/3;HARD;1093401865;1093402094;ACTIVE;1;1;1;1093400617;0;UNKNOWN;408653;201561;0;0;0;0;1;21;225;1;0;0.00;0;1;1;1;(No 
> output!)
> 
> 
> Updated configs:
> 
> define contact {
>    contact_name                    test-service
>    alias                           Test Service
>    service_notification_period     24x7
>    host_notification_period        24x7
>    service_notification_options    w,u,c,r
>    host_notification_options       d,u,r
>    service_notification_commands   notify-by-email
>    host_notification_commands      notify-by-email
>    email                           page-quanah
> }
> 
> define contactgroup {
>    contactgroup_name   ldap-qsys-admins
>    alias               LDAP Test Host Administrators
>    members             test-service
> }
> 
> define hostgroup {
>    hostgroup_name  ldap-qdev-group
>    alias           LDAP QDev Group
>    contact_groups  ldap-qsys-admins
>    members         ldap-dev1
> }
> 
> define host {
>    name                    ldap-qother-template
>    check_command           check-host-alive
>    max_check_attempts      2
>    notification_interval   2
>    notification_period     24x7
>    notification_options    d,u,r
>    notifications_enabled   1
>    register                0
> }
> 
> define host {
>    use         ldap-qother-template
>    host_name   ldap-dev1
>    alias       ldap-dev1
>    address     ldap-dev1.stanford.edu
>    parents     smr-a-3548
> }
> 
> define contactgroup {
>    contactgroup_name ldap-qapp-admins
>    alias             LDAP Administrators
>    members           test-service
> }
> 
> define service {
>    name                    ldap-qother-template
>    max_check_attempts      3
>    normal_check_interval   2
>    retry_check_interval    2
>    check_period            24x7
>    notification_interval   3
>    notification_options    w,u,c,r
>    notification_period     24x7
>    contact_groups          ldap-qapp-admins
>    register                0
> }
> 
> define service {
>    use                 ldap-qother-template
>    hostgroup_name      ldap-qdev-group
>    service_description Disk
>    max_check_attempts  3
>    check_command       check_disk!20%!10%
> }
> 
> 
> --
> Quanah Gibson-Mount
> Principal Software Developer
> ITSS/Shared Services
> Stanford University
> GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
> 
> 
> -------------------------------------------------------
> SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
> 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
> Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
> http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285
> _______________________________________________
> Nagios-users mailing list
> Nagios-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nagios-users
> ::: Please include Nagios version, plugin version (-v) and OS when 
> reporting any issue. ::: Messages without supporting info will risk being 
> sent to /dev/null

-- 
Joe Rhett
Senior Geek
Meer.net


-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null





More information about the Users mailing list