Scheduled checks falling far behind

[Litwin, Matthew]

On Oct 22, 2010, at 7:09 PM, Jonathan Angliss wrote:

On 10/22/10 19:29, Litwin, Matthew wrote:

--Service information--
Last Updated: Sat Oct 23 00:19:02 UTC 2010

--Service State Information--
Current Status:
  OK
 (for 7d 16h 14m 46s)
Status Information: CPU STATISTICS OK : user=0.12% system=0.00% iowait=0.00% idle=99.88%
Performance Data: 0.12;0.00;0.00;99.88;80;90
Current Attempt: 1/3  (HARD state)
Last Check Time: 10-23-2010 00:10:30<<<<
Check Type: ACTIVE
Check Latency / Duration: 612.633 / 2.052 seconds

This check latency is terribly high.  What is your overall latency look
like on the tactical overview page?  How many hosts and services are you
checking?

The doc on tuning has some hints on tweaking some settings that improve
these numbers, worth a quick read...

  http://nagios.sourceforge.net/docs/3_0/tuning.html

It's funny that the performance check output which I have been going by (foolishly) doesn't indicate any problems, and honestly doesn't really tell me anything useful.

Nagios Core 3.2.1
Copyright (c) 2009-2010 Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 03-09-2010
License: GPL

Website: http://www.nagios.org
Timing information on object configuration processing is listed
below.  You can use this information to see if precaching your
object configuration would be useful.

Object Config Source: Config files (uncached)

OBJECT CONFIG PROCESSING TIMES      (* = Potential for precache savings with -u option)
----------------------------------
Read:                 0.008913 sec
Resolve:              0.000526 sec  *
Recomb Contactgroups: 0.000075 sec  *
Recomb Hostgroups:    0.003510 sec  *
Dup Services:         0.024774 sec  *
Recomb Servicegroups: 0.048027 sec  *
Duplicate:            0.037472 sec  *
Inherit:              0.003409 sec  *
Recomb Contacts:      0.000000 sec  *
Sort:                 0.000001 sec  *
Register:             0.038721 sec
Free:                 0.003128 sec
                      ============
TOTAL:                0.168556 sec  * = 0.117794 sec (69.88%) estimated savings


RETENTION DATA TIMES
----------------------------------
Read and Process:     0.358370 sec
                      ============
TOTAL:                0.358370 sec


Timing information on configuration verification is listed below.

CONFIG VERIFICATION TIMES          (* = Potential for speedup with -x option)
----------------------------------
Object Relationships: 0.063165 sec
Circular Paths:       6.096125 sec  *
Misc:                 0.003806 sec
                      ============
TOTAL:                6.163096 sec  * = 6.096125 sec (98.9%) estimated savings


EVENT SCHEDULING TIMES
-------------------------------------
Get service info:        0.008075 sec
Get host info info:      0.000341 sec
Get service params:      0.000009 sec
Schedule service times:  0.007888 sec
Schedule service events: 0.055008 sec
Get host params:         0.000001 sec
Schedule host times:     0.000461 sec
Schedule host events:    0.010735 sec
                         ============
TOTAL:                   0.082518 sec


Projected scheduling information for host and service checks
is listed below.  This information assumes that you are going
to start running Nagios with your current config files.

HOST SCHEDULING INFORMATION
---------------------------
Total hosts:                     241
Total scheduled hosts:           241
Host inter-check delay method:   SMART
Average host check interval:     300.00 sec
Host inter-check delay:          1.24 sec
Max host check spread:           30 min
First scheduled check:           Sat Oct 23 16:39:54 2010
Last scheduled check:            Sat Oct 23 16:41:28 2010


SERVICE SCHEDULING INFORMATION
-------------------------------
Total services:                     4987
Total scheduled services:           4970
Service inter-check delay method:   SMART
Average service check interval:     296.38 sec
Inter-check delay:                  0.06 sec
Interleave factor method:           SMART
Average services per host:          20.69
Service interleave factor:          21
Max service check spread:           30 min
First scheduled check:              Sat Oct 23 16:40:08 2010
Last scheduled check:               Sat Oct 23 16:44:53 2010


CHECK PROCESSING INFORMATION
----------------------------
Check result reaper interval:       10 sec
Max concurrent service checks:      Unlimited


PERFORMANCE SUGGESTIONS
-----------------------
I have no suggestions - things look okay.


--
Jonathan Angliss


------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
::: Messages without supporting info will risk being sent to /dev/null


------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null