How to reduce a very high latency number

[Dan Hopkins]

Out of interest, how many of your clients submit checks to each master? (or do you mean they all submit their results to every master?) and do you see many failed OCSP command submissions in the client logs?

An install I'm testing at the moment has around 4000 services on 3 minute check split between an active master and a single slave (i.e. master performing half the checks actively) Of the 2000ish distributed services, I see anything up to 10 failed passive submits per hour with NSCA running as a daemon on the master. From some perliminary debugging, a connection is made but hangs and times out before doing anything useful.

I've ditched a nagios setup that was processing 10,000 monitors split over 7 or 8 slaves as the latency made it unusable. I'd assumed the pipe on the master just couldn't handle the volume of results coming in and was losing passive results, not that there was much time to investigate it properly (is there ever)


-- 
Dan


> -----Original Message-----
> From: nagios-users-admin at lists.sourceforge.net
> [mailto:nagios-users-admin at lists.sourceforge.net]On Behalf Of Marc
> Powell
> Sent: 24 May 2006 15:47
> Cc: nagios-users at lists.sourceforge.net
> Subject: RE: [Nagios-users] Re: How to reduce a very high 
> latency number
> 
> 
> I have 3 separate central servers (2 production and 1 for 
> testing), all
> running nsca as a daemon and all receiving the same 3590 
> passive service
> results every 5 minutes. I've never had a problem with missed 
> checks or
> high latency numbers. Stats from my clients --
> 
> Client1 submits 504(x3) passive checks -
> Metric	Min.	Max.	Average
> Check Execution Time:  	0.01 sec	21.40 sec	
> 3.394 sec
> Check Latency:	0.01 sec	2.18 sec	0.216 sec
> Percent State Change:	0.00%	5.72%	0.07%
> 
> Client2 submits 1260(x3) passive checks -
> Metric	Min.	Max.	Average
> Check Execution Time:  	0.04 sec	35.03 sec	
> 6.789 sec
> Check Latency:	0.00 sec	3.95 sec	0.839 sec
> Percent State Change:	0.00%	11.84%	0.08%
> 
> Client3 submits 824(x3) passive checks -
> Metric	Min.	Max.	Average
> Check Execution Time:  	0.09 sec	15.79 sec	
> 7.999 sec
> Check Latency:	0.01 sec	7.97 sec	2.034 sec
> Percent State Change:	0.00%	5.86%	0.02%
> 
> Client4 submits 293(x3) passive checks -
> Metric	Min.	Max.	Average
> Check Execution Time:  	0.09 sec	10.36 sec	
> 7.584 sec
> Check Latency:	0.00 sec	1.15 sec	0.249 sec
> Percent State Change:	0.00%	12.11%	0.12%
> 
> Client5 submits 720(x3) passive checks -
> Metric	Min.	Max.	Average
> Check Execution Time:  	0.11 sec	16.81 sec	
> 8.844 sec
> Check Latency:	0.00 sec	6.81 sec	1.009 sec
> Percent State Change:	0.00%	11.84%	0.04%
> 
> All very reasonable to me. Clients 3-5 are single proc PIII-800's
> running nagios and cricket and are due to be upgraded but even an
> average latency of 2 seconds there is nothing to fret over at all.
> 
> --
> Marc
> 
> > -----Original Message-----
> > From: nagios-users-admin at lists.sourceforge.net [mailto:nagios-users-
> > admin at lists.sourceforge.net] On Behalf Of Morris, Patrick
> > Sent: Wednesday, May 24, 2006 3:54 AM
> > To: Greg Cope; Jacob Ritorto
> > Cc: nagios-users at lists.sourceforge.net
> > Subject: RE: [Nagios-users] Re: How to reduce a very high latency
> number
> > 
> > How are you guys running the nsca daemon?  I've got systems that
> perform
> > thousands of checks with no problem.
> > 
> > I'm looking at a system right now that submits over 5300 checks to a
> > central server running nsca via xinetd, and it has a average service
> > latency of .153 secs.
> > 


-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid7521&bid$8729&dat1642
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null