high latency

[Daniel Wittenberg]

Someone else noticed that nagios is generating a ton of minor page
faults, and curious if that's normal and if that could be causing some
of the latency in the checks?  I've also got a tmpfs setup for the
status.dat and the checkresults directory to ease some of the disk i/o
since we're on a san-backed vm host.

I turned off embedded perl this morning and our latency has been holding
at < 10 seconds so far, so that seemed to help a lot.

Dan

-----Original Message-----
From: Daniel Wittenberg [mailto:daniel.wittenberg.r0ko at statefarm.com] 
Sent: Thursday, December 02, 2010 11:42 AM
To: Andreas Ericsson; Nagios Users List
Subject: Re: [Nagios-users] high latency

Not using SNMP for any of the checks, and most are passive checks.  For
the few active checks we are probably going to be using dnx.  

Embeded perl is interesting though, I hadn't tried that, thought it was
supposed to help with performance.  I don't think we have any obsessive
stuff running right now.

Right now hardware is 4 proc vmware esx, 4GB RAM.  For production there
will be 12 of those boxes with the number of hosts being about 1200-1500
per nagios server.

Dan

-----Original Message-----
From: Andreas Ericsson [mailto:ae at op5.se] 
Sent: Thursday, December 02, 2010 11:19 AM
To: Nagios Users List
Cc: Daniel Wittenberg
Subject: Re: [Nagios-users] high latency

On 12/02/2010 06:05 PM, Daniel Wittenberg wrote:
> Yeah, been running that since day one, since when rollout is done
we'll
> probably have about 18k servers and around 3 million service checks...
> 

170 services per host? Sounds like an awful lot of switches. I'd use
some cleverness to grab snmp-info once and parse the data afterwards
if I were you.

For that kind of installation, you'll need to use a distributed setup
of some sort. merlin, dnx and apparently mod-gearman should get you
going in the right direction.

-- 
Andreas Ericsson                   andreas.ericsson at op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

------------------------------------------------------------------------
------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for
grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when
reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null

------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null