trying to fix problem with excessive latency

[Frater, Greg J]

Hi All, 

This is just a me too post to hopefully help others down the road! 

>>>> I have inherited maintenance of a medium-sized Nagios installation.

>>>> We currently have 649 hosts and 5415 services. Our setup works 
>>>> nicely, with one exception: Nagios falls behind on host/service 
>>>> checks. Our usual latency once Nagios has been running for a while 
>>>> is about 190-200 seconds. Our Nagios host is reasonably powerful
and 
>>>> isn't struggling; it seems that Nagios itself is limited somehow.

We are similar in size 530 hosts, 3300 service checks with a service
check latency of around 300 seconds and 350 for hosts.  There are no
obvious performance bottlenecks on our sever the closest thing to it for
us was the CPU's ave of around 30% utilization across 4 procs with 4 GB
of RAM.  Our current configuration did not have any latency issues
running Nagios 3.1.2, our latency issues started a couple of weeks ago
with an upgrade to 3.2.2. 

>It looks like use_large_installation_tweaks did the trick. I could have
sworn I tried that before; likely I set it to 0 instead of 1 or
something foolish like that.

We've avoided this in an effort to keep our configuration as plain as
possible and because I remember reading that this was not expected to be
effective unless you had something like 1000+ hosts and many more
service checks.  Apparently that is not correct, at least under 3.2.2.

>Anyway, I wanted to test the individual components of
large_installation_tweaks to see what helped.
>* child_processes_fork_twice=0: latency drops from 149 to 18

This change took our latency from ~300s to .228 for services and ~350
seconds to .458 for hosts

>* free_child_process_memory=0:  latency drops from 149 to 0.6

I made this change with the 'child_processes_fork_twice=0' change still
in place and saw no measurable drop in latencies.

>* disabling summary macros in environment variables doesn't seem to be
>  individually configurable, but I expect that's disabled already by
>  enable_environment_macros=0.

Did not do this, was not deemed necessary

>So, thanks again to everybody who helped. Latency is now 0.35% of what
it used to be.
>
>-Corey

Ditto, this was a big help to us as well, thanks.

-greg

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null