<table border="0" cellpadding="0" cellspacing="0" width="650"> <tbody><tr><td width="650" colspan="2" align="left" valign="top"><p><a href="http://branding.transpire.com.au/rs/UDXZia"><img border="0" src="cid:image-8215-20330@branding.transpire.com.au" class="rocketseed-strip" usemap="#tpheaderblack" alt="" /></a><map id="tpheaderblack" name="tpheaderblack"> <area target="_blank" href="http://branding.transpire.com.au/rs/iPdKPb" title="Tell me more" alt="Tell me more" coords="515,0,650,16" shape="rect" /> <area target="_blank" href="http://branding.transpire.com.au/rs/DTE0d0" title="Transpire Website" alt="Transpire Website" coords="0,17,650,96" shape="rect" /></map></p></td></tr><tr><td width="605" align="right" valign="middle"><table style="border: 1px none #F00;" border="0" cellpadding="0" cellspacing="0" width="605"> <tbody> <tr> <td align="right"> <p style="font-size: 10px;"><a style="text-decoration: none;" href="http://branding.transpire.com.au/rs/o-qE_O"><font style="color: #666666;" face="helvetica,verdana,geneva">cleverthink: IT Risk Management Paper that Trasnpire sponsored http://t.co/xEoap4G83n</font></a></p> </td> </tr> </tbody> </table></td><td width="45" align="left" valign="top"> <a href="http://branding.transpire.com.au/rs/8dYc9H"><img title="Follow Us on Twitter" class="rocketseed-strip" src="cid:image-7615-20330@branding.transpire.com.au" border="0"></a></td></tr></tbody></table><table border="0" cellpadding="0" cellspacing="0"> <tbody><tr><td valign="top"><table border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td><br>I think I answered my own question.<div><br></div><div>The host_inter_check_delay_method just needs tweaking to higher than the smart value (currently 1.5 sec because of our reasonably high number 600-700 hosts) to stagger the checks over a more reasonable period...</div> <div><br></div><div><a href="http://nagios.sourceforge.net/docs/3_0/configmain.html#host_inter_check_delay_method">http://nagios.sourceforge.net/docs/3_0/configmain.html#host_inter_check_delay_method</a></div><div><br></div> <div>Its not ideal as it'll be checking each device sequentially by alphabetical order in roughly the same time period, but it will at least give more of a chance for each device to get its ICMP replies back before the next one is asked.</div> <div><br></div><div>Ben<br><br><div class="gmail_quote">On Thu, Mar 21, 2013 at 9:41 AM, Ben Sykes <span dir="ltr"><<a href="mailto:ben.sykes@transpire.com.au" target="_blank">ben.sykes@transpire.com.au</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi All,<div><br></div><div>Long time user, first time poster on the list.</div><div><br></div><div>I have a fairly large distributed monitoring setup currently in pilot that's monitoring a variety of devices at remote branch sites.</div> <div><br></div><div>The hostnames at all these sites are very similar since the naming standard includes the branch ID at the start of the hostname.</div><div><br></div><div>What we are seeing is Nagios' scheduler is using a sorted list to drive the host check scheduling decisions, which means all or the majority of devices at a particular site are being checked at once. With all those ICMP packets going down a long thin WAN link that may be close to 100% utilisation, we are seeing all the devices at the site go down at once, then come back up as soon as the next check is run in a more staggered manner.</div> <div><br></div><div>I have checked the source code and the host list Nagios uses is sorted after reading the config files and the scheduler routine simply walks the linked list of hosts, and adds them into the schedule.</div> <div><br></div><div>My ideas to solve it...</div><div><br></div><div>- Modify check_icmp with a wrapper script or similar that adds a random delay to the ping check to avoid the mass of packets (OK but will still lead to events where all the randomness adds up)</div> <div>- Modify the Nagios source code and recompile to remove the sorting of host lists (suboptimal)</div><div>- Increase the thresholds for ping timeouts etc (doesn't really let us track latency of each site as it's then affected by the ping grouping)</div> <div><br></div><div>Any ideas from the community that'd be useful?</div><div><br></div><div>Thanks</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Ben Sykes</div> </font></span><br>------------------------------------------------------------------------------<br> Everyone hates slow websites. So do we.<br> Make your web apps faster with AppDynamics<br> Download AppDynamics Lite for free today:<br> <a href="http://p.sf.net/sfu/appdyn_d2d_mar" target="_blank">http://p.sf.net/sfu/appdyn_d2d_mar</a><br>_______________________________________________<br> Nagios-users mailing list<br> <a href="mailto:Nagios-users@lists.sourceforge.net">Nagios-users@lists.sourceforge.net</a><br> <a href="https://lists.sourceforge.net/lists/listinfo/nagios-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/nagios-users</a><br> ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.<br> ::: Messages without supporting info will risk being sent to /dev/null<br></blockquote></div><br></div> </td></tr></tbody></table></td></tr><tr><td><br><table border="0" cellpadding="0" cellspacing="0"> <tbody><tr><td width="330" rowspan="1" align="left" valign="bottom"><table border="0" cellpadding="0" cellspacing="0" width="330"> <tbody> <tr> <td style="padding-left: 0px;"> <p style="color: #666666; font-family: helvetica, arial; font-size: 12px;"><font face="helvetica, arial"><span style="font-size: 22px;">ben sykes</span> </font><br><font face="helvetica, arial"> <span style="color: #3399cc;">consultant</span> </font><br> <br><font face="helvetica"> m. +61 404 308 201 </font><br><font face="helvetica, arial"> e. ben.sykes@transpire.com.au</font></p> </td> </tr> </tbody> </table></td><td width="320" rowspan="1" align="left" valign="bottom"><table width="320" border="0" cellspacing="0" cellpadding="0"> <tbody> <tr> <td> <p style="font-face: helvetica, arial; color: #666666; font-size: 12px;"><font face="helvetica, arial"> <font color="#3399cc"><strong>Sydney</strong></font> <br>Level 4, 93-95 Pacific Highway<br>North Sydney NSW 2060<br>T: +61 (0) 2 8221 8850 <br>F: +61 (0) 2 9925 8787 </font></p> </td> </tr> </tbody> </table></td></tr><tr><td width="650" colspan="2" align="left" valign="top"><a href="http://branding.transpire.com.au/rs/a2tRnW"><img usemap="#tpfooter" alt="more info: transpire.com.au" class="rocketseed-strip" src="cid:image-8145-20330@branding.transpire.com.au" border="0"></a><map name="tpfooter" id="tpfooter"> <area target="_blank" href="http://branding.transpire.com.au/rs/uG5l5u" title="who we are" alt="who we are" coords="37,9,85,44" shape="rect"> <area target="_blank" href="http://branding.transpire.com.au/rs/aKEYkg" title="what we do" alt="what we do" coords="154,9,213,44" shape="rect"> <area target="_blank" href="http://branding.transpire.com.au/rs/Zj3Wo1" title="work we've done" alt="work we've done" coords="274,9,342,44" shape="rect"> <area target="_blank" href="http://branding.transpire.com.au/rs/Cy2oo_" title="our opinions" alt="our opinions" coords="408,9,459,44" shape="rect"></map></td></tr></tbody></table></td></tr></tbody></table><table border="0" cellpadding="0" cellspacing="0" width="100%"> <tbody><tr><td></td></tr></tbody></table><table border="0" cellpadding="0" cellspacing="0" width="100%"> <tbody><tr><td><br><table width="650" border="0" cellpadding="0" cellspacing="0"> <tbody><tr><td width="650" align="left" valign="top"><table border="0" cellpadding="0" cellspacing="0" width="650"> <tbody> <tr> <td style="padding-left: 0px;"><span style="font-face: arial; font-size: 7pt; color: #808080; text-align: justify; padding: 0;"><font face="helvetica"><strong>legal disclaimer:</strong></font><br><br><font face="helvetica"> This email, including any attachments, may be confidential or privileged, and is sent for the personal attention of the intended recipient. If you have received this email in error, please delete it immediately. The views expressed are not necessarily those of Transpire Pty Ltd. Transpire is not liable for the effects of any virus which may be contained in this email.</font></span></td> </tr> </tbody> </table></td></tr></tbody></table></td></tr></tbody></table>