nagios 2.8 + NDO 1.4b2 + NDO2DB = high service latency

[Ton Voon]

On 3 May 2007, at 21:07, Jim Avery wrote:

> On 03/05/07, Janet Post <Janet.Post at excapsa.ca> wrote:
>> Hello,
>>
>> Is anyone else experiencing high service latency when using NDOUtils?
>> If I disable the event broker in the nagios config file, I have  
>> service
>> latency of 0.00/1.59/0.592.  When the event broker is running, the
>> service latency jumps to 389+ seconds.
>>
>>
>> Has anyone else experienced (and hopefully fixed) this anomaly?
>>
>
> I find that ndoutils sometimes slows my Nagios server down because
> it's busy deleting old records from some of the tables.  I found that
> tuning the MySQL database in various ways helped (not that I know much
> about MySQL), and reducing the number of days data retained helped a
> bit too.
>
> I can recommend phpMyAdmin if like me you're not all that familiar
> with MySQL.  It can show you what SQL is currently running and has a
> few tuning features which are easy to use.
>
> I'm currently thinking it might help a fair bit if I put the MySQL
> database on another server.  I haven't gotten around to doing that yet
> though.
>
> My server is quite busy even when ndoutils isn't running.  I don't
> think the burden of running ndoutils is huge, but for me it's enough
> to tip it over from latency of 5 seconds or so when I'm not running
> ndoutils to 30 seconds or more at times when I am.
>

Just to let you know that we've found DB contention issues with  
Nagios 2.8 with NDO 1.4b2. Mysql takes a very long time (around 30  
seconds) to run the (from memory) "DELETE FROM nagios_servicechecks  
WHERE instance_id = 1 AND entry_time < ....." query. This holds up  
other things, though I haven't fully understood where or why.

The problem is with the indexes for nagios_servicechecks. We dropped  
the unique key index, which was (instance_id, service_object,  
entry_time, uentrysec), and created a new index based on entry_time  
only. This is much quicker because (a) the index is smaller, (b)  
mysql can locate the correct rows quicker because it finds the rows  
based on time, rather than trying to find rows based on instance_id  
(which is every row in the table - effectively doing a full table scan).

(As an aside, mysql will not enforce the uniqueness on that index -  
it only works for tables of type innodb, not myisam.)

We plan on blogging about this in more detail within the next two  
weeks because it is a major enhancement to NDOutils in large scale  
environments, but we're just snowed under with work :(

Ton

http://www.altinity.com
T: +44 (0)870 787 9243
F: +44 (0)845 280 1725
Skype: tonvoon



-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null