nagios and memory utilization

[Stanley Hopcroft]

Dear Sir,

I am writing to thank you for your letter and say, <sigh>

On Fri, May 30, 2003 at 05:17:50PM +0400, Sergey V. Artjushkin wrote:
> Hello
> 
> Colleagues,
> I have small question about Nagios 1.0.
> Two days ago, I have replaced old Netsaint 0.6 monitoring tool with new Nagios 1.0 tool.
> At this moment, I have about 3000 services to monitor. Most of services is checked
> by my perl plugin. 
> Nagios is compiled with --with-perlcache,--enable-embedded-perl options and
> with Mysql support.
> Everything is working well, but I have noticed that sometime Nagios uses to many memory.
> It's seems like every time when Nagios have some service sheduled for checking,
> it forks new child process.
>

that unfortunately you are between a rock and a hard place.

It is my long experience with an embedded Perl Nag (ePN), that Nag
process memory consumption _grows_ over time: the size of the Nag
process is proportional to the running time of the process (time after
last restart, not trap a SIGHUP).

 
> # top 
> PID USERNAME PRI NICE  SIZE    RES STATE  C   TIME   WCPU    CPU COMMAND
>  3718    231    64   0   165M 63992K CPU0   1 244:58 15.23% 15.23% nagios
> 10440 nagios    63   0   585M   207M RUN    0 280:36 13.18% 13.18% mysqld
> 74528    231    -6   0   165M 66324K pipewr 1   0:00  6.30%  0.88% nagios
> 74566    231    -6   0   165M 66340K pipewr 1   0:00  8.71%  0.83% nagios
> 74500    231    -6   0   165M 66328K pipewr 1   0:00  3.31%  0.73% nagios
> 74516    231    -6   0   165M 66356K pipewr 1   0:00  4.04%  0.73% nagios
> 74525    231    -6   0   165M 66344K pipewr 0   0:00  3.77%  0.68% nagios
> 74547    231    -6   0   165M 66324K pipewr 0   0:00  4.90%  0.68% nagios
> ....
> 
> The parent Nagios process (PID 3718) use 165MB  of memory, and each child
> process use the same amount of memory. For example when 10 child process is running
> they use 1,6GB of memory!!  I think it is too many. 
> So my question is, how I can reduce the size of memory that Nagios used??
>

I think you can 

 . abandon ePN and accept the extra fork + exec tax of loading Perl

 . run without the Perl cache: in this case you will lose the advantage
of having the Perl parse tree of the plugin in core, but still have Perl
resident.

 . try and fix the leak ?

 . restart the Nag process periodically.

 . think about how Nag can avoid a fork per check.

In my case (~200 hosts + 350 services + quite a lot of Perl), my ePN 
hovers around 40-60 MB

tsitc> top -b
last pid: 84246;  load averages:  2.80,  1.16,  0.62  up 60+01:56:29
23:50:17
75 processes:  5 running, 70 sleeping

Mem: 37M Active, 22M Inact, 68M Wired, 6116K Cache, 35M Buf, 117M Free
Swap: 256M Total, 34M Used, 222M Free, 13% Inuse


  PID USERNAME       PRI NICE  SIZE    RES STATE    TIME   WCPU    CPU
30405 nagios          10   0 48660K  7280K nanslp  96:12  0.00%  0.00%


and this is fine for 256 MB RAM and no other RAM pigs.

I usually manually restart if ePN is > 80 MB.

Unfortunately this seems to be  real problem with embedded Perl
applications.

Perhaps asking on the Perl XS list would provide some ideas.
 
> Thank you for advance.
> 
> -- 
> With best regards.
> 
>

Always happy to help a descendant of the society that was subject to the
man of whom this was written

'So I ask our government
  To double
    To treble
      The guard 
        Over this tomb'

(Yevtushenko, quoted by E Ginzburg, 'Into the whirlwind'. Only ever
published in samzidat in her lifetime)

Yours sincerely.

-- 
------------------------------------------------------------------------
Stanley Hopcroft
------------------------------------------------------------------------

'...No man is an island, entire of itself; every man is a piece of the
continent, a part of the main. If a clod be washed away by the sea,
Europe is the less, as well as if a promontory were, as well as if a
manor of thy friend's or of thine own were. Any man's death diminishes
me, because I am involved in mankind; and therefore never send to know
for whom the bell tolls; it tolls for thee...'

from Meditation 17, J Donne.


-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null