- Remote Log file with NSCA \ Syslog

[Fred]

We use syslog-ng as an aggregator with syslogd's configured on multiple
nodes to forward messages to a consolidated log file.  We then use a custom
plug-in to match rules against the consolidated log to generate alerts
and create an HTML log of interesting happenings for each individual node.

I avoid NRPE and NSCA as using NRPE too often on a network causes a huge
performance skew on MPI jobs.  It is fine for occassional use, but if you
are using it more then once every few minutes it can disrupt large jobs that
have small MPI synchronization windows.  Our single biggest performance issue
across our cluster was NRPE.   We were using NRPE for 3 plug-ins, 1 at 1 minute
intervals and 2 at 5 minutes from our headnode to each compute node.  At 8
nodes that is 24 invocations.  At 128 nodes that is 384 calls which Nagios
attempts to spread out.  If you do not have these kinds of requirements
NRPE is a perfectly reasonable and well behaved, however, I would not suggest
its use for something like logging especially if you are doing SSL connections.

NSCA with high volume might get you into trouble with the nagios FIFO.  If
you don't configure nagios to read the FIFO aggressively (default is 15s) then
you can easily fill the FIFO and cause NSCA to block and hang and therefore
pend all jobs across the system that are writing to it.  Most filesystems have
a 4k block size so your FIFO will block pretty quickly if nagios only polls
it every 15 seconds.   If you are using distributed monitoring, nagios can
spawn many ocsp commands very quickly all of which create a connection to
NSCA which in turn forks many children all backed up writing to the FIFO. (this
assumes your submit script for ocsp calls send_nsca).   You could always 
configure it to use a separate file/fifo and avoid these issues, but it would
be easier to just use syslog-ng as you get the filtering capability as a bonus.

The simplest thing to do might be to just configure syslogd to forward a class
of messages to another syslogd if that is all you need.

-FredC

--- Sivan DERAY <sivan.deray at net-aptitude.fr> wrote:

> 
> Hi,
> 
> Does anybody uses Syslog or Syslog-ng with nsca to send message to
> Nagios ?
> 
> Otherwise, is there a way to analyse Remote linux\unix like log with
> nagios ?
> 
> Thanks in advance
> 
> 
> 
> Sivan DERAY
> IT Security Engineer
> 
> 
> -------------------------------------------------------
> SF.Net email is sponsored by:
> Tame your development challenges with Apache's Geronimo App Server.
> Download it for free - -and be entered to win a 42" plasma tv or your very
> own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
> _______________________________________________
> Nagios-users mailing list
> Nagios-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nagios-users
> ::: Please include Nagios version, plugin version (-v) and OS when reporting
> any issue.
> ::: Messages without supporting info will risk being sent to /dev/null
> 







-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. 
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null