Nagios-devel digest, Vol 1 #807 - 8 msgs

[sean finney]

hey

(finally got around to switching off digest mode, sorry for constantly
 breaking threads and all that)

On Tue, May 10, 2005 at 12:40:22AM +0200, Andreas Ericsson wrote:
> Actually, I made a mistake there. Only two fork()'s, as /bin/sh calls 
> execve() more or less immediately.

are you sure about that?  this is going off on a bit of a tangent,
but i believe "/bin/sh -c command" forks before it executes command
(as command can be "subcmd1; subcmd2").  but i digress.

> >and i'm suggesting
> >- nagios forks
> 
> nagios child does a symbol lookup of plugin_function

yes, but in the "grand scheme" of things, the lookup could be a one
time cost, leaving a function pointer for calling the plugin.

> >note that if this were in a multi-threaded arch, or if the child
> >processes were pre-allocated, even this fork would have a negligable
> >effect.
> 
> No more negligable than what's already the case. I'm not sure what you 
> mean by multithreaded arch (multithreaded design, SMP machine?).

code base, sorry for being ambiguous.  

> True. Didn't think of that. But with that in mind it might be better to do
> nagios -> fork("pure" child) -> execve()
> which would also save us the (currently not) superfluous fork()'s. This 
> is a trick question. Or is it? ;)

aha, got you!  

> Have you seen the state of the plugins today? It's a _LOT_ of shoddy 
> code running around in there (many plugins popen() by the way).

yes, i've been hacking with them for some time, and have recently joined
up and have been fixing some of the more obvious problems in cvs.
lots of popen()s that should otherwise be avoidable[1].

> >>popen() is fork() + dup() + execve(), more or less. Read glibc-2.3.5 
> >
> >popen is fork + dup + exec (/bin/sh -c) + fork + dup + exec (your command).
> >
> sh doesn't fork. It just calls execve() to overwrite itself with the new 
> process (there's no call named exec, btw) and dup() isn't used, it's 
> pipe() (my bad from the beginning), so

it exec's sh -c, which like i said earlier i'm fairly certain parses
the command argument and fork/exec's each sub command.  as for dup,
i would expect to see it somewhere in there, after stdout was closed.

> Your test was a bit rubbish (sorry, but it was) because it didn't 

i think you've illustrated that quite well :)

> >i think you and i are barking up two slightly different trees here.
> 
> We're both discussing performance improvements in nagios, right?

yes, but i was talking chiefly wrt avoiding all the fork/exec overhead,
and you're talking a "slightly" grander scheme improvement.  note that
these are not mutually exclusive.  in fact, i'd be willing to bet
that at some point the parallel nature of a multithreaded nagios
daemon will hit right back up against a serialized bottleneck, that
being the number of forked processes vs the number of processors
available.  in such a case this small improvement might prove to be
a little helpful afterall.

> mechanism (either fork() or pthread_create()) which makes the entire 
> notion somewhat ridiculous for plugins that are used less frequently 
> than say once every five seconds. The only plugin I can think of that 
> falls into that category would be a ping check and possibly interface 
> related snmp-checks for switches and routers in really huge networks.

the idea isn't so much to make any individual check any faster, but
to reduce the overall load on the server.  

to argue against myself it's worth noting that of the two plugins you
mention this being the most helpful, one already calls popen inside of
itself, and the second is currently a perl plugin :/

> I'd be happy to revise my opinion on this if you were to provide a PoC 
> that shows that dlopen() works significantly (5% or more) faster than 
> the popen() approach while running checks in parallell and reaping the 
> results in a satisfactory manner (1 line of output, no leaks, get 
> return-value). It would also be nice if the current level of protection 
> remains (plugins can sigsegv without affecting the main process). So far 
> I've seen nothing that even comes close to this.

i'll throw something together tonight, even if it is just as an academic
excercise.  we'll see, i guess :)


	sean

--
[1] some of the first work i did for nagiosplug was adding the ability
    to avoid popen for most os's in check_swap.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <https://www.monitoring-lists.org/archive/developers/attachments/20050509/6045ce8f/attachment.sig>