A different way?

[Steven D. Morrey]

It's very similar.
I call it semi-passive (although someone mentioned passive-aggressive might be a better name for it).
You still have an active nagios instance and it's still checking to make sure checks did execute on time (similar to active), it's just not doing the actual execution anymore (similar to passive), and instead of processing the "meaning" of the results of a check it would just process the outcome as directed by the rules for the host/service being monitored.

Let me give a for instance...
Under my current setup I dispatch a check to a DNX worker node, the check executes and the result is handed wholesale back to Nagios.
Nagios parses the result, and tries to divine if the service is up, down, flapping etc and then takes appropriate action.
Here's a breakdown of where time is spent.

Nagios event loop approx 0.07s handing service check to DNX
DNX average of 3 seconds round trip
Nagios up to 10 seconds to process the result depending on how many dependencies are involved, and as much as 30 seconds if a host check is required.

Now obviously this is because all of my service checks are active and not passive and I have 3,000 hosts and 30,000 service checks

Under the proposed design it would look more like this.

Nagios initializes and pushes all schedule pieces to all hosts.
Next nagios enters a passive mode where it listens for results, and audit mode where it watches the schedule looking for results that haven't come in yet.
On the flip side the execution daemon is running on each host and it's executing the checks, determining what is meant by the check "service up/down flapping etc" and passes that meaning back to nagios which subsequently takes the appropriate action.
All the while the auditor is watching for checks that were scheduled but haven't come in yet, and contacting hosts to find out whats up etc.

So really in some ways this is an expansion of the current passive model for checks, but in some ways this is a whole new model (compared to what we do now anyways) 

Those are my thoughts on the matter, what do you think?

Sincerely,
Steve
 
________________________________________
From: hemebond [hemebond at gmail.com]
Sent: Friday, September 25, 2009 2:19 AM
To: Nagios Developers List
Subject: Re: [Nagios-devel] A different way?

Isn't this the same as using passive checks? It sounds like what I've set up. I wrote a simple agent (script) that has its own schedule and runs the checks, sending the result back to a Nagios server.

2009/9/25 Steven D. Morrey <smorrey at ldschurch.org<mailto:smorrey at ldschurch.org>>
Hello everyone,

I've decided to take a break for a bit from multi-threading nagios to focus on DNX since that is my day job after all :)
While working on all of this I had a few thoughts that might make some good ideas if Nagios is ever re-designed again, say for a 4.x branch.

As you know, under nagios, all checks are dispatched by nagios to be executed on the local machine at set intervals.
Under a distributed nagios setup, you have multiple nagios instances running on various machines executing checks and passing the results back to a passive master controller.

Under DNX, we distribute the load to "worker nodes" which then execute the checks and hand the results back to an active master controller that then processes the result etc.

Profiling shows that (under DNX at least) 2/3rds of our time is spent in the reaper processing results, so  wouldn't it make more sense to flip the process around?

The checks are already executing on the local machine, so how about a daemon on each machine, the daemon would keep the schedule and execute service checks locally, processing the result and returning the results and the required actions (based on a local policy) to nagios which would then do the actual work of handling notifications etc and so forth.
This way nagios could be an auditor, if it doesn't receive a result on time as expected, then it could query the daemon to see whats gone wrong, if that fails then it could initiate a host check, etc.

>From a design standpoint this is a bit more work than the current setup, but it seems to me that this could allow for much greater flexibility and scalability in the long run.

Anyways I hope this sparks a little debate but I don't want to "come in and shake things up", or go around changing everything, stepping on toes all the while, it's just that putting the responsibility of actually executing the check and doing so on time, onto the computer it needs to execute on, just makes more sense to me.
It's not really dramatically different from what we do now, it's just adding a scheduler/timer to the existing execution framework and adding something to push the original schedule and any changes such as scheduled downtime to the appropriate machines, putting everything else into a semi passive mode effectively turning each machine to be checked into it's own "worker node"

Thoughts?

Sincerely,
Steve




 NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.



------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Nagios-devel mailing list
Nagios-devel at lists.sourceforge.net<mailto:Nagios-devel at lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/nagios-devel


------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf