Dynamically generated service checks

[David Rosenstrauch]

Marc Powell wrote:
> On Jul 15, 2009, at 10:19 AM, David Rosenstrauch wrote:
> 
>> Matthew Jurgens wrote:
> 
>> 2) If I understand correctly, Nagios loads its config files at startup
>> time and does not re-read them after that.  So if I'm regenerating a
>> config file, then that means I'd need to restart the Nagios service
>> afterwards, which is a bit of an onerous imposition.
> 
> Using the init script, restart stops and starts the daemon, reload  
> sends a HUP signal to the running process to tell it to re-read it's  
> config files. The init script verifies config before doing either. You  
> could manually HUP the running process if you're sure the config files  
> are syntactically correct.
> 
>>  And although
>> again, I could in theory do this in a cron job, I'm not sure I'm
>> comfortable with that.  There's the potential for the Nagios service  
>> to
>> not start up again successfully, and I don't like taking the risk that
>> this dynamic update procedure could potentially bring down the entire
>> Nagios system.
> 
> I've been doing this exact thing (nagios reload) hourly for several  
> years quite successfully. If you wanted to be paranoid about it, you  
> could script a run of '/path/to/nagios -v /path/to/nagios.cfg' and  
> only reload if that exits 0 else send yourself an e-mail with the bad  
> output.
> 
> --
> Marc

Thanks much for these pointers (and the quick response).  Didn't know 
about the SIGHUP thing.

Well, it's good to know that this is a viable option.  Still, re-writing 
a config file seems a bit of a kludgey way to handle this.

I'm mulling over another idea, which I'm calling a "rotating" service 
check (for lack of a better word).  Basic idea is:  I set up one single 
service (rather than a service for each tag/file), and each time the 
service runs it automatically "rotates" to check the next tag/file.  If 
the check fails, then it stops rotating and continues checking that 
tag/file until the problem gets fixed.

Still working out the details, though, and not sure if this is a viable 
solution for us or not.  On the one hand, assuming we run the check 
every 5 minutes, then the service check would rotate through and thereby 
check all of the tags within some reasonable period of time.  On the 
other hand, if it hits a tag that fails the check, we'd only get an 
alert on that particular tag and wouldn't know if other tags were 
failing the check too.


IMO, the ideal solution here would be if I could just submit passive 
check results for services that aren't explicitly configured in Nagios. 
  But alas, that's not allowed and it fails with messages like "Warning: 
  Passive check result was received for service 'foo' on host 
'mysql-dev', but the service could not be found!"


I'll have to mull this over some more ...

Thanks,


------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null