<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style></style></head><body bgcolor=white lang=EN-US link=blue vlink=purple><div class=WordSection1>Wow, I completely forgot that I’d responded to this. This is what I do. If you use this script, you’ll want to change the notification e-mail address, where it will send notifications when the failover server decides it needs to take over and when it decides to yield to the primary if the primary has come back online.<o:p></o:p><o:p> </o:p>-------------------------------------------------<o:p></o:p>Failover Configuration<o:p></o:p><o:p> </o:p>On the failover server, install the same OS the same way it's installed on the<o:p></o:p>primary monitoring server, but set Nagios to not start in runlevels 3 and 5 or<o:p></o:p>else the failover checking script will generate e-mail notifications when the<o:p></o:p>failover server is rebooted (Nagios will start before the failover server<o:p></o:p>notices it's running on the primary, and the message will come when the fail-<o:p></o:p>over server shuts the failover Nagios down).<o:p></o:p><o:p> </o:p>On the failover server, generate a public/private key pair. This is necessary in<o:p></o:p>order to avoid having to type in a password every time the state of the Nagios<o:p></o:p>process on the primary server is checked:<o:p></o:p><o:p> </o:p> # ssh-keygen -t rsa<o:p></o:p><o:p> </o:p>Take the default name and location (/root/.ssh/id_rsa and id_rsa.pub). Do not enter<o:p></o:p>a passphrase.<o:p></o:p><o:p> </o:p>Copy id_rsa.pub to the primary server:<o:p></o:p><o:p> </o:p> # rsync -avzu id_rsa.pub primaryserverhostname:/root/.ssh/<o:p></o:p><o:p> </o:p>On the primary server, append the id_rsa.pub to authorized_keys2:<o:p></o:p><o:p> </o:p> cat id_rsa.pub >> $HOME/.ssh/authorized_keys2<o:p></o:p> chmod 0600 authorized_keys2<o:p></o:p><o:p> </o:p>Download, compile, and install Nagios on the failover server the same way it's<o:p></o:p>installed on the primary server.<o:p></o:p><o:p> </o:p>Create a script named nagios_check.sh in /root/:<o:p></o:p>-----------------------<o:p></o:p><o:p> </o:p> #!/bin/bash<o:p></o:p> nagiospath='/usr/local/nagios'<o:p></o:p> alertaddress='you@yourdomain'<o:p></o:p> maxfaillimit='3'<o:p></o:p><o:p> </o:p> touch failed_nagios_checks<o:p></o:p> failedchecks=$(cat failed_nagios_checks)<o:p></o:p><o:p> </o:p> if [[ -z "${1}" ]]<o:p></o:p> then<o:p></o:p> echo Usage: nagios_check hostname<o:p></o:p> exit<o:p></o:p> fi<o:p></o:p><o:p> </o:p> nagiosstatusnow=$(${nagiospath}/libexec/check_by_ssh -H ${1} --command='/usr/local/nagios/libexec/check_nagios --filename=/usr/local/nagios/var/status.dat --expires=1 --command=nagios')<o:p></o:p> nagiosstatus="${nagiosstatusnow%%:*}"<o:p></o:p> nagiosrunninglocally=$(/etc/init.d/nagios status)<o:p></o:p><o:p> </o:p> if [[ "${nagiosstatus}" = "NAGIOS OK" ]]<o:p></o:p> then<o:p></o:p> echo -ne "[`date`] ${nagiosstatus} on ${1}. "<o:p></o:p> if [[ "${nagiosrunninglocally%% *}" = "nagios" ]]<o:p></o:p> then<o:p></o:p> echo -e Nagios is currently running on the failover server, and needs to be stopped.<o:p></o:p> /etc/init.d/nagios stop<o:p></o:p> /usr/bin/printf "%b" "[`date`] Nagios recovery on ${1} detected. Stopping failover Nagios.\n\n${nagiosstatusnow}" | /bin/mail -s "Nagios recovery on ${1}" ${alertaddress}<o:p></o:p> fi<o:p></o:p> echo -e "Failed ${failedchecks} checks: synchronizing files. Status: ${nagiosstatusnow} "<o:p></o:p> echo 0 > failed_nagios_checks<o:p></o:p> rsync --quiet --archive --compress --delete-during --exclude=var/spool/checkresults/* --exclude=var/archives/* --exclude=*~ --exclude=nagios.lock --exclude=nagios.cmd ${1}:${nagiospath} /usr/local<o:p></o:p> else<o:p></o:p> failedchecks=$((${failedchecks} + 1))<o:p></o:p> echo ${failedchecks} > failed_nagios_checks<o:p></o:p> if [[ "${failedchecks}" -lt "${maxfaillimit}" ]]<o:p></o:p> then<o:p></o:p> echo -e "[`date`] Uh-oh! Failed ${failedchecks} out of ${maxfaillimit} checks. Status: ${nagiosstatusnow} "<o:p></o:p> fi<o:p></o:p> if [[ "${failedchecks}" -ge "${maxfaillimit}" ]]<o:p></o:p> then<o:p></o:p> echo -ne "[`date`] ${nagiosstatus} on ${1}. "<o:p></o:p> if [[ "${nagiosrunninglocally%% *}" = "No" ]]<o:p></o:p> then<o:p></o:p> echo -e " Failed ${failedchecks} checks, and needs to be started on the failover server. "<o:p></o:p> /etc/init.d/nagios start<o:p></o:p> /usr/bin/printf "%b" "[`date`] Nagios on ${1} has failed ${failedchecks} checks. Starting Nagios on failover server.\n\n${nagiosstatusnow}" | /bin/mail -s "Nagios failure on ${1}" ${alertaddress}<o:p></o:p> else<o:p></o:p> echo -e "Failed ${failedchecks} checks, but is already running on the failover server. "<o:p></o:p> fi<o:p></o:p> fi<o:p></o:p> fi<o:p></o:p><o:p> </o:p>-----------------------<o:p></o:p><o:p> </o:p>Make it executable by root:<o:p></o:p><o:p> </o:p> chmod u+x nagios_check.sh<o:p></o:p><o:p> </o:p>Run crontab -e as root and add this line:<o:p></o:p><o:p> </o:p> * * * * * /root/nagios_check.sh primaryserverhostname >> /var/log/nagios_check.log 2>&1<o:p></o:p><o:p> </o:p>The *s set it to run every minute. The output is redirected to a log file, and the 2>&1 redirects both STDOUT and STDERR.<o:p></o:p><o:p> </o:p>At the top of the every minute now, the failover server will obtain a current replica of the<o:p></o:p>primary server's Nagios status (comments, acknowlegements, downtime, configuration files, etc).<o:p></o:p><o:p> </o:p>Add a file in /etc/logrotate.d called nagios_check:<o:p></o:p>-----------------------<o:p></o:p><o:p> </o:p> /var/log/nagios_check.log {<o:p></o:p> weekly<o:p></o:p> missingok<o:p></o:p> notifempty<o:p></o:p> }<o:p></o:p><o:p> </o:p>-----------------------<o:p></o:p><o:p> </o:p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>From: quanta [mailto:quanta.linux@gmail.com] Sent: Wednesday, October 13, 2010 7:17 AM To: nagios-users@lists.sourceforge.net Subject: Re: [Nagios-users] Using two nagios servers...<o:p></o:p></div></div><o:p> </o:p>Try something like this: #!/bin/sh RETURN_STATUS=`/usr/local/nagios/libexec/check_nrpe -H <primary_host> -c check_nagios | awk -F: '{ print $1 }' | awk '{ print $2 }'` if [ $RETURN_STATUS != "OK" ]; then sed -i 's/enable_notifications=0/enable_notifications=1/' /usr/local/nagios/etc/nagios.cfg sed -i 's/execute_service_checks=0/execute_service_checks=1/' /usr/local/nagios/etc/nagios.cfg else sed -i 's/enable_notifications=1/enable_notifications=0/' /usr/local/nagios/etc/nagios.cfg sed -i 's/execute_service_checks=1/execute_service_checks=0/' /usr/local/nagios/etc/nagios.cfg fi sudo /etc/init.d/nagios reload Note: you must add nagios user to sudoers group (without password prompt). On 08/16/2010 02:44 PM, <a href="mailto:ravishankar.gundlapali@wipro.com">ravishankar.gundlapali@wipro.com</a> wrote: <o:p></o:p>Hi,<o:p></o:p> <o:p></o:p>Even I run Nagios on Virtual machines.<o:p></o:p> <o:p></o:p>Please let me know where can I get the support for running cron job on my secondary Nagios server to monitor the Nagios service on primary Nagios server?<o:p></o:p> <o:p></o:p>Thanks,<o:p></o:p>Ravi G<o:p></o:p> <o:p></o:p><div><div style='border:none;border-top:solid windowtext 1.0pt;padding:3.0pt 0in 0in 0in;border-color:-moz-use-text-color -moz-use-text-color'>From: Chris Beattie [<a href="mailto:cbeattie@geninfo.com">mailto:cbeattie@geninfo.com</a>] Sent: Monday, August 16, 2010 6:51 PM To: Nagios Users List Subject: Re: [Nagios-users] Using two nagios servers...<o:p></o:p></div></div> <o:p></o:p>Your servers will probably be fine servicing the extra Nagios polling, unless they are overloaded already.<o:p></o:p> <o:p></o:p>Since I run Nagios on virtual machines, however, I tried to keep the load on my failover Nagios server minimized. My failover Nagios server runs a cron job that uses the check_nagios plugin to monitor the state of the primary Nagios server. If the primary server is up and running, the failover server will just rsync the state and configuration files from the primary. If the primary server becomes unavailable, the cron job will start the Nagios service on the failover server and keep it running until it detects the primary has recovered.<o:p></o:p> <o:p></o:p><div><div style='border:none;border-top:solid windowtext 1.0pt;padding:3.0pt 0in 0in 0in;border-color:-moz-use-text-color -moz-use-text-color'>From: <a href="mailto:ravishankar.gundlapali@wipro.com">ravishankar.gundlapali@wipro.com</a> [<a href="mailto:ravishankar.gundlapali@wipro.com">mailto:ravishankar.gundlapali@wipro.com</a>] Sent: Monday, August 16, 2010 7:45 AM To: <a href="mailto:nagios-users@lists.sourceforge.net">nagios-users@lists.sourceforge.net</a> Subject: [Nagios-users] Using two nagios servers...<o:p></o:p></div></div> <o:p></o:p>Hi All,<o:p></o:p> <o:p></o:p>I am planning to configure all the servers in my client environment in two Nagios servers(in two different locations) in order to create Back up.<o:p></o:p> <o:p></o:p>Please let me know whether there will be any overload on the servers as two Nagios servers will be polling them.<o:p></o:p> <o:p></o:p> <o:p></o:p>Thanks,<o:p></o:p>Ravi G<o:p></o:p><pre><o:p> </o:p></pre><pre><o:p> </o:p></pre><pre>------------------------------------------------------------------------------<o:p></o:p></pre><pre>This SF.net email is sponsored by <o:p></o:p></pre><pre><o:p> </o:p></pre><pre>Make an app they can't live without<o:p></o:p></pre><pre>Enter the BlackBerry Developer Challenge<o:p></o:p></pre><pre><a href="http://p.sf.net/sfu/RIM-dev2dev">http://p.sf.net/sfu/RIM-dev2dev</a> <o:p></o:p></pre><pre><o:p> </o:p></pre><pre><o:p> </o:p></pre><pre>_______________________________________________<o:p></o:p></pre><pre>Nagios-users mailing list<o:p></o:p></pre><pre><a href="mailto:Nagios-users@lists.sourceforge.net">Nagios-users@lists.sourceforge.net</a><o:p></o:p></pre><pre><a href="https://lists.sourceforge.net/lists/listinfo/nagios-users">https://lists.sourceforge.net/lists/listinfo/nagios-users</a><o:p></o:p></pre><pre>::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. <o:p></o:p></pre><pre>::: Messages without supporting info will risk being sent to /dev/null<o:p></o:p></pre></div></body></html>