<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> </head> <body bgcolor="#ffffff" text="#000099"> I've observed an interesting issue with Nagios. Our environment is a mix of UNIX, Linux, Apple, and Windows. The core of the network is Active Directory including two AD servers that are both our primary, internal DNS servers. All non-Windows systems have a resolv.conf that looks like: <blockquote>nameserver 10.1.1.13 nameserver 10.1.1.14 domain int.our.domain search int.our.domain </blockquote> About half of the servers have the nameserver entries inverted (ie: .14 first, .13 second). The issue is that anytime one of the nameservers is rebooted (at least once a month if staying current on patches thanks to Black Tuesdays), whichever hosts have that nameserver listed first in its resolv.conf start throwing the following errors: <blockquote>CRITICAL - Plugin timed out while executing system call. </blockquote> This occurs for multiple tests for each host. Obviously, there's a name resolution correlation here. If the nameserver with .13 is rebooted, all hosts (about half of them) that list this IP first in their resolve.conf then timeout for multiple tests. If the .14 server is rebooted, all the other hosts timeout. Interestingly, none of the Windows clients issue errors... only UNIX, Linux, and Mac's... only those with an /etc/resolv.conf. The end result is a host of "false positives", but more importantly it looks bad on availability reports and causes phones/pagers to go ballistic with unneeded emails. I'm trying to find a solution and I can't find one that I like: Solution 1) is to cluster the DNS servers. We have lots of clusters here. This isn't good, though, as you don't normally cluster DNS servers... they're meant to be redundant for a reason... one fails and it uses the next one. Solution 2) is to setup a service/host dependency. My thought would be either a host dependency that says if either .13 or .14 are down, then don't alert for any other host that uses them. Or a service to host dependency... if the DNS service is down, then don't alert on any of these dependent hosts. Honestly, I'm not sure if you can mix host and service dependencies like this... plus... if the DNS server is actually down, then the DNS service is down, so better to use a host dependency. The problem is that now we're not alerting on any dependent hosts which themselves could have a legitimate issue we want to know about. Plus, what happens if the DNS server actually dies and take a few hours/days to rebuild/restore? At this point, the dependent hosts aren't watched for a very long time. Solution 3) is to setup a UNIX/Linux DNS server that slaves all zones from the AD servers and have all UNIX/Linux/Apple clients query from this server. This would work except that A) I need two of them to keep redundancy and B) I've now added an extra layer of complication to resolve an application (Nagios)... not exactly good practice. Solution 4) is to set the timeout value of a host querying a DNS server. Perhaps adjust the client to timeout on the first listed nameserver after only 10 seconds, then try the next one? Since most Nagios tests have a minimum timeout value of 30 seconds, if the first DNS query timed out after 10 seconds, it would go to the next one with, hopefully, enough time to respond. The downside is having to adjust every single server. Has anyone else seen this? Anyone else using Windows AD servers to provide DNS for *nix servers? <pre class="moz-signature" cols="72">-- A. Davis Email: <a class="moz-txt-link-abbreviated" href="mailto:nccomp@gmail.com">nccomp@gmail.com</a> "There is no limit to what a man can accomplish if he doesn't care who gets the credit." - Ronald Reagan </pre> </body> </html>