Unexplainable disk space usage in nagios/var/spool/checkresults

[Marc Powell]

On Jan 26, 2010, at 9:44 AM, Casey Allen Shobe wrote:

> Hi!
> 
> I have Nagios installed on a 1GB partition, with everything installed under a nagios directory on that partition.  I did not expect Nagios to start consuming a large amount of space, and now that it is, I cannot determine why or in what files, which is baffling.
> 
> I've found the biggest culprit is the var/spool/checkresults directory within the Nagios installation.  a `du` command on the directory shows that it is using 287MB.  However, when I `ls -la` within that directory, I can't see any file using more than a single kilobyte, and there aren't that many files.  However, an `ls -ld` on the directory itself shows that the directory itself is using 286MB.  That doesn't make sense to me, and so I'm looking for input! 

My gut instinct is that this is jfs doing something unexpected, in which case I can't offer much advice. The first thing I'd verify though is that nagios isn't actually still holding a bunch of files open in that directory, even after they've been removed. The disk space would still be seen as used even though you can't see the files any longer. Use 'lsof | grep checkresults' and see if you have any output for files that can't be seen in the directory any longer. I expect that you would see thousands or more if that is the case.

> I wonder if the following approach would help correct things (but I'd like to understand why they're in this state first): Stop nagios, move the checkresults directory to checkresults.old, make a new checkresults directory, copy all the old files into it, remove the empty .old directory, start nagios.

That would certainly work in either case...

--
marc
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null