Nagios check latency
Justin Pryzby
justinp at norchemlab.com
Wed Aug 21 05:30:02 CEST 2013
I would have written: ps -fC nagios3 (but I guess your daemon is
called "nagios"). Note that all the "2ndary" lines are commands being
run (checks, notifications). This will show only the parent
"scheduler" processes: ps -O ppid -C nagios3 |awk '$2==1'
Justin
On Wed, Aug 21, 2013 at 02:49:27AM +0000, Muhamad Faiz wrote:
> Hi Justin,
>
> Is this the right way to check this? I have stop and start back only one line exist now. I'm watching the latency now.
>
> [root at nagios ~]# ps faux | grep nagios.cfg
> root 21594 0.0 0.0 103244 832 pts/0 S+ 10:42 0:00 \_ grep nagios.cfg
> nagios 29669 52.4 13.6 1677324 1095904 ? Rsl Aug13 6014:30 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21086 0.3 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21291 0.5 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21293 0.5 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21295 0.5 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21296 0.5 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21338 0.6 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21346 0.4 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21453 1.0 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21519 1.5 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21521 2.0 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21530 2.5 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21538 2.5 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21547 2.0 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21555 1.0 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21579 4.0 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21586 4.0 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21589 4.0 13.6 1677328 1094340 ? S 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
> nagios 21591 2.0 13.6 1677324 1094328 ? R 10:42 0:00 \_ /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
>
> Regards,
>
> Faiz
> Monitoring
>
>
> -----Original Message-----
> From: Justin Pryzby [mailto:justinp at norchemlab.com]
> Sent: Tuesday, August 13, 2013 9:51 PM
> To: nagios-users at lists.sourceforge.net
> Subject: Re: [Nagios-users] Nagios check latency
>
> Could you check that you're not accidentally running 2 nagios daemons?
>
> On Tue, Aug 13, 2013 at 03:49:19AM +0000, Muhamad Faiz wrote:
> > Hi guys,
> >
> > My Nagios setup are having high service check latency.
> >
> > Check nagios latency result is 240 seconds!
> >
> > ./check_nagios_latency.sh -w 30 -c 60
> > CRITICAL: Latency at 240.183 seconds!
> >
> > Modification in my Nagios.cfg file is only as below. Other than this, is default.
> >
> > # MODIFIED
> > admin_email=muhamad.faiz at asia.xchanging.com
> > admin_pager=muhamad.faiz at asia.xchanging.com
> > translate_passive_host_checks=1
> > log_event_handlers=0
> > use_large_installation_tweaks=1
> > enable_environment_macros=0
> > enable_embedded_perl=0
> > use_embedded_perl_implicitly=0
> > max_concurrent_checks=1600
> > service_perfdata_file_processing_command=process-service-perfdata-file
> > -pnp-bulk
> > host_perfdata_file_processing_command=process-host-perfdata-file-pnp-b
> > ulk
> > service_perfdata_file_processing_interval=60
> > host_perfdata_file_processing_interval=60
> > status_update_interval=40
> >
> >
> > # /usr/local/nagios/bin/nagios -s /usr/local/nagios/etc/nagios.cfg
> >
> > Nagios Core 3.5.0
> > Copyright (c) 2009-2011 Nagios Core Development Team and Community
> > Contributors Copyright (c) 1999-2009 Ethan Galstad Last Modified:
> > 03-15-2013
> > License: GPL
> >
> > Website: http://www.nagios.org
> > Timing information on object configuration processing is listed below.
> > You can use this information to see if precaching your object
> > configuration would be useful.
> >
> > Object Config Source: Config files (uncached)
> >
> > OBJECT CONFIG PROCESSING TIMES (* = Potential for precache savings with -u option)
> > ----------------------------------
> > Read: 0.415803 sec
> > Resolve: 0.006900 sec *
> > Recomb Contactgroups: 0.001862 sec *
> > Recomb Hostgroups: 3.819786 sec *
> > Dup Services: 0.637380 sec *
> > Recomb Servicegroups: 3.419804 sec *
> > Duplicate: 0.000417 sec *
> > Inherit: 0.013748 sec *
> > Recomb Contacts: 0.000000 sec *
> > Sort: 0.000000 sec *
> > Register: 0.616385 sec
> > Free: 0.029930 sec
> > ============
> > TOTAL: 8.962015 sec * = 7.899897 sec (88.15%) estimated savings
> >
> >
> > RETENTION DATA TIMES
> > ----------------------------------
> > Read and Process: 0.863815 sec
> > ============
> > TOTAL: 0.863815 sec
> >
> >
> > Timing information on configuration verification is listed below.
> >
> > CONFIG VERIFICATION TIMES (* = Potential for speedup with -x option)
> > ----------------------------------
> > Object Relationships: 0.080360 sec
> > Circular Paths: 0.001152 sec *
> > Misc: 0.011244 sec
> > ============
> > TOTAL: 0.092756 sec * = 0.001152 sec (1.2%) estimated savings
> >
> >
> > EVENT SCHEDULING TIMES
> > -------------------------------------
> > Get service info: 0.051265 sec
> > Get host info info: 0.032301 sec
> > Get service params: 0.000010 sec
> > Schedule service times: 0.002476 sec
> > Schedule service events: 0.002514 sec
> > Get host params: 0.000000 sec
> > Schedule host times: 0.000937 sec
> > Schedule host events: 0.001287 sec
> > ============
> > TOTAL: 0.090790 sec
> >
> >
> > Projected scheduling information for host and service checks is listed
> > below. This information assumes that you are going to start running
> > Nagios with your current config files.
> >
> > HOST SCHEDULING INFORMATION
> > ---------------------------
> > Total hosts: 11753
> > Total scheduled hosts: 7
> > Host inter-check delay method: SMART
> > Average host check interval: 300.00 sec
> > Host inter-check delay: 42.86 sec
> > Max host check spread: 30 min
> > First scheduled check: Tue Aug 13 11:43:54 2013
> > Last scheduled check: Tue Aug 13 11:43:54 2013
> >
> >
> > SERVICE SCHEDULING INFORMATION
> > -------------------------------
> > Total services: 18662
> > Total scheduled services: 170
> > Service inter-check delay method: SMART
> > Average service check interval: 535.76 sec
> > Inter-check delay: 3.15 sec
> > Interleave factor method: SMART
> > Average services per host: 1.59
> > Service interleave factor: 1
> > Max service check spread: 30 min
> > First scheduled check: Tue Aug 13 11:52:49 2013
> > Last scheduled check: Tue Aug 13 11:58:39 2013
> >
> >
> > CHECK PROCESSING INFORMATION
> > ----------------------------
> > Check result reaper interval: 10 sec
> > Max concurrent service checks: 1600
> >
> >
> > PERFORMANCE SUGGESTIONS
> > -----------------------
> > I have no suggestions - things look okay.
> >
> > Any advice will be much appreciated.
> >
> > Regards,
> > Faiz
> > "CONFIDENTIALITY NOTICE: This message and any attachment are confidential and may also be privileged. If you are not the intended recipient of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you are not the intended recipient please telephone or e-mail the sender and delete this message and any attachment from your system."
>
> > BEGIN:VCARD
> > VERSION:2.1
> > X-MS-SIGNATURE:YES
> > N;LANGUAGE=en-us:Faiz
> > FN:Faiz
> > ORG:YTLC Network Operations Monitoring Team
> > TEL;WORK;VOICE:+60 192260176
> > ADR;WORK;PREF:;;;;;;United States of America
> > X-MS-OL-DEFAULT-POSTAL-ADDRESS:2
> > EMAIL;PREF;INTERNET:Muhamad.Faiz at Asia.Xchanging.com
> > X-MS-IMADDRESS:Skype: me.faiz
> > X-MS-CARDPICTURE;TYPE=JPEG;ENCODING=BASE64:
> >
> > /9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAcFBQYFBAcGBQYIBwcIChELCgkJChUPEAwRGB
> > Ua
> > GRgVGBcbHichGx0lHRcYIi4iJSgpKywrGiAvMy8qMicqKyr/2wBDAQcICAoJChQLCxQqHB
> > gc
> > KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKir/wA
> > AR
> > CAAVADYDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtR
> > AA
> > AgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2Jygg
> > kK
> > FhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhI
> > WG
> > h4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+
> > Tl
> > 5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtR
> > EA
> > AgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRCh
> > Yk
> > NOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6go
> > OE
> > hYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4u
> > Pk
> > 5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD2/wAWnWYNI+1aDciKSAlpUKBg6dyPkY
> > 5H
> >
> > oBzzXK6d8SW07SpB4hVp7zf+5EcboZV753RoBj8c5rf+IXjOHwV4Ye7ba95NmO0hP8b46k
> > f3
> > R1P4DvXinhexuPFE51rU7YLbo5QFIwkckmMkDBXGMg8GvUwdCFWF6i0/F+R5OPxE6WlJ6/
> > gv
> > M9w0HxhDrWmz6hPbNYWcLBfOncbWJOPyGRz71uw3ttPGksE8ciO7RqysMMwJBA9SNp/I1h
> > xx
> > 6VP4HazgdrOxeL7PIIwGdAxww78nJ55POa5+HQfCyqInvnuYY12KkkCuUULsVd2zdtUYwP
> > UZ
> >
> > OSa5XTjKUuVNK/qdFKrKNOPPJNtb7Ho2ajEqGQxhxvABK55APfH4GvP4/DfhoTvdSajNKZ
> > B8
> > knlDcGDlt+4JkuCSNx5xiqmn+HtAtoWa/wBSWSZzmMQwhVj2qFDj5Bhh13ccn8SvYLu/uZ
> > bx
> > CXb7z0/rRXL+B9Ni03SrlIbmO5SW437o49mMRogzwMkhQScdSaKwlCzsbxqXVzo5LeKfBm
> > iS
> > THTeoOKq32lWl/YNazR7Ys5AjO0qfUEdKKKUZNPQc4xcWmiHSdCtdIaQ2jTESYDLJKWGfX
> > nv
> > WoEVSSABk5PHWiinKTlK7JpwjGNooasSRghFVQTkgDGSe9SYHoKKKk0sgooooGf/2Q==
> >
> > X-MS-OL-DESIGN;CHARSET=utf-8:<card
> > xmlns="http://schemas.microsoft.com/office/outlook/12/electronicbusine
> > sscards" ver="1.0" layout="right" bgcolor="ffffff"><img xmlns=""
> > align="midright" area="50" use="cardpicture"/><fld xmlns=""
> > prop="name" align="left" dir="ltr" style="b" color="000000"
> > size="10"/><fld xmlns="" prop="org" align="left" dir="ltr"
> > color="000000" size="8"/><fld xmlns="" prop="title" align="left"
> > dir="ltr" color="000000" size="8"/><fld xmlns="" prop="dept"
> > align="left" dir="ltr" color="000000" size="8"/><fld xmlns=""
> > prop="blank" size="8"/><fld xmlns="" prop="telwork" align="left"
> > dir="ltr" color="000000" size="8"/><fld xmlns="" prop="email"
> > align="left" dir="ltr" color="000000" size="8"/><fld xmlns=""
> > prop="addrwork" align="left" dir="ltr" color="000000" size="8"/><fld
> > xmlns="" prop="blank" size="8"/><fld xmlns="" prop="im" align="left"
> > dir="ltr" color="000000" size="8"/><fld xmlns="" prop="blank"
> > size="8"/><fld xmlns="" prop="blank" size="8"/><fld xmlns=""
> > prop="blank" size="8"/><fld xmlns="" prop="blank" size="8"/><fld
> > xmlns="" prop="blank" size="8"/><fld xmlns="" prop="blank"
> > size="8"/></card> REV:20121031T020436Z END:VCARD
>
> > ----------------------------------------------------------------------
> > -------- Get 100% visibility into Java/.NET code with AppDynamics
> > Lite!
> > It's a free troubleshooting tool designed for production.
> > Get down to code-level detail for bottlenecks, with <2% overhead.
> > Download for free and get started troubleshooting in minutes.
> > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.c
> > lktrk
>
> > _______________________________________________
> > Nagios-users mailing list
> > Nagios-users at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/nagios-users
> > ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
> > ::: Messages without supporting info will risk being sent to /dev/null
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> Nagios-users mailing list
> Nagios-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nagios-users
> ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
> ::: Messages without supporting info will risk being sent to /dev/null
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance Management.
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> _______________________________________________
> Nagios-users mailing list
> Nagios-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nagios-users
> ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
> ::: Messages without supporting info will risk being sent to /dev/null
>
------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and
AppDynamics. Performance Central is your source for news, insights,
analysis and resources for efficient Application Performance Management.
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
::: Messages without supporting info will risk being sent to /dev/null
More information about the Users
mailing list