Major speedup of CGI
Jason Lancaster
jason at teklabs.net
Wed Sep 17 21:05:25 CEST 2003
I'm wondering if anyone has been able to successfully implement this or
a similar patch to Nagios-1.1. The patch fails for me with hunk failed
errors. If anyone has a copy of a different patch that works, could you
send a copy my way?
Thanks,
Jason Lancaster
David Parrish wrote:
> Attached is another patch to Nagios 1.1 which hugely speeds up the CGIs.
> This is a big benefit when monitoring a lot (~ 7000) of services, but there
> will be a benefit for a smaller number too.
>
> Profiling shows that most of the time taken to run the CGI is in loading
> the list of services and sticking them into a sorted list.
>
> This patch turns loading the service list into a 2 step process:
>
> 1. Load the services into a binary tree (glib)
> 2. Flatten the tree down into the sorted list.
>
> Before, the CGI took between 3-10 seconds to run, now it takes 0.2 seconds.
>
> FYI I didn't write this patch, my boss did. I'm just submitting it.
>
>
>
> ------------------------------------------------------------------------
>
> Index: base/Makefile.in
> ===================================================================
> RCS file: /home/cvs/repos/nagios_source/base/Makefile.in,v
> retrieving revision 1.1.1.1
> diff -u -r1.1.1.1 Makefile.in
> --- base/Makefile.in 23 Jun 2003 01:56:10 -0000 1.1.1.1
> +++ base/Makefile.in 5 Sep 2003 01:45:51 -0000
> @@ -10,9 +10,9 @@
> SRC_XDATA=../xdata
>
> CC=@CC@
> -CFLAGS=@CFLAGS@ @DEFS@ -DNSCORE
> +CFLAGS=@CFLAGS@ @DEFS@ -DNSCORE `glib-config --cflags`
> #CFLAGS=-O3 -Wall -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -DHAVE_CONFIG_H -DNSCORE
> -LDFLAGS=@LDFLAGS@ @LIBS@
> +LDFLAGS=@LDFLAGS@ @LIBS@ `glib-config --libs`
>
> prefix=@prefix@
> exec_prefix=@exec_prefix@
> Index: cgi/Makefile.in
> ===================================================================
> RCS file: /home/cvs/repos/nagios_source/cgi/Makefile.in,v
> retrieving revision 1.1.1.1
> diff -u -r1.1.1.1 Makefile.in
> --- cgi/Makefile.in 23 Jun 2003 01:56:10 -0000 1.1.1.1
> +++ cgi/Makefile.in 5 Sep 2003 01:45:51 -0000
> @@ -25,9 +25,9 @@
>
> CP=@CP@
> CC=@CC@
> -CFLAGS=@CFLAGS@ @DEFS@ -DNSCGI
> +CFLAGS=@CFLAGS@ @DEFS@ -DNSCGI `glib-config --cflags`
> #CFLAGS=-O3 -Wall -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -DHAVE_CONFIG_H -DNSCGI
> -LDFLAGS=@LDFLAGS@ @LIBS@
> +LDFLAGS=@LDFLAGS@ @LIBS@ `glib-config --libs`
>
> CGIS=avail.cgi cmd.cgi config.cgi extinfo.cgi history.cgi notifications.cgi outages.cgi showlog.cgi status.cgi statuswml.cgi summary.cgi tac.cgi $(CGIEXTRAS)
>
> Index: common/common.h
> ===================================================================
> RCS file: /home/cvs/repos/nagios_source/common/common.h,v
> retrieving revision 1.1.1.2
> diff -u -r1.1.1.2 common.h
> --- common/common.h 23 Jun 2003 01:57:24 -0000 1.1.1.2
> +++ common/common.h 5 Sep 2003 01:45:51 -0000
> @@ -208,8 +208,12 @@
> #define OK 0
> #define ERROR -2 /* value was changed from -1 so as to not interfere with STATUS_UNKNOWN plugin result */
>
> +#ifndef TRUE
> #define TRUE 1
> +#endif
> +#ifndef FALSE
> #define FALSE 0
> +#endif
>
>
> /****************** HOST CONFIG FILE READING OPTIONS ********************/
> Index: common/config.h.in
> ===================================================================
> RCS file: /home/cvs/repos/nagios_source/common/config.h.in,v
> retrieving revision 1.1.1.1
> diff -u -r1.1.1.1 config.h.in
> --- common/config.h.in 23 Jun 2003 01:56:10 -0000 1.1.1.1
> +++ common/config.h.in 5 Sep 2003 01:45:51 -0000
> @@ -214,3 +214,7 @@
> #include <dirent.h>
> #endif
>
> +#define HAVE_GLIB_H 1
> +#ifdef HAVE_GLIB_H
> +#include <glib.h>
> +#endif
> Index: common/objects.c
> ===================================================================
> RCS file: /home/cvs/repos/nagios_source/common/objects.c,v
> retrieving revision 1.1.1.1
> diff -u -r1.1.1.1 objects.c
> --- common/objects.c 23 Jun 2003 01:56:10 -0000 1.1.1.1
> +++ common/objects.c 5 Sep 2003 01:45:51 -0000
> @@ -59,6 +59,8 @@
> servicedependency *servicedependency_list=NULL;
> hostdependency *hostdependency_list=NULL;
> hostescalation *hostescalation_list=NULL;
> +GTree *service_tree = NULL;
> +GHashTable *host_hash = NULL;
>
>
>
> @@ -99,6 +101,8 @@
> printf("read_object_config_data() end\n");
> #endif
>
> + build_config_lists();
> +
> return result;
> }
>
> @@ -107,6 +111,18 @@
> /**************** OBJECT ADDITION FUNCTIONS ***********************/
> /******************************************************************/
>
> +int service_list_add(char *key, service *v, void *data)
> +{
> + v->next = service_list;
> + service_list = v;
> + return 0;
> +}
> +
> +void build_config_lists(void)
> +{
> + g_tree_traverse(service_tree, service_list_add, G_IN_ORDER, NULL);
> +}
> +
>
>
> /* add a new timeperiod to the list in memory */
> @@ -747,6 +763,16 @@
> printf("\tNotification Interval: %d\n",new_host->notification_interval);
> printf("\tNotification Time Period: %s\n",new_host->notification_period);
> #endif
> +
> +#ifdef HAVE_GLIB_H
> + /* Cache host in hash table */
> + if (!host_hash)
> + host_hash = g_hash_table_new(g_str_hash, g_str_equal);
> +
> + if (host_hash)
> + g_hash_table_insert(host_hash, new_host->name, new_host);
> +#endif
> +
> #ifdef DEBUG0
> printf("add_host() end\n");
> #endif
> @@ -2173,40 +2199,7 @@
> strcpy(new_service->perf_data,"");
>
> #endif
> - /* add new service to service list, sorted by host name then service description */
> - last_service=service_list;
> - for(temp_service=service_list;temp_service!=NULL;temp_service=temp_service->next){
> -
> - if(strcmp(new_service->host_name,temp_service->host_name)<0){
> - new_service->next=temp_service;
> - if(temp_service==service_list)
> - service_list=new_service;
> - else
> - last_service->next=new_service;
> - break;
> - }
> -
> - else if(strcmp(new_service->host_name,temp_service->host_name)==0 && strcmp(new_service->description,temp_service->description)<0){
> - new_service->next=temp_service;
> - if(temp_service==service_list)
> - service_list=new_service;
> - else
> - last_service->next=new_service;
> - break;
> - }
>
> - else
> - last_service=temp_service;
> - }
> - if(service_list==NULL){
> - new_service->next=NULL;
> - service_list=new_service;
> - }
> - else if(temp_service==NULL){
> - new_service->next=NULL;
> - last_service->next=new_service;
> - }
> -
> #ifdef DEBUG1
> printf("\tHost: %s\n",new_service->host_name);
> printf("\tDescription: %s\n",new_service->description);
> @@ -2222,6 +2215,19 @@
> printf("\tEvent Handler: %s\n",(new_service->event_handler==NULL)?"N/A":new_service->event_handler);
> #endif
>
> +#ifdef HAVE_GLIB_H
> + /* Cache service in tree */
> + if (!service_tree)
> + service_tree = g_tree_new(strcmp);
> +
> + if (service_tree)
> + {
> + char *key = calloc(strlen(new_service->host_name) + strlen(new_service->description) + 2, 1);
> + sprintf(key, "%s-%s", new_service->host_name, new_service->description);
> + g_tree_insert(service_tree, key, new_service);
> + }
> +#endif
> +
> #ifdef DEBUG0
> printf("add_service() end\n");
> #endif
> @@ -3226,6 +3232,13 @@
> if(name==NULL)
> return NULL;
>
> +#ifdef HAVE_GLIB_H
> + /* Lookup host in the hash */
> + if (host_hash)
> + return (host *)g_hash_table_lookup(host_hash, name);
> +#endif
> +
> +
> if(hst==NULL)
> temp_host=host_list;
> else
> @@ -3499,6 +3512,18 @@
>
> if(host_name==NULL || svc_desc==NULL)
> return NULL;
> +
> +#ifdef HAVE_GLIB_H
> + /* Lookup service in the tree */
> + if (service_tree)
> + {
> + char *key = calloc(strlen(host_name) + strlen(svc_desc) + 2, 1);
> + sprintf(key, "%s-%s", host_name, svc_desc);
> + temp_service = (service *)g_tree_lookup(service_tree, key);
> + free(key);
> + return temp_service;
> + }
> +#endif
>
> if(svcptr==NULL)
> temp_service=service_list;
> Index: common/statusdata.c
> ===================================================================
> RCS file: /home/cvs/repos/nagios_source/common/statusdata.c,v
> retrieving revision 1.1.1.1
> diff -u -r1.1.1.1 statusdata.c
> --- common/statusdata.c 23 Jun 2003 01:56:10 -0000 1.1.1.1
> +++ common/statusdata.c 5 Sep 2003 01:45:51 -0000
> @@ -51,6 +51,7 @@
> #ifdef NSCGI
> hoststatus *hoststatus_list=NULL;
> servicestatus *servicestatus_list=NULL;
> +GTree *servicestatus_tree=NULL;
>
> time_t program_start;
> int daemon_mode;
> @@ -332,6 +333,7 @@
> result=xsddb_read_status_data(config_file,options);
> #endif
>
> + build_lists();
> return result;
> }
>
> @@ -341,6 +343,17 @@
> /********************** ADDITION FUNCTIONS ************************/
> /******************************************************************/
>
> +int servicestatus_list_add(servicestatus *key, void *v, void *data)
> +{
> + key->next = servicestatus_list;
> + servicestatus_list = key;
> + return 0;
> +}
> +
> +void build_lists(void)
> +{
> + g_tree_traverse(servicestatus_tree, servicestatus_list_add, G_IN_ORDER, NULL);
> +}
>
> /* sets program status variables */
> int add_program_status(time_t _program_start, int _nagios_pid, int _daemon_mode, time_t _last_command_check, time_t _last_log_rotation, int _enable_notifications,int _execute_service_checks,int _accept_passive_service_checks,int _enable_event_handlers,int _obsess_over_services, int _enable_flap_detection, int _enable_failure_prediction, int _process_performance_data){
> @@ -534,6 +547,16 @@
> return OK;
> }
>
> +int servicestatus_cmp(servicestatus *a, servicestatus *b)
> +{
> + int c;
> +
> + c = strcmp(b->host_name, a->host_name);
> + if (c)
> + return c;
> +
> + return strcmp(b->description, a->description);
> +}
>
> /* adds a service status entry to the list in memory */
> int add_service_status(char *host_name,char *svc_description,char *status_string,time_t last_update,int current_attempt,int max_attempts,int state_type,time_t last_check,time_t next_check,int check_type,int checks_enabled,int accept_passive_checks,int event_handler_enabled,time_t last_state_change,int problem_has_been_acknowledged,char *last_hard_state_string,unsigned long time_ok,unsigned long time_warning,unsigned long time_unknown,unsigned long time_critical,time_t last_notification,int current_notification_number,int notifications_enabled, int latency, int execution_time, int flap_detection_enabled, int is_flapping, double percent_state_change, int scheduled_downtime_depth, int failure_prediction_enabled, int process_performance_data, int obsess_over_service, char *plugin_output){
> @@ -543,6 +566,9 @@
> int status;
> int last_hard_state;
>
> + if (!servicestatus_tree) {
> + servicestatus_tree = g_tree_new( (GCompareFunc) servicestatus_cmp);
> + }
>
> /* make sure we have what we need */
> if(host_name==NULL)
> @@ -724,41 +750,7 @@
> /* scheduled downtime depth */
> new_svcstatus->scheduled_downtime_depth=scheduled_downtime_depth;
>
> -
> - /* add new service status to list, sorted by host name then description */
> - last_svcstatus=servicestatus_list;
> - for(temp_svcstatus=servicestatus_list;temp_svcstatus!=NULL;temp_svcstatus=temp_svcstatus->next){
> -
> - if(strcmp(new_svcstatus->host_name,temp_svcstatus->host_name)<0){
> - new_svcstatus->next=temp_svcstatus;
> - if(temp_svcstatus==servicestatus_list)
> - servicestatus_list=new_svcstatus;
> - else
> - last_svcstatus->next=new_svcstatus;
> - break;
> - }
> -
> - else if(strcmp(new_svcstatus->host_name,temp_svcstatus->host_name)==0 && strcmp(new_svcstatus->description,temp_svcstatus->description)<0){
> - new_svcstatus->next=temp_svcstatus;
> - if(temp_svcstatus==servicestatus_list)
> - servicestatus_list=new_svcstatus;
> - else
> - last_svcstatus->next=new_svcstatus;
> - break;
> - }
> -
> - else
> - last_svcstatus=temp_svcstatus;
> - }
> - if(servicestatus_list==NULL){
> - new_svcstatus->next=NULL;
> - servicestatus_list=new_svcstatus;
> - }
> - else if(temp_svcstatus==NULL){
> - new_svcstatus->next=NULL;
> - last_svcstatus->next=new_svcstatus;
> - }
> -
> + g_tree_insert(servicestatus_tree, new_svcstatus, new_svcstatus);
>
> return OK;
> }
> Index: contrib/Makefile.in
> ===================================================================
> RCS file: /home/cvs/repos/nagios_source/contrib/Makefile.in,v
> retrieving revision 1.1.1.2
> diff -u -r1.1.1.2 Makefile.in
> --- contrib/Makefile.in 23 Jun 2003 01:57:24 -0000 1.1.1.2
> +++ contrib/Makefile.in 5 Sep 2003 01:45:51 -0000
> @@ -5,8 +5,8 @@
> ###############################
>
> CC=@CC@
> -CFLAGS=@CFLAGS@ @DEFS@
> -LDFLAGS=@LDFLAGS@ @LIBS@
> +CFLAGS=@CFLAGS@ @DEFS@ `glib-config --cflags`
> +LDFLAGS=@LDFLAGS@ @LIBS@ `glib-config --libs`
>
> # Source code directories
> SRC_COMMON=../common
> Index: html/Makefile.in
> ===================================================================
> RCS file: /home/cvs/repos/nagios_source/html/Makefile.in,v
> retrieving revision 1.1.1.1
> diff -u -r1.1.1.1 Makefile.in
> --- html/Makefile.in 23 Jun 2003 01:56:10 -0000 1.1.1.1
> +++ html/Makefile.in 5 Sep 2003 01:45:51 -0000
> @@ -1,6 +1,6 @@
> CC=@CC@
> -CFLAGS=@CFLAGS@ @DEFS@
> -LDFLAGS=@LDFLAGS@ @LIBS@
> +CFLAGS=@CFLAGS@ @DEFS@ `glib-config --cflags`
> +LDFLAGS=@LDFLAGS@ @LIBS@ `glib-config --libs`
>
> prefix=@prefix@
> exec_prefix=@exec_prefix@
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
More information about the Developers
mailing list