vanished bug numbers in the nagios mantis tracker

[John P. Rouillard]

In message <501F8326.9000701 at op5.se>,
Andreas Ericsson writes:
>On 08/03/2012 05:15 PM, John P. Rouillard wrote:
>> In message <501BDA9E.8070606 at op5.se>,
>> Andreas Ericsson writes:
>>> On 08/03/2012 10:27 AM, Wim Hoekman wrote:
>>>> On 2012-08-01 10:37, Andreas Ericsson wrote:
>>>>> This one might resolve itself in Nagios 4, when we change how checks
>>>>> are run. I won't investigate it further until that's out anyway though.
>>>> The release of Nagios 4 would be a great time to import the Nagios patch
>>>> I've submitted a while ago.
>>>> (Subject: Additional image for action_url in host and service
>>>> definitions patch)
>>>>
>>>> That patch required a change to the object abi, so it was not included
>>>> in any minor release.
>>>
>>> Is there a reason why we just can't make this a custom variable with
>>> special meaning in the UI?
>>>
>>> I'm especially uninterested in patches that move more UI code into the
>>> core, since that's something that really should be going away rather
>>> than increase.
>> 
>> One of the things that frustrated me to no end when creating the
>> external correlation event broker
>> (http://www.cs.umb.edu/~rouilj/sec_nagios/nagios_sec_manual.txt) was
>> that the broker architecture provided no mechanism for the plugin to
>> add items to the objects and provide parsers for the objects in config
>> files.
>
>That would be pretty awesome, but "adding items to the object" is simply
>not possible. C-objects are carved in stone once the code is compiled,
>and nothing but the module will ever know or care about the extra data.

So add a field that is a pointer to a nagios maintained data store for
each object. Where each object in the datastore is a key/value (blob
pointer) pair. "adding items to the object" wasn't meant as add a new
field. More along the lines of add a new item to a linked list in the
object.

>> Custom variables help a little bit but introduce the need to parse the
>> value on every event which is not acceptable. The value should be
>> parsed once into a quickly accessible form (e.g. bitfield) and carried
>> around with the object.
>> 
>Well, no. You might want to add things other than flags in such variables
>and you may well have several modules at once competing for the dubious
>honor of putting extra junk into objects.

The object carries the data as a linked list, static array of
pointers.... rather than as a named item in the struct. As long as the
name for each item is unique there isn't a problem. (e.g. ec_ in my
examples). Nagios could even provide a register function to allow
modules to determine which prefixes are already registered so if Eric
Charles' plugin tries to register ec_ but my event correlation plugin
already has ec_ loaded, it gets an error message and gets to chose a
new prefix. Heck nagios could even return a prefix code that is used
by the module. There are many ways to handle the issue.

>> Are there any plans to add something like:
>> 
>>   add_new_custom_variable_property(SERVICE_OBJECT, "_ec_active_action", \
>>      func_parse_ec_active_action);
>> 
>>    where:
>>        void * func_parse_ec_active_action(char * property_name,
>>                          char * property_value)
>> 
>>   int add_property_value(*service_object, "_ec_active_action", void * valu
>e)
>> 
>>   void * get_property(*service_object, "_ec_active_action");
>> 
>> to allow the brokers to register parsers for specific custom
>> variables?
>> 
>
>No, and in time for Nagios 4 I doubt we'll do that since it doesn't
>really require a major version bump to be added and I just don't have
>the time for it right now. Patches are ofcourse welcome, but please
>make sure to base them on top of latest svn from sourceforge, or the
>git repo at www.github.com/ageric/nagios

Well changing the objects to provide a location in which to store this
info does require a version bump since it changes the layout of all
the object structures. But once that is done, they are more or less
futureproof.

So maybe for nagos 4 we can get a single pointer for
broker_data. Since all pointers are the same size, defining the
structure of the data and the api for accessing it can be done anytime
in the 4.x series.

>It would be neat if we could alter the way modules are loaded though,
>so one can pass parameters to them in a sane(r) way. Preferrably to
>allow users to pass core-grokable parameters that we add later.

Yeah there is realy no way to handle per nagios object settings in the
broker paramter list.

>> I envision the parsed properties (returned as a void *) are stored in
>> a linked list or array in the object struct and simply store a propery
>> name and a pointer to some blob returned by the
>> func_parse... function. The blob is opaque to nagios and only has
>> meaning to the event brokers.
>> 
>> This way brokers (and other plugins too) could add new data to the
>> internal objects without having to break abi compatibility.
>
>Well, no, they can't, because if several modules use the same mechanism
>they'll overwrite each others data or be unable to find their own.
>Unless you mean for the objects to have some sort of marker for each
>module, but that's just overly complex.

Not a marker for each module, but a marker for each data bit. Consider
(pseudo code):

  struct extended_attribute {
      string[10] identifier;
      void * value;
      extended_attribute * next; 
      extended_attribute * prev; /* if you want */
  } 

The extended_attributes would be a linked list made available in an
internal nagios structure using:

    service_object {
       void * extended_attribute;
       all other service object fields;
    }

Nagios provides commands to add/remove/get pointer to items from
the extended_attribute linked list. This keeps the data with the
object easily available to all modules (all they need is the nagios
strcture and the identifier and need to be able to understand the
structure of the value). All the data is publically available in the
service, host, .... structure.

(The above could be augmented by a nagios supplied unique (for this
running instance of nagios) number/handle that is used as part of the
retreival process. I think I described such a mechanism when I was
trying to get the external correlation mechanism into nagios 2 or 3.)

>> This also neatly (IMHO) works around the issues with serializing the
>> object to disk as the custom variable mechanism can pass the info
>> around to only those who are looking for it. In most other
>> applications, the overhead of parsing the custom variable's value
>> isn't a huge issue but in the event broker loop....
>
>In my mail to Janne Aho, I suggested adding an 'id' field to all

I'll try to find that email.

>objects. That would allow modules to utilize indexed arrays or large
>bitvectors to mark certain properties in objects and look them up in
>O(1) time without having to compute a hash and deal with collisions.

But doesn't that require that the object be changed for each module so
it has it's own array or vector?

>If we couple that with a mechanism allowing one module to obtain the
>pointers of another module, any module that wants to know about
>another module (such as for example livestatus wanting to know which
>mod_gearman worker executed a particular check) can just grab the
>ancillary data of that module and look it up in constant time.
>
>This means that either mod_gearman has to maintain a key/value vector
>for livestatus to use (or some other common form that all modules
>know about), or that livestatus has to know about the other module's
>dataformat, but it's pretty much the best we can do to allow constant
>time lookups of ancillary data.

Agreed if constant time lookup is required. I think that could work in
my case. Would be ugly to write and annoying to troubleshoot/debug but
it could be made to work and would provide the fastest lookup option.
But I fail to see how that would keep the object abi static while
allowing multiple modules to add/remove/change data.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/