topic Re: Nagios script output issue in Operating System - HP-UX

Nagios script output issue

Allanm — Fri, 27 Aug 2010 04:27:20 GMT

Hi Folks,

Nagios is acting a little weird for me, I have this external script which I hooked into Nagios, it merely does a curl/wget on a URL and returns the status based on string in the content/output. Initially for 2-3 hrs the script returns the right status and Nagios reports correctly i.e. OK, WARN, ERROR based on the exit from the script. After 2-3 hrs output which was (and should be) OK or WARN starts returning CRITICAL and the output line says "Application is" and not even "Application is ERROR" or "Application is FATAL".

There is nothing in the logs to suggest what could be the problem. Have you experienced this before and let me know the corrective action. I am running Nagios on Mac OSX.

Here is the script for the curious -

#!/bin/bash

read URL < "$1"

STATUS=`curl -s $URL |grep summary|awk -F\" '{print $2}'`
echo "Application is $STATUS"
echo "curl $URL"

case $STATUS in
OK)
exit 0
;;
WARN)
exit 1
;;
ERROR)
exit 2
;;
FATAL)
exit 2
;;
*)
exit 2
;;
esac

Thanks,

Allan.

Re: Nagios script output issue

Modris Bremze — Fri, 27 Aug 2010 04:55:21 GMT

If output says "Critical", it means exit code 2. In your script exit code 2 is when you receive "Fatal" or any other value (*) than "Ok", "Error" or "Critical".
I suggest you change exit code for * to 3 ("Unknown") and carefully inspect possible outputs from
curl -s $URL |grep summary|awk -F\" '{print $2} - it probably starts to return nothing or something you're not expecting at some point.

Re: Nagios script output issue

Modris Bremze — Fri, 27 Aug 2010 04:57:24 GMT

Got a little mistake there - Critical status in your script is for Fatal, Error and any value other than Warn or Ok. Sorry.

Re: Nagios script output issue

James R. Ferguson — Fri, 27 Aug 2010 10:56:15 GMT

Hi Allan:

> and the output line says "Application is" and not even "Application is ERROR" or "Application is FATAL".

Then your 'STATUS' variable is empty. That is, there was no match (or the 'curl' returned nothing).

As an aside you can eliminate the extra process (the 'grep') and let 'awk' to all the work:

# STATUS=`curl -s $URL|awk -F\" '/summary/ {print $2}'`

...which is better written without the backticks but as $( ... ) like:

# STATUS=$(curl -s $URL|awk -F\" '/summary/ {print $2}')

Lastly, you have a large number of questions with unassigned points (166 of 285 responses). It would be appreciated if you could evaluate the help you received.

http://forums.itrc.hp.com/service/forums/pageList.do?userId=CA1505975&listType=unassigned&forumId=1

Regards!

...JRF...

Re: Nagios script output issue

Allanm — Fri, 27 Aug 2010 14:31:15 GMT

Thanks for your answers,

When I run the script from commandline I get the correct status (Warn or OK) but those still return CRITICAL.

I would try what JRF suggested.

JRF, would assign points surely over the weekend.

Thanks,
Allan

Re: Nagios script output issue

James R. Ferguson — Fri, 27 Aug 2010 15:34:34 GMT

Hi (again) Allan:

I would run 'curl' with the '-o file' option to redirect its STDOUT into a file. Then, examine the return code from 'curl'. If that indicates success, then parse the output file as you require.

This is what I was eluding to when I said "(or the 'curl' returned nothing)".

Regards!

...JRF...

Re: Nagios script output issue

Allanm — Fri, 27 Aug 2010 16:46:33 GMT

Hey JRF,

Thanks for the reply. I am first trying the solution without the grep. As the failure happens in 2-3 hrs. I would wait and watch.

I am thinking of avoiding the output to file in order to minimize the handling of files.

Lets see how it goes.

Thanks,
Allan.

Re: Nagios script output issue

Allanm — Mon, 30 Aug 2010 00:06:58 GMT

Folks,

I switched to "check_http" plugin (official nagios plugin) but that also has same problem.

Initially when I start nagios then the status on the Nagios Web interface is same as the one returned from commandline.

After some time the status becomes critical but is not as same as the commandline. Command line returns the correct status of OK instead of what the Nagios web interface shows as "CRITICAL"

Result from commandline(& browser) -
./check_http -H xyz.com -p 2222 -u /abc -t 3
OK

Result from Nagios Interface -
nodename nor servname provided, or not known
HTTP CRITICAL - Unable to open TCP socket

I am thinking that this has something to do with Nagios as the box is behaving just fine based on the commandline result(and verified on the URL through the browser).

Please help!

Thanks,
Allan.