topic Re: monitoring script in Operating System - Linux

monitoring script

Piotr Kirklewski — Wed, 18 Jul 2007 17:05:51 GMT

Hi there

Command ps -e|grep httpd output is this :

2528 ? 00:00:00 httpd-matrix
29651 ? 00:00:00 httpd-matrix
29652 ? 00:00:00 httpd-matrix
32038 ? 00:00:00 httpd-matrix
18798 ? 00:00:00 httpd
18801 ? 00:00:00 httpd
18802 ? 00:00:00 httpd
18803 ? 00:00:00 httpd
18804 ? 00:00:00 httpd

I need to build a script to monitor httpd:

#!/bin/sh

adminmail=peterk@mark.com

if [ -z "`ps auxw|grep "httpd"|grep -v grep`" ]; then
echo "apache down!!!"|mail -s "apache is down" $adminmail

/etc/init.d/httpd start

fi

The problem is that the command ps auxw|grep "httpd"|grep -v grep` finds httpd-matrix (mongrel cluster process) and script thinks httpd is there, so it never executes /etc/init.d/httpd start.

The question is, what can I change in this command :

ps auxw|grep "httpd"|grep -v grep

to exclude httpd-matrix from the search results ?

Cheers

Re: monitoring script

David Bellamy — Wed, 18 Jul 2007 17:13:51 GMT

try ps auxw|grep "httpd"|grep -v grep|grep -v "httpd-matrix"

Re: monitoring script

James R. Ferguson — Wed, 18 Jul 2007 17:52:32 GMT

Hi:

You need to use the UNIX95 option of '-C' of 'ps':

# PIDS=`UNIX95= ps -C httpd -o pid=`

...will return an empty list if no processes with a basename of 'httpd' exist; or a list of one or more processes if it is running.

This eliminates the fuzzy matching of the process list with 'grep' which can lead to spurious matches you don't want!

Note the space (or tab) after 'UNIX95=' and before 'ps'. This sets the UNIX95 behavior only foir the duration of the command line. The equal sign after the 'pid' suppresses the header line "PID" from the output.

Regards!

...JRF...

Re: monitoring script

Steven E. Protter — Wed, 18 Jul 2007 18:13:51 GMT

Shalom,

This form also works.
PIDS=$(UNIX95= ps -C httpd -o pid=)

If you use post 1 some day you will whack a process you did not intend to whack.

I had a process kill script working like post 1 and discovered to my horror in the lab that it could take down an entire system.

It was easy enough to pull the script out of production at my current job, but I had to try and convince my previous employer to pull the script. In the end, all I could do was email management a disclaimer warning them that using this one script could damage their systems and hope for the best.

Anyway, good luck.

SEP

Re: monitoring script

Piotr Kirklewski — Thu, 19 Jul 2007 03:03:07 GMT

Fine
It works great with:

ps -e|grep "httpd"|grep -v grep|grep -v "httpd-matrix

If I run the script from command line it's working fine.

But when cron is trying to run it then I get this message:

Starting httpd: Syntax error on line 117 of /etc/httpd/conf.d/ssl.conf:

SSLCertificateFile: file '/etc/httpd/conf/ssl.crt/server.crt' does not exist or is empty [FAILED]

I don't understand why? As everythink is fine when I run the script from root account, I did chmod 777 on it, its in root crontab.

Wht's wrong ?

Re: monitoring script

Ralph Grothe — Thu, 19 Jul 2007 03:45:00 GMT

Maybe a suggestion for further "refinement"?

Usually, you want to parse the PID of the httpd parent process (as you may have dozens of children currently servicing different clients).

So you could match like this

httpd_pid=$(UNIX95= ps -e -o pid= -o ppid= -o comm=|awk '$3~/httpd/&&$2==1{print$1}')

if [[ -z $httpd_pid ]]; then
# put event handler or notification code here
fi

But even this is a poor test because you only see if some httpd parent proc is running
although the webserver for some reason might not be servicing its clients at all.
So better fetch a page or better yet, the HTTP server response to a HEAD request.
You can use any scriptable HTTP client for that like curl, wget, or even netcat or telnet.
I however, prefer the Perl LWP client reference script which comes with any current Perl installation.

E.g.

$ lwp-request -m head http://localhost/
200 OK
Connection: close
Date: Thu, 19 Jul 2007 08:39:00 GMT
Accept-Ranges: bytes
ETag: "8f3d-2028-303d3cc0"
Server: Apache/2.0.55 HP-UX_Apache-based_Web_Server (Unix) DAV/2 PHP/5.0.4 mod_jk/1.2.10
Content-Length: 8232
Content-Type: text/html
Last-Modified: Fri, 10 Feb 2006 14:03:07 GMT
Client-Date: Thu, 19 Jul 2007 08:39:00 GMT
Client-Peer: 127.0.0.1:80
Client-Response-Num: 1

But still this response is too much.
All you are usually interested in is the HTTP server's response code, normally a 200
signalling successful delivery.

$ lwp-request -m head -sd http://localhost/
200 OK

This is a painless test with a terse response, easily parsable, and much better than simply a process table lookup.

Re: monitoring script

Dennis Handly — Thu, 19 Jul 2007 03:45:10 GMT

>ps -e|grep "httpd"|grep -v grep|grep -v "httpd-matrix

Should I assume the missing quote on the end is just a typo here?

You can combine the grep -v:
$ ps -e | grep httpd | grep -v -e grep -e httpd-matrix

>If I run the script from command line it's working fine.

Is it getting to?: /etc/init.d/httpd start

When you're in a crontab, you don't have the same PATH. In fact you don't have /sbin there.

Re: monitoring script

Piotr Kirklewski — Thu, 19 Jul 2007 04:58:39 GMT

Well as I said before its working great if I run the script from comamnd line, the only problem is why its not working when cron is trying to do that ?
Other scripts that are set in crontab are working fine, so my assumption is that it's Apache only problem.
On the other server I running similar script but for Apache2 and the situation is similar but cron is comming with different error message:

(13)Permission denied: make_sock: could not bind to address [::]:80 (13)Permission denied: make_sock: could not bind to address 0.0.0.0:80 no listening sockets available, shutting down Unable to open logs

How do you think where is the problem?
As before if I run this script from command line the everythink is fine.

Cheers

Re: monitoring script

Ralph Grothe — Thu, 19 Jul 2007 05:20:42 GMT

If you are getting ssl.conf related errors
I assume that you didn't tell Apache to load the SSL engine.
I don't no what your config looks like,
but the HP port comes with this load directive in httpd.conf:

LoadModule ssl_module modules/mod_ssl.so

This means that a macro SSL is required to be set to execute this block,
which is usually achieved by something like

/opt/hpws/apache/sbin/httpd -k start -DSSL

If you have an apachectl script lingering somewhere look for the startssl case/esac block.