Shell Script for checking webserver pages are working or not

[Shivkumar]

James, I appreciate your help.

As always best regards to you.
Shiv

[James R. Ferguson]

Hi Shiv:

I inadvertently dropped the shell shebang line --- the first line of a script that specifies the program (here, the shell) to use:

# cat wget.sh
#!/usr/bin/sh
typeset LOG=wget.log
typeset LST=wget.list
rm -f ${LOG}
wget --spider -a ${LOG} -i ${LST}
awk '/^Resolving/ {if (/failed/) {print $2,"bad"} else {print $2,"ok"}}' ${LOG}
exit 0

Regards!

...JRF...

[Shivkumar]

Thanks Yang and others!!

Regards,
Shiv

[rmueller58]

Your best bet would be to use LYNX rather then WGET as if you have robots.txt to disallow robots, WGET will be rejected but Lynx will not..

Use #lynx -mime_header

to get your 200 status code.

[Steven Schweda]

> [...] if you have robots.txt to disallow
> robots, WGET will be rejected [...]

Of course, that depends on what's in the
server's "robots.txt" file. Also, as
"wget -h" will tell you, the user may decide
how wget identifies itself (here, from "GNU
Wget 1.10.2c" on VMS):

-U, --user-agent=AGENT identify as AGENT instead of Wget/VERSION.

On the other hand, I'm sure that wget causes
cancer, so it's probably best to avoid it for
that reason, too.

[Shivkumar]

Someone told me to use openssl. Is it possible to use openssl ?

Thanks,
Shiv

[Steven Schweda]

> Someone told me to use openssl. Is it
> possible to use openssl ?

It probably is possible, but shouldn't you
be asking "How?" of the one who told you to
use it?

Wget can incorporate OpenSSL, which it uses
for a URL which begins "https://". All you
said in your original posting was "the page",
so everyone has been assuming that you meant
something simple, like "http://xxx...". If
you say more than "the page", you might get
better responses.

It's possible to do the job with Telnet, too,
but it's not simple, and if you are "not good
in shell scripting", then it's probably not
the easiest or best way to attack the
problem.

And besides wget, there's also "curl":

http://curl.haxx.se/

Either one should be able do the job
(depending on exactly what "the job" is, of
course).

[James R. Ferguson]

Hi Shiv:

I might add that if you drop the requirement for a *shell* script, the number of different solutions expand further. Depending upon your requirements, *Perl* scripts with the 'LWP::Simple' module would work, too.

Regards!

...JRF...

[Ralph Grothe]

If you have got the openssl suite installed on your box (HP offers a free bundle) then you could use the incredibly versatile openssl command for most things crypto.
e.g.
You can pull certificate info of an SSL/TLS enabled webserver by issuing something like
(I assume you have the webserver running on the same box, but any other reachable IP also will work)

$ openssl s_client -connect 127.0.0.1:443

However, this may block if the DocumentRoot is AuthType "protected" because the webserver is sending an Authentication request which your client has to answer.

If you have a recent Perl you could pull the header of any https website.
If it is AuthType protected you could use the
-C switch like below.
The HTTP response header received also contains entries about the SSL-Cert-Issuer (which I omitted below).
Most important the HTTP response status is sent in the first line (usually you would expect a 200 which is OK)

$ lwp-request -m head -C grothe:riddle https://127.0.0.1/|head -1
200 OK


You could also get the nagios-plugins, extract the tarball and only compile the check_tcp and check_http plugins.
They are specifically designed for monitoring purposes and offer a whole lot of options to virtually test any HTTP services.
When invoked with -h or --help you get a full screen where the options are explained.

Unfortunately the nagios server where I've got those check commands available is a legacy AIX system that neither has a /dev/urandom nor an entropy gathering daemon why I only get this nasty error

$ check_http -H somehost -N -S -a grothe:riddle
CRITICAL - Cannot make SSL connection
35196:error:24064064:random number generator:SSLEAY_RAND_BYTES:PRNG not seeded:md_rand.c:474:You
need to read the OpenSSL FAQ, http://www.openssl.org/support/faq.html
Error on receive

But HP-UX provides patches that would create those random devices to make openssl or other SSL libs happy.