Checking a URL from OpenVMS

[Harold Johnson_1]

Hi All. Does anyone know how I can check for the availability of a URL from VMS?
ie: using a .com file, check that a URL (such as http://login.yahoo.com/config/mail?.intl=ca) is available.

thnx

[Volker Halle · Accepted solution]

Harold,

welcome to the OpenVMS ITRC forum.

cURL has been ported to OpenVMS:

http://curl.haxx.se/download.html

You can use it to download WEB pages from an OpenVMS system from the DCL prompt, e.g.

$ curl http://www.openvms.org

Volker.

[Harold Johnson_1]

Thnx Volker. I downloaded the OpenVMS version of the EXE and tried running it with your example, but I keep getting the following error. Any ideas?


%DCL-W-MAXPARM, too many parameters - reenter command with fewer parameters
\HTTP\

[Steven Schweda]

You could also try Wget:

http://antinode.org/dec/sw/wget.html

but for the %DCL-W-MAXPARM problem, you
should read the FAQ:

http://h71000.www7.hp.com/faq/openvms_faq.html?jumpid=/go/openvms/faq
http://h71000.www7.hp.com/faq/vmsfaq_stmlf.txt

Look for:

8.2 How do I run a program with arguments?

[Hein van den Heuvel]

Exactly how did you activate it?
With the commance Volker showed?
How did you define 'curl' ?
Did you try using:
$MCR your-curl-location:curl ....

Or did you try having the curl location be part of your DCL$PATH search list logical?

Did you google for help?
Google: +vms +site:curl.haxx.se +"too many parameters"
This points to a seemingly relevant article:
http://curl.haxx.se/mail/archive-2002-10/0119.html

Btw.. you may also try 'lynx' as a simple, command line based, http reader.

Hein.

[Harold Johnson_1]

Yes, I had a brain fart and forgot about the $ in the equivilence name. doh

thnx, it works like a charm!

cheers

[Harold Johnson_1]

How about an address like the following:

http://ila.educ.gov.bc.ca/dev60cgi/rwcgi60?ila&report=DoNotDelete.rep&destype=cache&desformat=HTML

I'm attempting to run an oracle report and get the output back. In a regular windows browser, the output is displayed, but using CURL, I get an:

Error: The requested URL was not found, or cannot be served at this time

any ideas?

thnx

[Steven Schweda]

It's hard to test that URL from here.

Showing the actual command used is often
more helpful than a vague description of the
actual command used.

Did you quote the URL, as it contains DCL
special characters ("&")?

If cURL has any debug options, you might try
to persuade it to show you the URL it's
actually using (after "&" interpretation).

[Harold Johnson_1]

The url as shown is exactly the same URL as put into a browser. It looks like CURL cannot deal with this type of command. I'll keep playing with it.

cheers

[Steven Schweda]

You still haven't shown the actual command.

If the URL is on a DCL command line, did you
quote it?

[David B Sneddon]

I have tried the above URL in a browser and get
a timeout. I have also tried the above URL with cURL,
with it quoted and unquoted and get a timeout in both cases...

$ curl -v "http://ila.educ.gov.bc.ca/dev60cgi/rwcgi60?ila&report=DoNotDelete.rep&destype=cache&desformat=HTML"
$ curl -v http://ila.educ.gov.bc.ca/dev60cgi/rwcgi60?ila&report=DoNotDelete.rep&destype=cache&desformat=HTML


Dave

[David B Sneddon]

...and the output was

* About to connect() to ila.educ.gov.bc.ca port 80
* Trying 142.36.182.246... connection timed out
* couldn't connect to host
* Closing connection #0


Dave

[Steven Schweda]

> [...] and get a timeout. [...]

Well, of course. You didn't think that this
fellow's Oracle data base would be generally
accessible, did you? (He keeps his cURL
command a secret, so this should not be
amazing.)

I'm still betting on argument quotation (but
for case preservation, as the ampersands
appear to be harmless here, contrary to my
initial fear).

As usual, a straight answer to a simple
question might reduce the guesswork.

Note that Wget _will_ preserve command-line
case with SET PROCESS /PARSE_STYLE = EXTENDED
(assuming a sufficiently modern C RTL).

Judging from the results I got from a:

curl 7.15.1 (ALPHA-HP-VMS) libcurl/7.15.1 OpenSSL/0.9.6g

trying 'curl -V' and 'curl "-V"', I'd guess
that argument quotation is important with
cURL. Or (again trusting the C RTL), first
DEFINE DECC$ARGV_PARSE_STYLE ENABLE.

For details:
HELP CRTL Feature_Logical_Names

For details on how Wget does it
automatically, see:

HELP CRTL decc$feature*

and/or, look at its VMS-specific source code.

[Joseph Huber_1]

BTW, the original question was to check if an URL is available:

To avoid downloading the URL, just get the header with "pipe curl --head url >head.tmp".

Then one can use $STATUS to check if the server is reachable at all. If success, then one could parse the output for the HTTP status response like
("HTTP/1.1 200 Sending Processed HTMLX"). Code 200 is the HTTP success.

[Joseph Huber_1]

And, for Perl lovers :-)

at ftp://ftp.multicians.org/

there is (amongst others) url.pl, and urltest.pl, it works like this

perl URLTEST.PL http://www.multicians.org

http://www.multicians.org ok HTTP/1.1 200 OK

urltest could be modified to return VMS $status for use inside DCL procedures.

[Martin Vorlaender]

Sepp,

just a few notes...

>>>
To avoid downloading the URL, just get the header
<<<

BTW: Lynx also has a -head option.

>>>
And, for Perl lovers :-)
<<<

All perl programmers doing serious web programming install the libwwwperl (or LWP) which comes with some ready-made example programs including GET, HEAD, and POST.

cu,
Martin

[Joseph Huber_1]

Ah fine, yes i see LWP in my perl installation.

Just saw urltest immediately when looking for something containing URL :-)

And before I (with my minimal experience) write something using the LWP package I ready use urltest like this:

$ pipe perl perl_root:[utils]urltest.pl "url to test" |-
(read sys$pipe line ; line=f$element(0," ",line) ; -
define/job http_code &line)
$ if f$trnlnm("HTTP_CODE").eqs."ok" ...

[Joseph Huber_1]

I tried a similar one-liner with "lynx -head" and "curl --head", but these utilities apparently open stdout in a certain way, so that sys$pipe delivers one character per read:
not very convenient.

So without Perl (URL or LWP), one has to do it using lynx or curl by redirecting output to a temporary file, read the first line, check the second word of the line for the httpd response.

[Martin Vorlaender]

>>>
And before I (with my minimal experience) write something using the LWP package...
<<<

That's why I refered to the example programs - you don't need to do anything with them:

$ perl perl_root:[utils]lwp-request. -m HEAD http://www.openvms.de/
200 OK
Connection: close
Date: Thu, 01 Jun 2006 13:00:04 GMT
Accept-Ranges: bytes
ETag: "c4769c-f84-3fe30791"
Server: Apache/1.3.26 (OpenVMS) mod_jk/1.2.3-dev PHP/4.3.2 mod_perl/1.25 mod_ssl/2.8.10 OpenSSL/0.9.6g
Content-Length: 3972
Content-Type: text/html
Last-Modified: Fri, 19 Dec 2003 14:13:37 GMT
Client-Date: Thu, 01 Jun 2006 13:00:04 GMT
Client-Peer: 195.137.216.132:80
Client-Response-Num: 1
$

cu,
Martin

[Joseph Huber_1]

Thanks martin, so finally another one liner:

$ pipe perl perl_root:[utils]lwp-request. -m HEAD "testURL" | -
(read sys$pipe line ; line = f$element(0," ",line) ; define/job http_code &line)
$ if f$trnlnm("HTTP_CODE").eq.200 !success ...

[Harold Johnson_1]

Wow. Lots of comments, some of which are worth reading.

The command I use is:

$CURL "url..."

That's it. Yes, the db is not available to the public. Thnx for the help all, I'll try some of the suggestions out.

[Volker Halle]

Harold,

if you have received useful replies, please have a look at

http://forums1.itrc.hp.com/service/forums/helptips.do?#33

and consider to assign points to the answers you've received.

Thanks,

Volker.

[Harold Johnson_1]

I will continue to use cURL as this utility serves 90% of my needs.

thnx all

[Steven Schweda]

Does that mean that you _did_ get it to
work, or that you didn't?

I gather that cURL has a "-v" option which
might tell you more about what it's doing.
(_You_ should have such an option.)

I'd be interested to know if Wget did any
better (or differently).

[Harold Johnson_1]

The main reason for the thread was to find out if there was a utility to retrieve web pages from VMS. Using cURL, I can do this. I can't get it working with the sample that I provided, but at the least, it lets me know that the Oracle report server is available and running. The remaining 10% will be to see if I can get the sample actually running.

thnx