curl http download

[Richard Jenkins_1]

Hi

I'm trying to use Internet Express curl to do an http download of all the files in a remote directory to the same base filename in the local directory. If it were ftp I would simply do an 'mget *'.

Help please!

Richard.

[paolo barila]

Hi,
did you try this from curl manual:

URL
The URL syntax is protocol dependent. You'll find a detailed
description in RFC 3986.

You can specify multiple URLs or parts of URLs by writing part sets
within braces as in:

http://site.{one,two,three}.com

or you can get sequences of alphanumeric series by using [] as in:

ftp://ftp.numericals.com/file[1-100].txt
ftp://ftp.numericals.com/file[001-100].txt (with leading zeros)
ftp://ftp.letters.com/file[a-z].txt

No nesting of the sequences is supported at the moment, but you can
use several ones next to each other:

http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html

You can specify any amount of URLs on the command line. They will be
fetched in a sequential manner in the specified order.

Since curl 7.15.1 you can also specify step counter for the ranges, so
that you can get every Nth number or letter:
http://www.numericals.com/file[1-100:10].txt
http://www.letters.com/file[a-z:2].txt

[Ivan Krastev]

To use same name use "-O" option is curl.
For example :

curl -O http://server.com


regards,
ivan

[Steven Schweda]

> [...] If it were ftp [...]

FTP includes the ability to get a directory
listing from the FTP server. HTTP, in
general, does not. If you don't know what
the remote file names are, and if the HTTP
server won't tell you, then you're probably
doomed.

Now, if there's an HTML document which
includes links to all the files of interest,
then a program like wget can be told to suck
down that document, and follow the links
therein, and that, perhaps with some other
selection options, might do what you want.

http://www.gnu.org/software/wget/wget.html

(Look for "recursive" in the documentation.)