Zip/Unzip Data File Size Limitations?

[John T. Farmer]

Just put InfoZips two utilities, Zip 2.3 and Unzip 5.50 on our Alpha/OpenVMS 7.2-1 system. I can zip and unzip small test files. Then I tried a 4.2 million block file and Zip does see the file. A small test file of the same name is zipped and unzip with ease.

Is there a file size limitation on what Zip & Unzip can handle? Is there a newer version that fixes this issue? This 4mil block file is text and should compress down significantly for ftp purposes.

Thanks,

John Farmer
john dot farmer at genworth dot com

[jaw_1]

Yes, but do note that it is the beta version, and also requires a C compiler to build. Take a look at:

ftp://ftp.info-zip.org/pub/infozip/OLD/beta/

James

[Steven Schweda · Accepted solution]

The file size limit is imposed by the C
run-time library, and it's 2GB (except on
some old VAX versions, where it's 4GB). At
4.2M blocks, you seem to be just over the
edge.

Not that it'd help with large files, but the
current released versions are UnZip 5.52 and
Zip 2.32

http://www.info-zip.org/
http://www.info-zip.org/UnZip.html
http://www.info-zip.org/Zip.html

There are some significant benefits to using
the newer versions, compared with UnZip 5.50
and Zip 2.3. (File I/O speed, "-V" archive
portability, less pointless Zip archive file
copying, and so on.)

> ftp://ftp.info-zip.org/pub/infozip/OLD/beta/

Yes, and at VMS V7.2-1, you may just have a
C RTL new enough for large-file support.

If you need binaries because you lack a C
compiler (and you're too lazy to use one of
the TestDrive Alpha systems), let me know,
and I can probably whip up some useful object
files.

[John T. Farmer]

Sorry, typo "and Zip doesn't see the file". But you folks already figured that out.

So what I'm hearing, is the size limitation is due to the C RTL. We don't have C installed on our system, so I'm at the mercy of someone creating that for me. I'd be happy to test it if someone can build it.

"(and you're too lazy to use one of
the TestDrive Alpha systems)"
What is a TestDrive system?

Thanks,

John

[Robert Gezelter]

John,

The "Test Drive" systems are a couple of systems operated by HP that have fairly public access (obviously within some limits). The pointer to the OpenVMS Test Drive is http://www.testdrive.hp.com/os/#ovms

- Bob Gezelter, http://www.rlgsc.com

[Steven Schweda]

> What is a TestDrive system?

http://www.testdrive.hp.com/
http://www.testdrive.hp.com/current.shtml

The price is right, but you'll need to look
elsewhere if you want to test drive a VAX.

> We don't have C [...]

Try:

http://antinode.org/ftp/info-zip/unzip60c_alphal.zip
http://antinode.org/ftp/info-zip/zip30e_alphal.zip
or:
ftp://antinode.org/info-zip/unzip60c_alphal.zip
ftp://antinode.org/info-zip/zip30e_alphal.zip

And try to ignore everything else in that
neighborhood, if you know what's good for
you.

These kits were built with "HP C V7.1-015 on
OpenVMS Alpha V7.3-2", so you'll probably
need to re-link. (For details, see
[.VMS]INSTALL_VMS.TXT.) The source is not
exactly the same as the original BETA kits,
but it shouldn't be much worse. There may be
some extra crud in there, too, as those were
my development directories. The warranty on
this stuff covers even less that the normal
it's-all-your-fault disclaimer would imply.

If a re-link fails, you may need to find
someone with a comparably old VMS version to
build the stuff. (Or perhaps install a new
C compiler kit, license or no, and/or a C RTL
patch to get the required stuff, if any.
It's been a while since I've had to worry
about linking on old VMS versions, so I'm
not sure how much can go wrong.) Wake me
when it gets worse.

[Steven Schweda]

Someone needs better (more VMS-tolerant) FTP
(proxy?) software:

29-SEP-2006 08:38:30.39 User:anonymous ident:proxy@ status:07649912 CWD dir:SYS$POSIX_ROOT^:^[info-zip^]zip30e_alphal.zip

alp $ sysmsg %x07649912
(SYS$COMMON:[SYSMSG]TCPIP$MSG.EXE;1:)
%TCPIP-E-FTP_BADDIR, invalid directory

"zip30e_alphal.zip" is not part of the
directory name. (My best guess: The request
was "CWD /[info-zip]zip30e_alphal.zip", which
didn't have a chance of being right.

Ask me why I map my anonymous FTP directory
to "/ftp/" on the Web server. Go ahead, ask.

On the bright side, having an FTP server with
which lameware can't cope does foil many
break-in attempts and abuse exploits (as
well as some legitimate users).

[Wim Van den Wyngaert]

We encountered the same problem with Sybase dumps. We now split the dump into 5 pieces by doing the dump to 5 devices.
There might be a utility to split a file into smaller pieces. Or you can write it yourself in any language.

Wim

[John T. Farmer]

Do I understand correctly, that the 2gb input file limit (C RTL) still exists even on newer versions of VMS, i.e. 8.x? Not that I'll be upgrading soon, but just wonder ed. I'm dealing with some pretty large files for data warehouse conversion.

thanks,

John

[Steven Schweda]

> Do I understand correctly, that the 2gb
> input file limit (C RTL) still exists even
> on newer versions of VMS, i.e. 8.x?

Large-file (>2GB) support in the C RTL seems
to be available beginning with VMS Alpha
V7.2 or so. After that, it's available in
everything except VAX (any version).

To make use of this large-file support, an
application (say, for example, Zip or UnZip)
may need to be re-coded to use larger
variables for file sizes and byte offsets,
and it must be re-built requesting large-file
support (defining the C macro "_LARGEFILE").
For some of the fine print, see
SYS$COMMON:[DECC$LIB.REFERENCE.DECC$RTLDEF]DECC$TYPES.H

In the case of Zip and UnZip, some new
archive formats are needed to accomodate
large file size data, so there were more
changes needed than just replacing "%d" with
"%lld" in a few places.

Zip 2.x and UnZip 5.x do not and will not
offer large-file support, anywhere, anytime.

Zip 3.x and UnZip 6.x will offer large-file
support. If you can accept the current,
unreleased, BETA versions (Zip 3.0e, UnZip
6.00c), you can have it now (with no known
major problems).

In fact, on VMS, for maximal downward
compatibility, the current builders for Zip
3.x and UnZip 6.x do not enable large-file
support by default. You need to specify
"LARGE" (somehow) at build time to get it.