topic Re: ZIP Problem in Operating System - OpenVMS

ZIP Problem

Dieter Rossbach — Tue, 03 Feb 2004 08:37:26 GMT

I have a problem with ZIP 2.3 on my VMS/ALPHA system. I try do compress savesets and not all of them are processed:

$ dir *.bck

19_B5_V_040201.BCK;1 5084730 1-FEB-2004 19:08:04.36
19_B5_V_040201.ZIP;1 1 3-FEB-2004 13:07:22.04
21_A5_V_040131.BCK;1 9608067 31-JAN-2004 19:07:57.04
21_A5_V_040131.ZIP;1 333616 3-FEB-2004 13:21:56.57

file 21_A5-V_040131.BCK went into 21_A5_V_040131.ZIP without any problems, trying the same with 19_B5_V_040201.BCK;1 resulted in:

$ zip "-V" 19_A5_V_040131.zip 19_A5_V_040131.BCK
adding: 19_A5_V_040131.BCK (stored 0%)

The backup file is OK, I can do BACKUP/LIST and /RESTORE from that file and it was NOT accessed by any other process when I tried the ZIP.

My zip version:

Zip 2.3 (November 29th 1999).

Regards

Dieter

Re: ZIP Problem

Lokesh_2 — Tue, 03 Feb 2004 08:49:54 GMT

Hi,

from the output you posted, I guess the file is 19_B5* and not 19_A5*....

sorry if I am wrong here,
Best regards,
Lokesh

Re: ZIP Problem

John Eerenberg — Tue, 03 Feb 2004 08:51:54 GMT

Hi Dieter,

I too have had similar results using zip on large files.
Two things to check for.
1) Is the NoBackup flag on the savesets? A Dire/Full on the filename will show this.
2) Zip creates (at least the zip I use and we look to be the same) a temp file on SYS$Scratch. Since SYS$Scratch is usually the same as SYS$Login, then you must have at least that much freespace or roughly 10,000,000 blocks for your largest file.

john

Re: ZIP Problem

Dieter Rossbach — Tue, 03 Feb 2004 09:05:32 GMT

1. NoBackup:

Both Savesets are created with the same procedure, all flags are the same

2. Size of temporary file:

the larger file can be zipped, the smaller not ... it can't be a problem of sys$scratch (and there is a lot of space)

Dieter

Re: ZIP Problem

Wim Van den Wyngaert — Tue, 03 Feb 2004 09:49:47 GMT

Did some tests and found that "stored 0%" is shown when zipping small files (less than 6 bytes). Is the protection OK ? Did you try it with all privs enabled ? Did you do an anal/rms to check for file corruption ?

I also got the message while it zipped the file. Check if the file was put in the archive.

Re: ZIP Problem

Martin P.J. Zinser — Tue, 03 Feb 2004 10:23:31 GMT

Hello Dieter,

a number of issues compressing large files with VMS file attributes have been solved in the latest Zip beta. Please send me mail at
zinser@zinser.no-ip.info and I shall give you instructions where to download it.

Greetings, Martin

P.S. I do use this particular beta already for a while in a production environment at work without encountering problems ;-)

Re: ZIP Problem

David M Smith_1 — Tue, 03 Feb 2004 13:03:55 GMT

Dieter, I believe this is an inherent implementation limit in ZIP V2.3. In the distribution I downloaded, there is a file in the [.PROGINFO] directory called ZIPLIMIT.TXT which lists a number of limitations on the sizes of ZIP archive entries and the like. If you check this file, I believe you will see that the maximum size of a ZIP archive is 2 GB, and your very large file must exceed that limit. I've seen similar "mysterious" behavior when someone in our company tried to ZIP a very large (in our case, about 6000000 blocks) file.

It would be nice if ZIP gave some indication of what is going on, but I suspect that this is the problem.

Re: ZIP Problem

David B Sneddon — Tue, 03 Feb 2004 19:59:34 GMT

Dieter

I have come across other issues with ZIP (none
of them major) but the following will show one
such issue:

Here I zip the same file but get different results depending on the filename...

$ copy [-]login.com []a.a
$ zip a.zip a.a
adding: A.A (deflated 60%)
$ del a.zip;
$ rename a.a a.arc
%I, DBS0:[SCRATCH]A.A;7 renamed to DBS0:[SCRATCH]A.ARC;1
$ zip a.zip a.arc
adding: A.ARC (stored 0%)

It looks as though ZIP is making some assumptions based on the filename and may also be making assumptions based on the contents of the first block or so of the file (I think this is how the *nix "magic" stuff works). There may be some string in the first block that is causing it to not be compressed.

Regards
Dave

Re: ZIP Problem

Martin P.J. Zinser — Tue, 03 Feb 2004 21:28:15 GMT

Hello David,

the "do not compress" rule is defined by the user actually. Have a look at the logical zipopt . Most probably it does contain something like "-n .arc:...". If you think about it a moment this makes sense. Trying to compress an already compressed file will at best gain you very little (while costing CPU) while at worst actually can inflate the stored file (and still cost CPU ;-). AFAIK no "first block" magic is performed in Zip for VMS.

Greetings, Martin

Re: ZIP Problem

David B Sneddon — Tue, 03 Feb 2004 21:50:31 GMT

Hi Martin,

In my example the symbol ZIP == "$dbsexe:zip ""-V""".
No zip logicals are defined.

Dave

Re: ZIP Problem

David B Sneddon — Tue, 03 Feb 2004 22:04:58 GMT

I just checked the source code and in ZIP.C there is a list of special suffixes:
.Z, .zip, .zoo, .arc, .lzh, .arj

Dave

Re: ZIP Problem

Wim Van den Wyngaert — Wed, 04 Feb 2004 05:15:19 GMT

The stored 0% is a hardcoded text in the program. It means that no compression has been done. It seems to try the compression and if no result, simply stores the file.
Some extentions are directly stored without discussion.

Btw : I did some tests and found that /level=1 compresses Sybase dumps almost as good as /level=6 (the default) but cpu consumption is less than half.

Re: ZIP Problem

Dieter Rossbach — Wed, 04 Feb 2004 09:48:14 GMT

@everybody: thank you.

Here are some more tests, influenced by your replies:

$! (1)

$ zip "-V" 12_A0_V_040130.zip 12_A0_V_040130.BCK;1
adding: 12_A0_V_040130.BCK (stored 0%)
$ ! elapsed time unter a second !!!
$ !

$ unzip -l 12_A0_V_040130.zip
Archive: DISK$SICHERUNG:[000000]12_A0_V_040130.ZIP;1
Length Date Time Name
------ ---- ---- ----
0 01-30-04 21:05 12_a0_v_040130.bck
------ -------
0 1 file

---------------------------

$! (2)
$!
$ zip 21_A5_V_040131.zip 21_A5_V_040131.BCK
adding: 21_A5_V_040131.BCK (deflated 73%)
$!
$! elapsed time about 42 min
$!

$ unzip -l 21_A5_V_040131.ZIP;1
Archive: DISK$SICHERUNG:[000000]21_A5_V_040131.ZIP;1
Length Date Time Name
------ ---- ---- ----
624363008 01-31-04 19:07 21_a5_v_040131.bck
------ -------
624363008 1 file

$ unzip 21_A5_V_040131.ZIP;1
Archive: 21_A5_V_040131.ZIP;1
replace 21_a5_v_040131.bck? [y]es, [n]o, [A]ll, [N]one, [r]ename: r
new name: x.bck
inflating: X.BCK;

$ dir/size=all x.bck,21_A5_V_040131.BCK;1

Directory DISK$SICHERUNG:[000000]

X.BCK;1 9608067/9608112 31-JAN-2004 19:07:57.00
21_A5_V_040131.BCK;1 9608067/9608112 31-JAN-2004 19:07:57.04

$ Backup/compare x.bck 21_A5_V_040131.BCK;1
$! no errors ...

---------------------------

@ WIM:
> Did some tests and found that "stored 0%" is shown when zipping small files
> (less than 6 bytes).

sure, this is correct ...

> Is the protection OK ?

yes

> Did you try it with all privs enabled ?

yes

> Did you do an anal/rms to check for file corruption ?

no, but I restored that .BCK-file without any error -> the source file is OK

> I also got the message while it zipped the file. Check if the file was put in the archive.

see (1) it is put in the archive with 0 Bytes

---------------------
@David M Smith

> If you check this file, I believe you will see that the maximum size of
> a ZIP archive is 2 GB, and your very large file must exceed that limit.

That sounds like a good explanation in the first place, but as I am able to successfully zip larger files (see ex. (2) ) there must be another reason.

----------------------
@David Sneddon

> It looks as though ZIP is making some assumptions based on the filename and may also be
> making assumptions based on the contents of the first block or so of the file (I think
> this is how the *nix "magic" stuff works). There may be some string in the first block
> that is causing it to not be compressed.

Maybe .. but I trp to compress backup savesets, the first block of that file should have a fairly fixed format ...

----

It looks very much like zip finds a problem and writes a wrong message ...

Dieter

Re: ZIP Problem

Dieter Rossbach — Thu, 05 Feb 2004 01:52:24 GMT

zip 2.4h solved the problem ...

Thank you

Dieter

Re: ZIP Problem

David M Smith_1 — Mon, 09 Feb 2004 14:21:14 GMT

You said: V2.4h solved your problem -- where did you find V2.4h? I have looked on the Info-ZIP site and I don't see any reference to that.

Thanks.

Re: ZIP Problem

David M Smith_1 — Mon, 09 Feb 2004 14:25:13 GMT

In an earlier post, David Sneddon wrote:

>I just checked the source code and in >ZIP.C there is a list of special suffixes:
>.Z, .zip, .zoo, .arc, .lzh, .arj

This was mysterious to me, too, until I found it documented in the help text (I use the CLI version) under the following:

/STORETYPES=(.ext1,.ext2,... )

For files with the specified extensions, Zip does not try to compress the data but stores them verbatim. This speeds up operation on files that have already been compressed and where a second compression step usually does not gain much space. The default list of extensions where compression is suppressed is (.Z,.zip,.zoo,.arc,.arj).

I ran into this when ZIPping a VMSINSTAL kit which happened to have a .Z saveset!

Re: ZIP Problem

Ian Miller. — Tue, 10 Feb 2004 04:09:16 GMT

to get zip 2.4h see earlier posting from Martin.