ZIP limitation?

[Willem Grooters]

Art,

I'm not certain this applies to your problem:
If you do:

1. $ zip "-Vwj" zipfile.zip $1$dga77:[dir.subdir1]*.*;*
2. $ zip "-Vwj" zipfile.zip $1$dga77:[dir.subdir2]*.*;*

and so on, so you're adding files beyond an existing one. This takes time since the EXISTING data needs to be copied first, I don't know how efficient zip will do this. Uisng this, each addition will take more and more time.
Since you omit the directory information (-j option) it might be that each addition might mean that checks need to be done - and this requires scanning thealready stored information. Again, the more files you have in the archive, the longer it would take.

If next is a directory of 45000 files, you can expect low performance for several reasons: the size of the directory itself will add to the already heavy processing.

Consider to make a separate zip of each directory, and zip these all into one.

1. $ zip "-Vwj" zipfile1.zip $1$dga77:[dir.subdir1]*.*;*
2. $ zip "-Vwj" zipfile2.zip $1$dga77:[dir.subdir2]*.*;*

...
lastL
$ zip "-V" allzips.zip zipfile*.zip

Willem

[Peter Barkas]

I have experience of the performance difference between the two approaches described by Willem.

I was zipping about 40-60 files totalling about 1.5gb or so. Doing them all at once was fine but incrementally adding each one separately was much much slower.

[Art Wiens]

Steven:

"[T]he OpenVMS Wizard cannot and does not
recommend storing large numbers of files
in a single (very large) directory."

I'll try and mention that to the long gone application programmers! ;-)

That's why we call them "legacy" systems.

Willem:

No that doesn't apply, I'm zipping individual directories to new zip archives.

Cheers,
Art

[Steven Schweda]

> I'll try and mention that [...]

Slap 'im around a little for me, too.

> That's why we call them "legacy" systems.

Some legacy.

I recall some suggestions in comp.os.vms a
while ago, involving dispersing the files
from the one over-full directory into
multiple (less over-full) directories, and
creating a search-list logical name which
points to the list of new directories.

New files are still a problem, as they keep
getting put into the first directory in the
list, but finding the old files could be
faster.

This assumes that the directory name is not
hard-coded into the "legacy", of course.

Thoroughly re-engineering an application
without actually changing it can be tricky.
Of course, if "unmaintained" means "never to
be looked at again", you may be happier just
suffering for a while with slow Zip.

I'm still pushing Zip 2.31, however, as it's
better, even though it won't solve your big
problem.

[Art Wiens]

>> I'll try and mention that [...]

> Slap 'im around a little for me, too.

If they were still around it might be a closed fist! :-O

>> That's why we call them "legacy" systems.

> Some legacy.

Perhaps "inheritance" is a better analogy, but grannie didn't like me too much!

Ah well, like a fellow I used to work with always said:

"It's all pensionable time."

I'll give v2.31 a whirl.

Cheers,
Art

[Willem Grooters]

Legacy/Inheritance, meaning "oldfashioned"?

On a VMS system, it will take time (obviously) to DIR a directory of 10.000 files.
On a Unix system, listing a directroy of that size will tell you the commandbuffer is full and show you nothing else.

What do you prefer?

Willem

[Steven Schweda]

If you're foolish enouh to say "ls *", it may
fail. Do you think that a plain "ls" will
have any trouble? Have you tried it, or are
you just guessing?

I prefer less inane comments.

[Art Wiens]

Ok, I guess it's time to close this thread ;-)

VMS is a fine system. The way that some things were implimented by my predecessors could have been done a bit "cleaner" ... other things are quite clever and flexible.

Management keeps referring to it as a "sunsetting platform" except the sun is having a hard time rising on *nix, so I continue to try and patch holes and keep the boat afloat.

There is no "best" system/application out there.

Cheers,
Art