Operating System - OpenVMS
cancel
Showing results for 
Search instead for 
Did you mean: 

Is there an (UN)ZIP api?

 
Art Wiens
Respected Contributor

Is there an (UN)ZIP api?

I am not a programmer. Ok, now that we have that out of the way, I would like to recommend to our programmers to adopt the use of ZIP and UNZIP in their application development rather than inevitably ending up with production directories that are chock full of small files. To that end is there an "api" that they can use in their Basic programs to utilize ZIP and UNZIP? Or is it all up to DCL procedures? I have done a bit of reading but nothing jumped out at me.

Cheers,
Art
16 REPLIES 16
Steven Schweda
Honored Contributor

Re: Is there an (UN)ZIP api?

So far as I know, there's Windows DLL stuff,
but nothing other than (optimistic?) comments
about other environments. (See "api.c" in
both kits.)

Your problem description is a little too
fuzzy for me to say whether it would make
sense to explore this possibility further.
Art Wiens
Respected Contributor

Re: Is there an (UN)ZIP api?

For example there is a "billing" program which ends up generating ~2,000 invoices per month per division (seperate .txt files with PCL codes that get printed once and then sit on disk "for ever" ... they might want to see a couple in subsequent years if a deadbeat customer hasn't paid).

I occasionally run ZIP procedures and organize them into semi-annual files e.g. 2009_1.zip (Jan to Jun) and 2009_2.zip (Jul to Dec). If the Basic program(s) could add them into a ZIP file "automatically" as they are produced and perhaps the print routine could extract them from the archive, print them and then delete them. There are lots of other examples where lots of individual files get created per division, used maybe once and then occupy disk space thereafter.

Obviously it can get done with DCL, but if I could shift the work to them ... ;-)

Art
Hoff
Honored Contributor

Re: Is there an (UN)ZIP api?

If you're having this issue, then the zip tools probably aren't a good solution. Filling a directory with a plethora of scat files simply isn't good design practice. Maybe a stopgap, but there be dragons here.

One of the available interfaces here with a full callable API specifically intended for packaging files is the OpenVMS Librarian utility, and a full callable interface is available. Look in the OpenVMS Utility Routines Reference Manual for the LBR$ calls.

A more typical solution is some variation of a database, which (when it involves source code files) is usually some form of source code management tool. Git or Mercurial (Hg) are the common modern distributed version management solutions, Subversion (SVN) is an older approach, and the classic VMS solution is DECset CMS package.

The source code control systems are also often involving regression tests (looking for problems in the code before it goes into production) as well as prepackaged automated release paths for the builds.

Some (sneaky, clever) sites will also insert a pretty printer into the source code control check-in path, so the source code is always consistently formatted.

Based on your comments, there are probably project oversight issues lurking here too, and those will likely need to be resolved before this swamp can be drained.

I generally recommend find some insistent and persistent curmudgeon for, um, the continuous enforcement of proper code management conduct, and definitely look to get the source pools under control.
Hoff
Honored Contributor

Re: Is there an (UN)ZIP api?

Ah, you slipped a reply in there, and I see you're headed in a different direction with your question than I'd guessed.

OpenVMS does have callable file compression with DCX and (with recent releases) zlib. There are open source analogs, and versions of zlib and such are kicking around.
Steven Schweda
Honored Contributor

Re: Is there an (UN)ZIP api?

> Obviously it can get done with DCL, but if
> I could shift the work to them ... ;-)

LIB$SPAWN works from BASIC, doesn't it?
There might be some need to take care about
simultaneous access by multiple processes, of
course. I normally use Zip+UnZip in a batch
mode, to pack or unpack some kit with static
content, but Zip has a bunch of different
update/freshen/filesync/... modes, as well as
the simple add-an-archive-member scheme, and
UnZip should be able to manage selective
extraction.

> [...] Librarian [...] database [...]

Might also be useful. (Especially if the
files have UNIX-like attributes which won't
get wrecked by non-VMS-native software.)
John Gillings
Honored Contributor

Re: Is there an (UN)ZIP api?

Art,

Rather than change your existing code, adding a ZIP then UNZIP and print phase, I'd go for the simpler route and do regular post processing, in whatever language is appropriate.

Run a daily job that collects files which have been printed and ZIPs them (or insert them into a text library). As well as avoiding any changes to your existing application, you have the flexibility to change the mechanism you use - ZIP, Librarian, BZIP, database, offload to another system etc...
A crucible of informative mistakes
Art Wiens
Respected Contributor

Re: Is there an (UN)ZIP api?

"the OpenVMS Librarian utility"

Is there some practical limit how many "modules" can be in a text library?

"OpenVMS does have callable file compression with DCX"

Do you have any further pointers for that product? I can't seem to find anything on it.

Cheers,
Art
Art Wiens
Respected Contributor

Re: Is there an (UN)ZIP api?

Never mind on the DCX question ... found it (in the same manual - doh!).

Art
Hein van den Heuvel
Honored Contributor

Re: Is there an (UN)ZIP api?

>> "the OpenVMS Librarian utility"

Yeah, I thought about suggesting that also, but though better of it. Zip is relatively well understood and universally available.
Using that will be so much more transparent for the rest of the world.
You can just expose them on a Samba share and allow Windoze user to just use gui's to look at the file.
How would that work for you using VMS libraries?

I abhor repeated spawns, preferring an API, like most of us.
But hey, if it gets the job done easily and just called once per file and optionally optimized ( NOCLISYM? NOLOGNAM?...)
And compared to the price of thousands of files in a directory it is all good again.
Maybe optimize the temp files on a RAM drive?

Cheers,
Hein
Craig A Berry
Honored Contributor

Re: Is there an (UN)ZIP api?

No one seems to have mentioned the backup API, which now has the BCK_OPT_K_DATA_FORMAT_COMPRESS option. That sounds promising and is documented here:

http://www.openvms.compaq.com/doc/84final/4493/4493pro_001.html#4493_bu_chap

Another thing to keep in mind is I think you are looking for archiving as well as compression, and the two things don't necessarily come in one package. For example, I think zlib only compresses a data stream but doesn't know on its own how to assemble compressed files into an archive.

Hein van den Heuvel
Honored Contributor

Re: Is there an (UN)ZIP api?

I thought of, and should have mentioned the backup API.
I discared it because best I know savesets are static.
You can not add files after the fact.
With zip you can.

fwiw,
Hein
abrsvc
Respected Contributor

Re: Is there an (UN)ZIP api?

I am currently contracted with a client that has a similar problem with file creation. The solution here was to create daily backup savesets and store them on an archive server. These savesets are rarely accessed and reside on their own server system. The savesets are created directly over the network and the text files are deleted after 20 days in rotation. We have found that any "lookups" usually occur within the 20 day window. On the rare occassion that a file is needed, we restore it from the daily run.

We chose to run this via a self-submitting batch job rather than using the API through an application.

Dan
John McL
Trusted Contributor

Re: Is there an (UN)ZIP api?

John Gillings and abrsvc have practical ideas but a couple of other ideas that might be helpful even if they do involve more effort.

Have you seen zlib (see http://www.zlib.net/)? We run it here on VMS to create and write compressed files.

Maybe think about separating the creation of the printer output from the data preparation. The main processing would sort out what's to appear on the invoice and write it to a growing indexed file, but another function or image would be responsible for reading that data and formatting it. You'd probably end up storing less data and have the flexibility of probably making changes in one place if the output document had to be modified.
Jur van der Burg
Respected Contributor

Re: Is there an (UN)ZIP api?

In newer vms versions (V8.3 or later) there's compress$share with entries ZLIB$UNCOMPRESS and ZLIB$COMPRESS2 wich is used by backup. An application could use that.

Fwiw,

Jur.
Peter Barkas
Regular Advisor

Re: Is there an (UN)ZIP api?

FWIW, my experience of repeatedly adding files to existing zip files is that performance really starts to suffer with large zip files.

Unless something has changed since then, the new file is not 'tacked on the end', the whole zip file is read and rewritten with the new file inserted.
Steven Schweda
Honored Contributor

Re: Is there an (UN)ZIP api?

> Unless something has changed since then,
> the new file is not 'tacked on the end',
> the whole zip file is read and rewritten
> with the new file inserted.

Zip is somewhat averse to damaging an
existing archive while building its
replacement. Tacking stuff onto the end of
an existing archive could lead to damage, if
something goes wrong while tacking. Other
(riskier) schemes may be possible, but are
not currently on the Zip to-do list.