Calculating block & file size of files in multiple savesets

[Kenneth Toler]

If all I have is the compressed savesets, is there a way to calculate the total uncompressed size for the total block & file size of multiple savesets?

I am trying to let the user know when they have selected too many savesets.

I need this info to compare to the free blocks on the hard drive.

If the total number of blocks required to expand the multiple savesets exceeds the total available blocks on the hard drive, then I will alert the user to de-select savesets until it is within the memory left on the hard drive.

[labadie_1]

>>> If all I have is the compressed savesets, is there a way to calculate the total uncompressed size for the total block & file size of multiple savesets?


No, as some data compress a lot and others not. You can't say "I have 100 M compressed, so I will always get 136,3 M of uncompressed data."

Today, disk space is cheap. May be the simplest solution is to have a huge free space.

[Hoff]

I appreciate what you are trying to do here -- pre-emptively detecting this case and issuing an error message -- but there's no reliable means to externally calculate the efficiency of compression.

You can certainly guess (eg: 2.0 to 2.5x is an oft-quoted ratio, or you could determine this ratio approximation empirically for your data) but there's no way I'm aware of to know what the input volume was, unless you have some control over the process and record this material yourself. This could be via a metadata file stored on the media in parallel with the compressed saveset, or via a metadata file inserted into the saveset, or via tagging the saveset with a site-specific ACE containing this data (this latter case assuming no use of /interchange).

Or you could wing it, restore the saveset, and catch the error when it arises. In most cases, I'd tend to code to catch the allocation error regardless, as -- barring a completely quiescent target disk -- there can be a parallel allocation that arises during the restoration, and derails your restoration process. Even with a careful and correct check, the restoration can still fail.

There is a semi-related discussion in the OpenVMS FAQ, around estimating available capacity of output media when generating a compressed saveset via BACKUP compression or drive-level compression http://64.223.189.234/node/1 This is the logical reverse of your question, that too is related to determining compression efficiency.

Stephen Hoffman
BoffmanLabs LLC

[labadie_1]

By the way, which tool do you use to compress ?

zip, gzip, bzip2, any other ?

[Kenneth Toler]

I used the backup command for OpenVMS 7.3-2

[Hoff]

BACKUP has an undocumented compression capability circa V8.3, but (unless that got backported and released and documented) you're probably not using it on V7.3-2; BACKUP itself does not compress data by default, and the aggregate saveset output of BACKUP is probably somewhere around 10% larger than the aggregate input given the default grouping and recovery data added into the saveset by BACKUP; the saveset is somewhat larger than the input data. (You can probably dial this ratio in yourself, depending on your particular BACKUP command.)

The tape drive can compress data and that data can include BACKUP saveets, and the same difficulty in estimating compression efficiency exists there.

The BACKUP compression command is BACKUP /DATA_FORMAT=COMPRESSED -- AFAIK, this is latent and undocumented, and not (yet?) supported. It's been discussed in various forums around the net.

There's a write-up on BACKUP, compression and encryption I/O throughput here: http://64.223.189.234/node/85
This topic doesn't cover the compression efficiency.

[Steven Schweda]

> By the way, which tool do you use to compress ?
> zip, gzip, bzip2, any other ?

> I used the backup command for OpenVMS 7.3-2

So, how is that _compressed_? Or, by
"compressed", did you just mean "collected"
(into a BACKUP save set)?

Not that it's likely to matter here, but note
that a Zip archive includes uncompressed and
compressed size data for the files it
contains, and zipinfo (unzip -Z) can reveal
such information without actually unpacking
the archive.

And the size of a(n uncompressed) BACKUP save
set is at least a fair guide to the size of
the data therein.

[Dean McGorrill]

>I used the backup command for OpenVMS 7.3-2

did you make a saveset of savesets? if so

$ back/list yoursaveset.bck/save

will give you the total blocks used.

[Kenneth Toler]

I am dealing with multiple independent savesets, not savesets within savesets.

I want to let the end user know when they have selected too many savesets.

[Kenneth Toler]

Doesn't the following give you the compressed blocksize?


"$ back/list yoursaveset.bck/save

will give you the total blocks used."

I am looking for the uncompressed block size.