DCL..FILE QUESTION

[Anthony Pratico]

Is there a cleaner way to get an INDEX files record count, I do not see anything IN LEXICAL F$FILE, and the only way I can see doing this is by SEAR/STAT of the file, which is really clugey at best.

[Steve-Thompson]

Hi Anthony

$anal/rms/fdl/out=a.a

Then edit a.a and you get a record count in "analysis of key_0" ...

Any good?

Steven

[Anthony Pratico]

Thanks Steve, I was hoping for some type of LEXICAL function that I have not found yet. I was planning on looking at the .anl file to get what I need but still this is a bit clugey as well...Thanks for the input!

[Hein van den Heuvel]

RMS only maintains a FILE LENGTH HINT for simple (sequential) files on ODS 5 disks.
See F$FILE FILE_LENGTH_HINT : "String Record count and data byte count in the form (n,m), where n is the record count and m is the data byte count. An invalidated..."

So one pretty much has to count!

ANAL/RMS/FDL requires exclusive file access which is often not acceptable.

SEARCH/STAT is not a bad tool.
CONVERT/SHARE is much similar

Best tools are DIX (freeware) and SELECT (EGH, commercial).

My RMS_TUNE_CHECK freeware tool will count and set symbols.

I'll include a count tool I am still working on. Hope the attached version is runnable code... haven't touched it in weeks and I do not recall exactly. It got a little out of hand... function creep... I may have lost track of the original simple goal :-). Also, I got 'stuck' with a minor RMS bug which the tool ran into.

What problem are you really trying to solve though? Why do you need the count, and how precise?

[send Email if you need more details]

Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[Hein van den Heuvel]

Just curious...

which VMS version

how often would je need this?

how big a file? thousands or millions of records?

Actively shared?

Global buffers?

btw... you know that SEAR/STAT also sets up a dcl symbols with counts right?

Hein.

[Anthony Pratico]

I am trying to basically calculate how much space a particular index compressed file will need when it is uncompressed. I have the calcualtion of how to do it but I need the record count to complete the formula.

record size*# of records/512 = the size of uncompressed file

I am talking some where in the 100's to million's of records per file and hundreds of files to caclulate.

I am running VMS 7.3-2

[Robert Gezelter]

Anthony,

Unless you are stripping the indices, the formula will not give an accurate result. Depending on the definition of the file, you will need to allow for compression, and for indices.

If the file is indexed, you will also need to allow for the effect of partial bucket fill.

- Bob Gezelter, http://www.rlgsc.com

[Hoff]

Calculations with compression are pretty ugly; um, are highly dependent on the particular compressability of the particular data involved.

I'd probably either guess (predict) the record count, or count the records directly. This then can allow you to ballpark the uncompressed size, and the compressed size is probably smaller.

Alternatively, you could use knowledge gained from existing files and/or existing counts and/or test runs to predict the resulting sizes; to use empirical means to size the files.

Disk storage itself is comparatively cheap these days, and some of what was once important in this area has become somewhat less critical. (Adding more or bigger disks can make for different problems, but I digress.) Particularly if you use staged storage, with fast and expensive disk where you need it, and slow and cheap disk elsewhere.

Stephen Hoffman
HoffmanLabs.com

[Hein van den Heuvel]

>> I am trying to basically calculate how much space a particular index compressed file will need when it is uncompressed.

Ah, I'm glad I asked then, so an exact count is not needed and is only a component to the total.

Well, strictly speaking you do NOT need the record count for that, but the LEVEL1 index record count, which can be derived from that.

Please get out my earlier suggestions Fekko's DIX and my Tune_Check. They can count records and buckets more quickly than ANAL/RMS can.

EDIT/FDL/INTERACTIV will happily predict.
And my tunecheck tool would product the exact answer to the question. Sample output:

! Total_Records=50654766,rrv=540926,del=1,lrl=159,user_size=159,data_size=107
! fill=97% in 10 ranges 0-100 :0:0:0:0:0:2:0:0:0:96
! hit=1066860,miss=141828,blocks_read=13606600,series>5=4783,max=108214,avg=242
For example
! LVL/BKS/DAT/IDX: 5/1/12663692/395743, 4/2/12663692/191880, 3/5/12060660/71795, 2/22/11730598/15686
! Compressed: 4/1/12663692/156344, 3/3/12663693/50256, 2/12/11918772/11688

>> I have the calcualtion of how to do it but I need the record count to complete the formula.

But I ask you again.
What problem are you really really really trying to solve?

Performance? IO? Locking? Contention? Space?

IMHO RMS INDEX compression has NOTHING to do with SPACE, everything with CONTENTION, Binary search versus sequential scan, bucket sizes and buffer usage. The space is the least of your concerns... NORMALLY.
Space would only be relevant if you had to burn it to a CD or such.

>>> record size*# of records/512 = the size of uncompressed file

Not good enough my friend.
There is KEY compression, INDEX compression and DATA compression, and records must fir entirely in buckets. Buckets have overhead, records have overhead.

>>> I am talking some where in the 100's to million's of records per file and hundreds of files to caclulate.

For autotation purposes the best thing you can do is to write a program/script which just grabs 1,000 - 10,000 record from a file, then loads them in a sample indexed files with all compression enabled and analyze that for the final effect.

>> I am running VMS 7.3-2

Much the same for all versions.

Are you coming to the OpenVMS Bootcamp?
I'll be talking about this and many more details. I'l be bringing the latests versions of my excel spreadsheet to do this very work and the other tools to help.

Regards,

Hope this helps some more,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting