A question about secondary keys in RMS indexed files

[Jeremy Begg]

Hi,

I'm working on a procedure which uses the CONVERT utility to cleanup some RMS indexed files on a regular basis.

Some of the files are quite small so I probably won't bother with them, but I thought I would do a trial run anyway.

I have Hein's RMS_TUNE_CHECK utility which is very handy for quanitifying the CONVERT benefits, but the output from one of my small files has me slightly confused.

Here's what it tells me about the file before it is CONVERTed:

$ sidr -t prod_data:iasf001.dat
- RRVS: 128 moved records in 308 record sample. 41.0%. Convert! Smaller fill?

Key 1

Found 314 sidr records in 1 buckets
There were 308 user data record pointers.
There were 17 deleted, 0 RU_delete and 0 RU_update pointers.

Top Ten Table of Sidrs with more than 1 duplicate

Duplicate count, Buckets, Key value
--------------------------------------------------------
2 1 25
2 1 40
2 1 61
2 1 68
2 1 76
2 1 117
2 1 139
2 1 149
2 1 155
2 1 157
$

OK, nothing strange about that. I then CONVERT the file and have another look:

$ convert/share prod_dat:iasf001.dat sys$login: /stat

CONVERT Statistics
Number of Files Processed: 1
Total Records Processed: 308 Buffered I/O Count: 35
Total Exception Records: 0 Direct I/O Count: 140
Total Valid Records: 308 Page Faults: 351
Elapsed Time: 0 00:00:00.10 CPU Time: 0 00:00:00.07
$ sidr -t iasf001.dat

Key 1

Found 308 sidr records in 2 buckets
There were 308 user data record pointers.
There were 0 deleted, 0 RU_delete and 0 RU_update pointers.
No duplicate key values found.

What puzzles me is the CONVERTed file appears to have lost a lot of duplicate keys. Or is it the case that those duplicate keys would have been pointing to deleted records?

Thanks,
Jeremy Begg

[P Muralidhar Kini]

Hi Jeremy,

In case you have not already read this, you may find this helpful -
http://h71000.www7.hp.com/openvms/journal/v2/articles/rms.html

This is a well written technical journal article on
"RMS Performance: Duplicate key chains" by Hein.

Hope this helps.

Regards,
Murali

[Hein van den Heuvel · Accepted solution]

Hi Jeremy,

First of all, like you said, there is little or no point to worry about converting/tuning a file like you describe.

You are correct in questioning the output for those simple cases to be able to deal with more complex/problematic cases should they arise.

Anyway, there are two effects that may play a role here.

1) When record with duplicate alternate key is deleted, or the alternate key is change and thus effectively deleted, RMS leaves a stub flag byte behind to help concurrent streams retain their position for '$GET NEXT' operations.
These are the "17 deleted" tune check referred to in your example.

2) RMS can do 'fast deletes' on indexed files. In that case, RMS only deletes the main record during a $DELETE and does not bother to try deleted the corresponding alternate key pointers to the deleted records. When a subsequent access is done using such alternate key, then it will find the record deleted upon de-reference and at that time the alternate key will be deleted while the structures to get there are already/still in the buffer cache. This is a great optimization.

Hope this helps,
Hein van den Heuvel

[Jeremy Begg]

Murali, thanks for the TJ reference. I probably read that article when it came out but it doesn't hurt to be reminded that they are still a good reference!

Hein, thanks for the detailed reply (which confirms what I suspected what is going on).

I was about to ask "how do I enable 'fast delete' but then I remembered I have the RMS reference manual close at hand, and found this listed as the RAB$V_FDL option in the RAB$L_ROP field. Is there a way to have this occur automatically for all programs which might access the file (i.e. without having to modify all calls to $DELETE or their high-level language equivalents)?

Thanks,
Jeremy Begg

[P Muralidhar Kini]

Hi Jeremy,

>> how do I enable 'fast delete'
>> RAB$V_FDL option in the RAB$L_ROP field
Yes thats right. The Fast delete performance option of RMS is controlled by the
RAB$V_FDL option in the RAB's RAB$L_ROP field.

>> Is there a way to have this occur automatically for all programs which might
>> access the file
By default, the fast delete option would be off. You need to enable it using the
RAB$V_FDL option. I dont think there is any other "automatic" way to trigger the
fast delete option to get enabled automatically without using the RAB$V_FDL option.
(i.e. you have to modify the program using the $DELETE to get the fast delete feature)

Regards,
Murali

[Jeremy Begg]

I noticed that File Definition Language has a "CONNECT" clause allowing the file designer to specify various run-time options.

I presume the only way to use that information in a program would be for it to call FDL$PARSE() as part of the process of opening the file.

[P Muralidhar Kini]

Hi Jeremy,

>> I noticed that File Definition Language has a "CONNECT" clause allowing
>> the file designer to specify various run-time options.
Yes, thats correct. Using the CONNECT attribute you can specify run time
options to your program.

>>I presume the only way to use that information in a program would be for it to
>>call FDL$PARSE() as part of the process of opening the file.
Yes again.
You use the EDIT/FDL utility to specify the run time option using the CONNECT
attribute. The program has to use the FDL$PARSE and FDL$RELEASE
routines in order to get the run time values.

Check the following link which talks more about the run time options -
http://www.openvms.compaq.com/doc/73final/4506/4506pro_025.html
-> Chapter 9 Run-Time Options

Regards,
Murali

[Jeremy Begg]

Yes, I have that manual and have read it.

Thanks for confirming my suspicions.

Regards,
Jeremy Begg