RMS: Descending key?

[Willem Grooters]

I have an indexed sequential file, where a field contains the date as a number (yyyymmdd).
This fields is used as a part of a (segmented) secondary key to access data, from a certain date and from there on BACKWARDS in time.
The data is NOT static, can be changed, and duplicates are allowed.

It would be very handy if this key could be defined in reverse order - but I cannot find a way to get this done in creating the file.

I do NOT have the ability to handle this in the program, sort the file on beforehand or have the index read-in and sorted in memory, so the key MUST be defined "order=descending" or likewise from the start.

Is this possible and if so, how to achive this?

Willem

[Wim Van den Wyngaert]

Willem

Gefeliciteerd met de mooiste hoed.

I remember something of adding TYPE=DSTRING to the FDL
but I don't remember if this also requires program changes (you read reversed by default).

[Kris Clippeleyr · Accepted solution]

Willem,

Check the "OpenVMS Record Management Utilities Reference Manual, Chapter 4, File Definition Language Facility".
You can specify a key of "TYPE DSTRING" in the FDL file. This should give you a key in descending order.

Greetz,

Kris

[Wim Van den Wyngaert]

Willem,

My memory came back after 15 years.
You will have to modify your programs unless the program can handle the reversed chain.

Wim

[Hein van den Heuvel]

Ha die Willem,

Mooie nieuwe hoed!

Besides the 'obvious' set of DESCENDING KEYs that RMS offers it provides an other alternative through RAB$L_ROP bit RAB$V_REV.

Check it out in:
http://h71000.www7.hp.com/doc/731FINAL/4523/4523pro_011.html#rmsref_rab_rop
Notably table:
http://h71000.www7.hp.com/doc/731FINAL/4523/4523pro_011.html#combo_table

This is specifically handy if you can not change the file, but can change the program (a little).
How does it work? Think about how 'three lefts make a right'. If the 'reverse' key is in the same (SIDR)data bucket, it simply goes left instead of right. If it was the first key, it must go up, left, down and find the right-most. The performance of this is less than that of a dedicated key, but very respectable for modests access. If compares favourably to having to maintain/update an additional index structure. High insert/modiy -> check out _REV. Low insert/modify but hight read reverse -> check out adding an index.

The attentive reader may want to check out XABKEY. You'll see that the code for a each descending key type = ascending + 32 (just one more bit ;-).


Wim,
You don't have to do much in a program to use a descending key. Just select the right RAB$B_KRF and presto!

Groetjes,
Hein.

[Willem Grooters]

Wim,
The advantage of this program: it's brand new so no chnages ;-)

Hein,
As stated: I have to rely on the standard methods and cannot use the RMS structures directly. Still, thanks for the links.

Anyway: the program reads backwards, just what I needed.

Willem

[Hein van den Heuvel]

>>> I have to rely on the standard methods and cannot use the RMS structures directly. Still, thanks for the links.

I was talking about standard methods, fully supported and accesible by most/all languages (Cobol, C,...)

The +32 remark was just for the curious folks. All descending keys types are fully defined both in XABKEY and the FDL equivalents.


btw... I am sure you realize this, but if you add an index/key with the descending key type, then you do not need a new field in the record. You can just re-use the fields from the existing ascending key.

For example:

file; org indexed;
key 0; name "sleutel"; seg0_l 10; seg0_p 0;
key 1; name "omhoog"; seg0_l 2; seg0_p 10; seg1_l 6 seg1_p 20;
key 2; name "omlaag"; seg0_l 2; seg0_p 10; seg1_l 6 seg1_p 20; type dstring;

Groetjes,
Hein.

[Willem Grooters]

Hein,

That's funny - I have the the same structure: a descending key in two segments. However, I run into unexpected behaviour.

Consider this:
Given ascending keys:

Segment1 Segment2

AAAAA _____ (where _ = space)
AAAAA 12345
AAAAA 23456
BBBBB 12345

Normally, if we have to read in part of a file, we try to locate the first record that matches our criteria - where a part of the key contains a value that may, or may not exist. In the latter case a status "KeyNotFound" is returned if data exists where the key value is 'higher', which may be just what we need.
So, when I specify "AAAA 12346", I get status KeyNotFound but record "AAAAA 23456" is current.

For this key in descending order, that would mean the record retrieved would contain the value just BELOW the requested one.
So when the key to search is "AAAAA 23457" - which doesn't exist - I expect to get status "key not found", and "AAAA 23456" to be current as the starting point.
But no: I get status "EndOfFile" and the last record!

(There is no NULL specification in the FDL, since ths first segment will always be filled; the second can be empty. and I only have a descending order on these segments)

Is this a mis-understanding on my side, a dependency of our environment (Synergy Dibol), or just a bug?

Willem

[Willem Grooters]

***** PLEASE IGNORE PREVIOUS CONTRIBUTION *****

It's my fault: completely overlooked a typo in FDL and program....


Willem

[Wim Van den Wyngaert]

Working wizards also make mistakes.
Non working wizards don't make mistakes and get promoted to manager.

Wim

[Jim Strehlow]

When our application required both ascending and descending "dates", we had one column of data ascending
20040101
20040102
20040103

and the program also stored the 9's complement in another column also in ascending order.

99999999 - 20040101 = 79959898
99999999 - 20040102 = 79959897
99999999 - 20040103 = 79959896

Reading rows 79959896, 79959897, and 79959898 in ascending order yields 20040103 then 20040102 then 20040101.

(We included time also; but you get the idea.)

We had a verification utility we ran occasionally (monthly) to ensure no one accidentally "corrupted" the 9's complement data.

Jim, Alameda, CA, USA

[Hein van den Heuvel]

Well Jim, applications such as yours is exactly why RMS engineering decided to offer reverse sorted indexes. It's just to error prone and too imposing on the data design to require a dummy fields to solve this classic problem. Between the risk of data inconsistency and the waste of space the descending index clearly is the way to go, and RMS Engineering believes that for many applications the _REVerse index lookup is even a better solution (a little more work on lookup, a lot less work on insert.)

Hein

[Jim Strehlow]

Understood, however our COBOL application did not allow for descending keys at the time. It might still be a restriction(?) in COBOL.

I just wanted to illustrate an unelegant work-around for those who encounter a language-specific dilemma.

Jim

[Doug Phillips]

Hein wrote <...exactly why RMS engineering decided to offer reverse sorted indexes. . . .>

Sorry, but I agree with Jim 100%. We've had to do this in our applications, too, because it is normally only one "SEGMENT" of the key that must be inverted, not the whole "KEY".

A typical business example is a sales history file. Primary key would be:

Customer+InvoiceDate+InvoiceNumber+(...)

For inquiry, Customers should be presented in ascending sequence, and within each customer the "newest" invoices should be shown first and then in descending order. Currently, RMS offers no way to do this without creating a "complement" segment.

Likewise, I've never found a real use for "NULL" keys. Null "SEGMENTS" would be nice, but if an entire key can be null, then the key and probably the entire data record structure is IMHO either very poorly designed, or possibly the application using such a file doesn't live in my world.

Doug

[Hein van den Heuvel]

Doug wrote:

'Currently, RMS offers no way to do this without creating a "complement" segment.'

Right. This is true. Somehow RMS Engineering got the business justification to do what was planned all along: a data-type per key segment. The XABKEY already has an array for xab$b_typ, just like pos and len, but it is not being used :-(.

Doug also wrote:

"Likewise, I've never found a real use for "NULL" keys. Null "SEGMENTS" would be nice, "

Probably true for you, but this is less clear cut to me.
What is clear cut that RMS _should_ simply have had full null key value string, not just the byte. I think this did not happen in a desire to stay with the existing in memory XABKEY and on-disk KEYDEF ($KEYDEF in sys$Library:lib.mlb) structures.
A full function null key would need a 255 byte additional field.
But I have a hard time imagining a multy segment key (cust+datge+invoice) with lots of records in the file having the same null/dontcare value. That sounds like wrong design to to me, mixing record types too much. When you mix text and binary segments, it woudl be hard to define a valid single-byte null key, but I do not see that much in practice either, precisely because rms does not allow one to mix types. That leaves the ascii number strings mixed with text where perhpas a space is not valid for the numbers, and a zero looks odd for the text.

Cheers,
Hein.

[Doug Phillips]

Hien wrote: >>But I have a hard time imagining a multy segment key (cust+datge+invoice) with lots of records in the file having the same null/dontcare value.<<


I suspect the null key came about because it was envisioned that a file could contain lots of records where some field's value would be assigned post record creation and used as a secondary key. There would then be lots of records with a blank or zero secondary key value that would add to the dreaded duplicate key overhead if not excluded.

However, as a very wise man has said;-), duplicate keys should be avoided whenever possible. So, part of the secondary key should contain something else that would make it unique. SO...

Null keys aren't very useful. Null "segments" make more sense. I'd rather see:

"SEGn_NULL" with a value definition that would cause exclusion of index entries that matched the value in that segment, or better still,

"SEGn_EQUAL" with a value definition that would cause exclusion of index entries that did NOT match the value in that segment.

I guess mixed ascending and descending segments within a key would be way too much to even mention:-)

Doug