Some weird behaviour accessing RMS file

[Willem Grooters]

Given the file and processing in the attached file, I find a weird behaviour I fail to understand.
When running the sequence as shown, for some keys I find morer than one record, even where SEARCH reveals there is only one. It doesn't matter what environment is used. It will happen in DCL or in our (Dibol) development environment.
ANALYZE/RMS doesn't reveal any error.
This behaviour is gone after CONVERT, but all of a sudden, it does reappear, but it can not be predeteremined what keys will be involved, or how many. Again, CONVERT solves the problem.
My impression is that something is messed up in the secondary index but I couldn't find out what.

Who can explain what's going on and how to prevent this?

Willem

[Ian Miller. · Accepted solution]

it may be worth looking at the file with Hein's rms tools from
http://h71000.www7.hp.com/freeware/freeware60/rms_tools/

may tell you something.

Sometimes ANAL/RMS/INTERACTIVE and wandering around the file structure may help you understand the problem. I've found it helpful on occation.

[Bojan Nemec]

Willem,

Looking at yours FDL I see that the definition of the secondary index is not properly defined. If RMS is working ok this will not be an issue but you newer knows.

Key 0 is a subset of key 1. For key 0 you have: CHANGES no and DUPLICATES no. For key 1: CHANGES yes and DUPLICATES yes which is not possible.

I dont know if this is a reason for yours problems, but if it is, this is an RMS bug.

Bojan

[Jan van den Ende]

Willem,

Reading Bojan, (and knowing what "PLOT" is :-) ), I conclude that you have a fixed series of plots, which are eigther in status "V", or not; and you want to display the plots in status "V", excluding all other statusses. Correct?
I think I will have to agree with Bojan, key1 should be NOduplicates (but changes YES of course).
Btw, kind of strange way to define the alternate key as a primary with one variable character IN FRONT. Histeric/historic reasons probably?

Cheers.

Have one on me.

jpe

[Bojan Nemec]

Jan,

You are right. I missed the possibility to change only the first character. For such implementations (when you have 2 states and need to read fast only records with one state) you can use null keys. This reduce some testing when reading the file.

Bojan

[Willem Grooters]

Thanks to Hein's marvelous program some light at least - see first part of attachement.
All the records that have > 1 in the "dups" column (?) give problems. Given the known output from the program, I guess if there is more than 1 'duplicate', but less than 48, a second 'valid' record will show op, and a third if 48 or higher. This is consistent with the output.

Give this knowledge, I tried to find the cause using ANA/RMS/INTER once again. The thing I found, on V1200 for instance, were three references to this data with a valid pointer (one where "deleted" flag set, and two without any flag set). See second part of attachement.

I'm still waiting for info on the running environment where this happened, but does anyone has some idea how this could happen?

[Willem Grooters]

STUPID system. Had to retry (Server unvailable, filter too slow) and so attachement is missing. Sorry for that.

Bojan, Jan,
Right. It's a kind of plotting: state-based. hence, there is some logic in the way keys were set up.
The primary key names a unit. All units are known, no data is added to the file by THIS program. We use it to extract particular data on that unit that is not related to it's state.
The secondary key is used to extract unit data based on it's state. To get all units in a certain state, this argument is of higher significance that the unit number - and therefore it is set in front of the number.
If the state changes, this key will change and be written back. That may well introduce the problem here, since references are deleted and added.

[Willem Grooters]

Additional info:
This particular program can be used simultaniously by about 8 people, spread over three nodes in a cluster, but working on the same files.
Is it imaginable that this could lead to some issues in synchronization for this particular file (lots of updates)?

Willem

[Jan van den Ende]

Willem,

multi-user access from multiple nodes _MAY_ influence the performance (forced to wait), but should _NOT_ influence the integity of the data!!

Cheers

Have one on me

jpe

[faris_3]

Hi,

Do you use some special options
like read regardless of lock ?

Otherwise this seems like a bug.
You should open a call with your local
CSC if all rating 1 patches are applied.

[Bojan Nemec]

How is possible to get a duplicate key if this is not allowed (see my and Jans previous post)?
Do you use RMS Journaling?

Bojan

[Jan van den Ende]

Bojan,

re-read your own post:
the thing is that duplicates ARE allowed (although logically they should not be able to occur)

[Willem Grooters]

Bojan,

That's just one of those mysteries.

I just checked - dumped the whole file by block - and there is just ONE record where the primary key is "1200" - and so there is just ONE where the secondary key is "V1200".
As to be expected: the same record.
However, as I observed - see previous attachment - there are multiple (adjacent)references in the secondary key index that point to this very same record. At least, it explains why I got this record twice when reading sequentially accoring this second key. RMS seems to be happy with is but I think it's rather confusing.

[Jan van den Ende]

Willem,

since the secondary key should also be unique, can you change your FDL to
KEY1 Duplicates NO, and Convert your file once more?
One out of two: you will NOT get multiple records on secondary key any more, OR, the code that tries to write a second record to the same key will fail, giving a better idea what is wrong, and/or the opportunity to correct.

Cheers.

Have one on me.

jpe

[Willem Grooters]

jan,

Passed to user. I'll have to wait now for the results.

Willem

[faris_3]

>However, as I observed - see previous >attachment - there are multiple (adjacent)>references in the secondary key index that >point to this very same record.

But within these references have you got all but one which have the flag
irc$v_deleted = 1 ?


The ones which have irc$v_deleted = 1
should not be returned by RMS.

[faris_3]

Can you post an $ analyze/rms /fdl file.dat

Do you use RMS Journaling ?

[faris_3]

Additional info:
>This particular program can be used >simultaniously by about 8 people, spread >over three nodes in a cluster, but working >on the same files.
>Is it imaginable that this could lead to >some issues in synchronization for this >particular file (lots of updates)?

I reproduced the problem on V7.3 by using multiple process updating in parallel the
secondary key of the same record.

You should open a call with your local CSC.

Which version of OpenVMS are you using ?

[Hein van den Heuvel]

Interesting stuff. It does sound like a bug.

Willem, what exact VMS version/patch level? And how about an ANA/RMS/STATS output to make the picture complete?

Faris, can you reply with more details on your reproducer? Source code + fdl in attachment? How reliable dies this reproduce? Email me?

Willem>> This behaviour is gone after CONVERT, but all of a sudden, it does reappear,

Convert only uses record data, no old key structures, making completely new key/index structures. So it does sound like an alternate key corruption which is coming back (bug).

Bojan>> "For key 1: CHANGES yes and DUPLICATES yes which is not possible."
Correct, duplicates are not possible in this case. But please note that 'duplicates yes' is only a request to RMS to make sure no record with duplicate key value is being inserted. The file structure is exactly the same whether duplicates are allowed or not. So dups=no requests RMS to do a tiny bit more work. And since you know there can not be duplicates in this case it would seem a little cheaper not to ask to check. Whether it actually is cheaper or not does not matter, just showing why it may make sense to have this seemingly contradictory definition.
HOWEVER... whether duplicates are allowed or not DOES change the internal to RMS update behaviour. And RU-journalling adds yet an other twist.

So, Faris/Willem I'd like you to recovert with dups=no, to see whether this may deliver a workaround.

Jan>> kind of strange way to define the alternate key as a primary with one variable character IN FRONT

Actually, that would make sense for at least two reasons.
1) to de-duplicate the alternate record key. Otherwise you may have too many 'V' records and loose insert performance. Admittedly this is only significant with tens of thousands of dups and painfull millions of dups, but it is nto a bad idea.
2) to impose an order 'get next sequentially' with the key or refence set to an alternate key with dups will essentially return them in 'order of arrival' (which convert can destroy). That may or may not be desirable. Adding a primary key, or date field makes the order predictable.

Bojan>> when you have 2 states and need to read fast only records with one state) you can use null keys.

YES. but not the problem here.

Willem>> Thanks to Hein's marvelous program some light at least - see first part of attachement. All the records that have > 1 in the "dups" column (?) give problems

Thanks :-)! But the particular area you use in the program is little tests. It may well reported 'deleted' sidr array entries as real. RMS would/shoudl not return those. But since you have actuall problems on tose very records, I guee it is showing something.

Willem>>less than 48, a second 'valid' record will show op, and a third if 48 or higher.

Well, your FDL did NOT show the AREA info. If you did not do anyting special, then the bucket size might be just 2 (default) or even 1. Every sidr entry is 7 bytes. so with a bucket size of 2, you can store only (2*512 - 15 - keysize) / 7 =~ 150 dups / sidr. So with 48 or 60 sidr entries(for updated=deleted) you may well need to cross into a second bucket. This may well turn out to be the breaking point.
SO for an other potential workaround... reconvert with bucketsize = 24 or 63 or so (no science here, just a large number).

Faris>> The ones which have irc$v_deleted = 1
should not be returned by RMS.

Correct.


Hope this helps some,
Hein.

[Willem Grooters]

@faris

But within these references have you got all but one which have the flag
irc$v_deleted = 1 ?

Almost right. All but TWO (pointing to the same record), and that's what's wrong.

Great you could reproduce the problem. I guess it's a SMALL file and a SMALL program. Could you please zip it all up and mail it with an explanation to willem-at-grootersnet-dot-nl (That's the only way I can get it passed the gates...). I'll put the call.

I added the ANA/RMS/FDL output.
Asked the source of the problem, they're running 7.1-2 (don't know of patches but since it occurs on 7.3 as well I don't think that matters), no RMS journalling for what is known.

@Hein:
The program just put me on the track to dig deeper. Good program to check the structures! We did have problems with performance and this program will be quite some help!

Willem

[faris_3]

reproducer sent to Willem and Hein

[Jan van den Ende]

Hein,

The way I read your response, looks like explaining the reason for adding the primary key to the secondary. Well, that always looked logical to me, and you made it doubly clear in your Brussel session.
I was alluding to the fact that the status field was IN FRONT, instead of APPENDED TO, the primary key.
But on second thought, I guess it is only on esthetic, and not on thechnical, grounds that the remark had some vaule.

Cheers.

Have one on me.

jpe

[Willem Grooters]

I ran reproducer on VMS 7.3-1 but couldn't reproduce it - it might have been solved 'under the hood'. Still, call posted to HP, to be sure.
Anyway, it's not a big problem, since the file in question is a small one and convert, if required, is done within half a minute (tested it here), but kept as a reminder where it is not that easy. Just use issued workaround, and hope for the best. Otherwise: update VMS (to 7.3-1 at least).