Multiple open files of the same file

[Cor Mom]

As a starter: Happy new Year !

We are running OpenVMS 8.2-1 and 8.3 and write our application in Pascal. We are currently porting our application from Alpha to Itanium. We have noticed that some applications which often open and close files with the Pascal OPEN ans CLOSE keep their files open after a file close. This means that some files are opened hundreds of times by the same process. The only way to close the files is to stop the process.

Has anybody seen this behaviour before on Itanium, than I would like to share the solution with you.

regards,
Cor

[Wim Van den Wyngaert]

I would say that the close failed.
http://h71000.www7.hp.com/doc/82final/6083/6083pro_020.html#error_recovery_param_sec

Did you handle the close errors in some way ?

And sure it was not reacting the same on alpha ?

Wim

[Hein van den Heuvel]

If I read this correctly, then the same (pascal) application code on Alpha properly closes the files and probably re-uses the channels, whereas on Itanium it builds up used channels and (probably) RMS Ifabs.

Sounds to me you may have stumbled into a buglette which may or might not be dependend on a specific way you use the code.

Can you reduce the problem to an example with just a fews opens/closes proving the point? ($HIBER after last close, before exit). Care to share that here?

You may well want to submit a formal problem report to HP with such reproducer

For further analysis I would suggest.
$SET FILE/STAT[/SHARE] sample.dat
$ANALYZE/SYSTEM ... SHOW PROC/CHAN and ... SHOW PROC/RMS=(FAB, IFAB, FSB)

btw... You realize of course that the repeated open and close for the same file from the same process is 'sub optimal' (nice for: sucks big time!). This problem hopefully triggers a review of the need to do so.

Hth,
Hein van den Heuvel
HvdH Performance Consulting

[John Reagan]

Is the application multi-threaded? The Pascal RTL has smarts to deal with multi-threaded applications where more than one thread might be doing a CLOSE at the same time. The RTL has some internal data structures that require exclusive access. If the RTL thinks that one thread is doing a CLOSE, it will just mark the file as "close pending" and return. The RTL's design is such that the other thread would then call $CLOSE for the file variable when it is done closing its own file.

John
Pascal Project Leader

[John Reagan]

Another question... You have compiled /CHECK right? Talking to a co-worker, we came up with some scenario where your program might walk over some critical piece of RTL static memory and trick the RTL into thinking it was multi-threaded and that some other thread (which doesn't really exist) was in the middle of doing a file CLOSE.

[Cor Mom]

Ok guys, its not a bug, that's for shure.

The original code opens the file to log an event, but instead of closing the file it flushes the buffer to the file, but the file remains open. The next time it opens the same file using a new file variable, logs the event and flushes its buffer. Now the file is open two times, and so on. Because we are not allowed to make large modifications, I replaced the Flush with a Close. There is a manual step in the build procedures to copy new object files into an object library which is used to link against, which I did not know. That's why I did not see the result of my modifications. Now it works ok, it is not the most beautiful code I have seen, but I have leave it in this condition.

Thanks for support you have given me.

Cor

[Cor Mom]

I am still not ready with this thread.

Recently I solved two situations in which a file was opened with the Pascal OPEN procedure, but never closed. On Alpha this was never a problem, i.e. there were NO remaining open files after the read operations. Op I64 the files remain open until they are explicitly closed. Of course I say you should close a file when you are ready with reading, but the Pascal manual say the following at the Pascal CLOSE procedure:

"Each file is automatically closed when control passes from the block in which it is declared"

If you ask me, this means that when a file has been opened using a local declared file variable and after opening and reading the file the procedure is left without closing the file, that the file should me closed automatically, because the file variable gets out of scope.

If this is the correct interpretation, then I think we have a bug here.

Regards,
Cor

[Wim Van den Wyngaert]

Or may be the 2nd open failed on Alpha (for whatever reason) and the variable of the 1st open was still valid. And so, you could continue your file operations (on Alpha).

Wim

[Cor Mom]

Wim,

That crossed my mind too. In one of the two situations this could be happened, but the other I think not, because this file will be handled in DCL afterwards. On Alpha this gave no problems, on I64 there comes an error that the file is opened by another user.

Cor

[Hein van den Heuvel]

>> and the variable of the 1st open was still valid.

If it is still valid, welll then it must still bd valid and the bug would in in the Alpha version.

Do you mean that the old value in emory happened to still be there? So any uninitialized value would be interpreted as a potential file handle and could close a random file when opening a new one? Yikes.


Back to the basics.
Try to create a simple recreator.
(As simple as possible, but no simpler. :-)
USe that to 'see' exactly when the file is closed in the Alpha version. I would use a debugger in 1 session, and $shwo dev/file in another or anal/sys show proc/chan. (Yes i would possibly use 'shwo' as this is a common mistype I make and my login.com sets up an alias for that :-)

So is it closed on Aplha?
When is it closed?
Returning from a function?
Open of the next 'instance'?
Do you believe it should be closed according the the Pascal language/user manual?
Fix code, or submit bug report as appropriate. And do keep us poseted!

Groetjes,
Hein.

[John Reagan]

Ah, now we have enough info!

The Pascal RTL on I64 does not automatically close files when they leave scope like the Alpha RTL does. It is a bug on my list. I didn't think anybody but me (and the standard) relies on such things. I'll try to get it fixed for V8.3-1 or whatever it is called. In the meantime, use an explicit CLOSE in the routine.

[John Reagan]

A little more info...

There are two cases where auto-closing occurs.

1) If the routine is being unwound, the RTL will auto-close the appropriate files. Works fine on VAX and Alpha. Doesn't work on I64 V8.2 and V8.2-1. Fixed for V8.3.

2) At the routine exit, there is an invisible call to PAS$CLOSE_LOCAL. The RTL tries to auto-close the appropriate files. Works fine on VAX and Alpha. Doesn't work on I64 at all. Not even V8.3. When I fixed case #1 above, I realized that existing object and images that called into the PAS$CLOSE_LOCAL routine were passing the wrong piece of information. I didn't want to invalidate existing objects and executables so I simply ignored this case hoping that one me and the standard would notice. I obviously was wrong. I apologize.

I'll try to fix it without requiring a new compiler to match, but I'm not sure. So we'll need a new RTL and perhaps a new compiler.

I've added it to my list.

[Cor Mom]

John,

Thank you for your info.

It is all clear now to me now. Until yesterday I did't know there was something like an auto-close. Personaly I should prefer a compiler warning that a file is open when the file-variable get out of scope in stead of an auto-close, because now the programmer is not aware that he has forgotten to close the file.

Regards,
Cor

[John Reagan]

The compiler could only issue such an info for variables actually declared. Although that would be the first time we'd issue an info for relying on a standard (albeit confusing) feature.

For heap-allocated file variables that you forgot to close, we don't know at compile-time, only at run-time. We don't have any options to control the run-time behavior like raise an error instead of auto-close.

I'll add it to my to-do list and will think about it some more for the next release.

[Jan van den Ende]

Ha die Cor!,

from your Forum Profile:


I have assigned points to 10 of 28 responses to my questions.


The other not-fully assigned issues are from 2004 & 2005!

Maybe you can find some time to do some assigning?

.. Special entry for this thread: be extra generous to John Reagan. He does not show up here often, but WHEN he does!
...

http://forums1.itrc.hp.com/service/forums/helptips.do?#33

Mind, I do NOT say you necessarily need to give lots of points. It is fully up to _YOU_ to decide how many. If you consider an answer is not deserving any points, you can also assign 0 ( = zero ) points, and then that answer will no longer be counted as unassigned.
Consider, that every poster took at least the trouble of posting for you!

To easily find your streams with unassigned points, click your own name somewhere.
This will bring up your profile.
Near the bottom of that page, under the caption "My Question(s)" you will find "questions or topics with unassigned points " Clicking that will give all, and only, your questions that still have unassigned postings.

Thanks on behalf of your Forum colleagues.

PS. - nothing personal in this. I try to post it to everyone with this kind of assignment ratio in this forum. If you have received a posting like this before - please do not take offence - none is intended!

PPS. - Zero points for this.

Proost.

Have one on me.

jpe