Is BUFSIZ re-entrant?

[Neil Burman]

Dear Readers,

I have a program written in DEC BASIC running on an Alpha 2100 running VMS V7.1. The program runs in a batch job. It reads from a tcp/ip socket and processes the results.

I am using a Read Attention AST to check for incoming data on the port. The general flow of the program is…

.
.
Queue Read Attention AST (HEAD_READ_AST)

While not timeout
CALL CHECK_TIMEOUT
EndWhile

Continue Processing
.
.

Everything runs fine and dandy for anything between a couple of weeks and a month or so. Then the following occurs….

%OTS-F-FATINTERR, fatal internal error in Run-Time Library
%TRACE-F-NOMSG, Message number 0009804C
image module routine line rel PC abs PC
DEC$BASRTL 0 000000000003B060 00000000000CD060
DEC$BASRTL 0 000000000003A264 00000000000CC264
DEC$BASRTL 0 0000000000045AC8 00000000000D7AC8
DEC$BASRTL 0 000000000000E2F8 00000000000A02F8
----- above condition handler called with exception 00178014:
%OTS-F-FATINTERR, fatal internal error in Run-Time Library
----- end of exception message
0 FFFFFFFF85E6E6BC FFFFFFFF85E6E6BC
DEC$BASRTL 0 000000000003B060 00000000000CD060
DEC$BASRTL 0 000000000003A264 00000000000CC264
DEC$BASRTL 0 0000000000045AC8 00000000000D7AC8
DEC$BASRTL 0 000000000000E2F8 00000000000A02F8
----- above condition handler called with exception 00178014:
%OTS-F-FATINTERR, fatal internal error in Run-Time Library
----- end of exception message
0 FFFFFFFF85E6E6BC FFFFFFFF85E6E6BC
DEC$BASRTL 0 000000000003B060 00000000000CD060
DEC$BASRTL 0 000000000003A264 00000000000CC264
DEC$BASRTL 0 0000000000045AC8 00000000000D7AC8
DEC$BASRTL 0 000000000000E2F8 00000000000A02F8
----- above condition handler called with exception 00178014:
%OTS-F-FATINTERR, fatal internal error in Run-Time Library
----- end of exception message
0 FFFFFFFF85E6E6BC FFFFFFFF85E6E6BC
DEC$BASRTL 0 000000000003B060 00000000000CD060
DEC$BASRTL 0 000000000003A264 00000000000CC264
DEC$BASRTL 0 000000000004BE8C 00000000000DDE8C
ADD_SERVER ADD_SERVER HEAD_READ_AST 23252 0000000000008B84 0000000000048B84
0 FFFFFFFF800B5458 FFFFFFFF800B5458
DEC$BASRTL 0 000000000003AF8C 00000000000CCF8C
DEC$BASRTL 0 000000000003A264 00000000000CC264
DEC$BASRTL 0 000000000004BE8C 00000000000DDE8C
ADD_SERVER ADD_SERVER CHECK_TIMEOUT 42336 0000000000014E1C 0000000000054E1C
ADD_SERVER ADD_SERVER ADD_SERVER 8095 0000000000003720 0000000000043720
0 FFFFFFFF85F8F118 FFFFFFFF85F8F118
Q_BATCH job terminated at 5-JUL-2005 14:26:06.06


Looking at the code in the .LIS file…. In HEAD_READ_AST ….

I2T2 23247 DECLARE LONG CONSTANT ITEM_SITE = 1%, &
I2T2 23248 ITEM_OFFICE = 2%, &
I2T2 23249 ITEM_AREA = 3%, &
I2T2 23250 ITEM_REGION = 4%
I2 23251 %END %IF
I1T1 23252 IF BUFSIZ (QRY_DAT) = 0%
I1T1 23253 THEN
I1T1 23254 RES$FILE% = QRY_DAT

In CHECK_TIMEOUT….

I1 42335
I1 42336 IF BUFSIZ( DOCNUM_FILE%)= 0%
I1 42337 THEN
I1 42338 OPEN "DOCNUM_DAT" AS FILE #DOCNUM_FILE% &

Note the simultaneous calls to BUFSIZ. I presume that there is some re-entrancy conflict h

[John Gillings · Accepted solution]

Neil,

I'd expect all BASIC RTL routines to be AST reentrant, and in this case especially if the parameter referred to a different file variable.

If it worries you, maybe bracket the suspect section with a $SETAST disable and restore. Something like:

wasset=$SETAST(%VAL 0) .EQ. SS$_WASSET

critical section

IF wasset THEN $SETAST(1)

(apologies if the BASIC syntax isn't correct, hopefully you get the idea)

I'd also recommend enabling process dump so you can do a post mortem and/or log a case to elevate the dump into VMS engineering.

[John Gillings]

Neil,

After a bit of "fiching"... I don't have an exact alignment between the V7.1 DEC$BASRTL and your traceback numbers, but it's fairly close.

Unfortunately the module I think you're in has lots of points that signal FATINTERR. It's documented as "Environment: User mode, AST level or not ot mixed". So, your code *should* work.

The signal I think you're hitting is doing a sanity check to do with a recursive I/O on the same LUN. From your code, I'm kind of assuming QRY_DAT and DOCNUM_FILE are different LUNs? Can you confirm that?

Following the revision history of the module, I can't see anything that looks relevant. Can you reproduce this on a later version?

Oh, and now I notice that you're V7.1 forget process dumps, and forget elevating to engineering (way out of support window). It may be the best you can do for the old system is avoid the issue by blocking ASTs around the non-AST level call to BUFSIZ. Just make sure you save and restore the state, rather than blindly block and unblock.

If you can build a test program that hammers BUFSIZ with ASTs and reproduce the error on a current version, that would be worth submitting to Basic engineering (but it's very unlikely they would backport any fixes to V7.1).

[John Gillings]

Neil,

OK, now I think it's a bug. I can reproduce it under V8.2. I'll let the Basic folk know.

However, please remember that everything is run on metrics these days. They will probably need a formal elevation against a service contract in order to fix this.

I also note that it appears you're using BUFSIZ as a means to find out if a file is open on a particular channel. Although that's a neat trick, I don't think it's an intended use for the function. It might be "cheaper" to just maintain your own flag. Particularly at AST level. Consider that BUFSIZ on an open channel probably needs to interrogate the I/O data base.

[Neil Burman]

Hi John, many thanks for your help on this. I have implemented a work-around avoiding the BUFSIZ and have suggested to the custodian of the code that they may wish to report the problem to HP. Thanks once again, Neil.

[John Gillings]

Neil,

I've sent a reproducer (see below) to BASIC engineering. They now know about it and *might* fix it on the basis of my report, but it will take a formal elevation to push it up the priority stack. I can't lodge a formal elevation on your behalf as this hasn't been reported to me through a support contract (that's what paying is for!)

My concern here is that although it's convenient to use BUFSIZ to determine if a LUN has been opened, it's a VERY expensive way to do it, and the cost paths are the wrong way around.

Consider, in your code you'll execute BUFSIZ the first time when the LUN is closed - it will quickly determine that the LUN isn't open and return a 0. From then on it will be executed against an open LUN. BUFSIZ will work its way down through the RTL and operating system to ask the device driver for the buffer size. That's a lot of stuff to execute many times, effectively to implement a simple first time flag.

You'd be much better off declaring flags DOCNUM_FILE_OPEN and QRY_DAT_OPEN and use them to determine if you've already opened the file. Indeed in BASIC re-opening an open file is not considered an error, so you could even execute the OPEN unconditionally with impunity.

That doesn't mean the BUFSIZ bug isn't important. It just means there are better ways to do what your code does, and as a side effect, avoid this bug.

Simple reproducer (note, in production code, the second $SETAST should be conditional on the first $SETAST returning SS$_WASSET):

2 sub ast(long astcnt, w,x,y,z)
external long ast
declare quad delta
delta=-100000
astcnt=astcnt+1
IF BUFSIZ(2%)=0%
THEN
OPEN "F2" AS FILE #2%
END IF
call sys$setimr(,delta,ast,astcnt,)
subend


3 external long ast
declare long maincnt,astcnt
maincnt=0
astcnt=0
call sys$dclast(ast,astcnt,)
while 1=1
maincnt=maincnt+1
! call sys$setast(0% by value)
IF BUFSIZ(1%) = 0
THEN
OPEN "F1" AS FILE #1%
END IF
! call sys$setast(1% by value)
next
END

[Neil Burman]

Hi John, many thanks. I believe the history of the code is that the core of it was originally written in BASIC (plus-2?) on a PDP-11 and was converted to run on Alpha by an outside company. Some of the include files that I need for my application unfortunately contain the BUFSIZ method of open LUN detection.

I agree entirely with your comments about the use of BUFSIZ to determine whether a file channel is open or not. Whether the PDP-11 heritage may explain why this was done (an attempt to conserve physical memory by not using common flags perhaps?) I am not sure.

Thanks once again for looking into this. I shall pass on your suggestions to the company that runs the code (I am but a mere contractor). Best regards, Neil.

[John Gillings]

Neil,

BASIC engineering have identified the problem. It was a "day 1" bug in the handling of recursive I/O (CCB context stack and unstack routines had a small timing window which could expose an inconsistent stack state).

Interesting that any AST that interrupted an I/O in progress, and itself performed I/O, could theoretically have caused the same problem, but as far as we can tell, yours is the first report.

The fix will be checked into a future release of OpenVMS (since it's an RTL, the fix goes with the OS, not the compiler).