COBOL I-O loosing records when disk fills

[John T. Farmer]

Hi all,

I have a COBOL batch program reading EDI data and writing applcation files. Occassinally, we run out of disk space during the job. The COBOL program keeps running happily along writing to a device that is actually full. It is not aborting the program and doesn't appear to be returning a failing file status. I know there must be tons of things to check for. I'm first wondering if there is a problem with the program writing to Cache (logical I-O getting a successful write) and then cache failing on the physical I-O. Are there RMS or COBOL settings I should should be investigating?

Environment: Alpha DS20e, OpenVMS v7.2-1, COBOL v2.6-1060, RMS file system

We have plans to upgrade VMS to 7.3-2 in first quarter 2007 (to support a disk subsystem upgrade), then to VMS 8.x by 3rd quarter. We will upgrade all layered products at the appropriate level at each VMS upgrade.

Thanks for any leads,

John Farmer
john dot farmer at genworth dot com

[Hoff]

The COBOL code involved here likely contains a bug or three.

It's very likely that the code is either not looking at the FILE STATUS data value arriving back from what is likely a failed WRITE, or some other bug-like construct lurks within the USE AFTER EXCEPTION procedure.

From the COBOL WRITE documentation:

"When a program tries to write beyond a sequential file's externally defined boundaries (for example,attempting to write to a full disk device), an exception condition exists as follows:

* The contents of the record area are unaffected.
* The value of the FILE STATUS data item for the file indicates a boundary violation.
* If a USE AFTER EXCEPTION procedure applies to the file, it executes.
* If there is no applicable USE AFTER EXCEPTION procedure, the program terminates abnormally. "

You can also potentially install and use the LD logical disk utility from the OpenVMS Freeware or (better) off Jur's site (see the OpenVMS FAQ for that link) as a test harness; with LD, you can easily create a comparatively small (logical, test) disk, and run your COBOL code against it to see what might happen when the LD disk fills -- and this without filling up one of your production disks.

There are various ECOs for the older OpenVMS releases, and the earliest releases with XFC were a little shaky -- that before the XFC ECOs were applied. I'd definitely look at the COBOL code and its error handling first, but I would apply all mandatory ECOs and -- if you're using XFC in your configuration -- the XFC ECO. XFC is faster than VIOC, so you will likely want to be using it.

And unless there's some specific reason for the two-step, I'd likely upgrade to V8.3 and skip the extended stop on V7.3-2; there's not a particularly huge nor widespread difference between OpenVMS Alpha V7.3-2, V8.2 and V8.3; it's not a classic major upgrade, the version got bumped to line up with the OpenVMS I64 version. (Why test everything twice?) V8.3 looks to be the current release through 2009, per the roadmap, too.

Stephen Hoffman
HoffmanLabs

[Hein van den Heuvel]

As Hoff writes, sounds like a Cobol coding bug.

Neither XFC, nor VIOC doe write behind caching toe program tells it to write, then it will be written, not just an 'oki doki'. Of course, between the cobol write and the system write there is an RMS or RTL buffer. There the data is accepted and 'approved' without the IO occurring. I would have to check when exactly RMS does the exent. Easy enough with SET FILE/STAT ...
How much data do you think the program 'pretends' to write while the disk is full?
$SHOW RMS will tell you the size of its buffers.
RMS can signal a 'WBH' error on the CLOSE, not the WRITE when it has trouble eventually flushing the buffer.

It wouldn't be something as easy as a large pre-allocation / extent (again $SHOW RMS) allowing those writes to continue for real while the disk is full for others?

While intriguing to figure out exactly where it goes wrong, this problem also has a 'doctor it hurts when I do this' aspect to it. Now that you know, just make sure you don't get close to the disk being full . Management (is that you? :-) should be readily convinced that running into a full disk is not the way to do business.
Spend your time cleaning / archiving / expunging rather then studying this secondary problem?

If you really really want to get down to the details, then I have a 'fake cobrtl' which allows you to trace every IO statement. You could enable that while running into the ground (DECram or LD devices work well for this!)
The trace will tell you the RMS status and/or COBOL FILE STATUS and some sample data for each write.

Hope this helps,

Hein van den Heuvel
HvdH Performance Consulting.

[Jan van den Ende]

John,


Hein wrote
>>>
Management (is that you? :-) should be readily convinced that running into a full disk is not the way to do business.
<<<

Perhaps you could calculate the cost of the time you already spent on this.
Now convert that price into the number of gigabytes of diskspace you could buy for that money :-)

hth

Proost.

Have one on me.

jpe

[Robert Gezelter]

John,

I concur with Hoff and Hein as to the likelihood of a bug.

I would suggest that other than a code inspection, the best course of action is to create a "sandbox" version of the application, reading from a standard set of data records, simulating the EDI source, and the output going to a VERY small LD device.

I would definitely watch for how the compltion status from writes is handled. Another case that many programs have tripped over is the sequence of error returns when the diskquota is exceeded and goes into overdraft. This is not likely in your case, but is worth checking.

- Bob Gezelter, http://www.rlgsc.com

[John T. Farmer]

I completely agree that "insufficient disk space" is not acceptable in this day and time. I've been working towards the goal of upgrading our disk devices on our production DS20e as well as brining up another DS20e for D/R and developement. Sadly, the system was installed in 2000 with no subsequent upgrades to hardware or software (o/s or layered products). So, I'm working through all that.

What I found was that due to the age of our system, there were potential disk/controller incompatibilities at every turn. A CPU & disk I-O subsystem upgrade is scheduled for installation late 1st quarter. Someone mentioned why bother with going only to VMS 7.3 instead of 8.3-x. I didn't know that version 8 was not a major release so will consider that. Also found that I don't have the updates service on my contract, so it would be best to buy just one media version for the upgrade.

Thanks,

John