MButler,
Ok, now the picture comes into focus. My recommendation is to take one step back and focus on the real issue: performance.
Fact: The program is in FORTRAN.
Fact: Data volume is "high"
Unknown: What percentage of the CPU is used?
Unknown: What (precisely) does "high" mean [quantitatively]?
Unknown: What configuration are we speaking of?
Generally, in FORTRAN, the CPU cost is in formatting input and output. If the formatting can be bypassed, the overhead actually associated with RMS processing of an appropriately formatted file is very low.
What is the performance difference between tape and disk? Often, the answer is rooted in the fact that since tape is a sequential medium, not a shared, direct access medium, there is no need to control space allocation. Indeed, when doing a bandwidth intensive operation such as BACKUP, the overhead of the File Extend operations produce the most impact on performance.
File extension is expensive, and it is common to all disk usage, whether one uses RMS or bypasses it.
So, in the case described, what would I do as a first cut at solving the problem of changing writing to tape to writing to disk?
I would implement an interface common to all of my approaches, and implement it first using good RMS practices. If, as is likely, this approach yields 10,000% of the required performance (yes, I do mean a factor of 10 times the required performance), the need is satisfied without resorting to the low levels of using QIO directly.
First, I would abstract the interface to the output management, along the lines of CREATE_OUTPUT, WRITE_OUTPUT, and CLOSE_OUTPUT (while I was creating the interface library, I would also at a minimum define the corresponding READ routines). In your case, this all corresponds to a handful of subroutines and a named FORTRAN COMMON to maintain status.
Second, I would start by creating a standard RMS file, using the FORTRAN OPEN statement. I would specify a significant block size, blocking factor, and most importantly, file extension size and initial allocation. [Note: Hein and I have a friendly disagreement on whether blocking factors in excess of two are useful, the context of a rapidly growing file with extensions is one of the contexts where IMHO it is beneficial.]
Note that the working set and associated quotas may need to be increased to allow for the resulting buffer pool.
Third, Measure performance of the result. In most cases, this the combination of unformattted (or effectively unformatted) records, RMS buffer pool, large allocations, and large extents is very much more than sufficient to solve the problem.
The next step would be to very that all RMS settings that optimize output performance are correctly enabled (e.g., write-behind).
In almost all cases, that is sufficient. In any event, digging in and managing the file yourself using QIO (including manually extending the file, buffer pooling, etc.) are far, far more development effort. The approach that I am describing should require minimal time to implement and prove.
If performance is still an issue, more exotic approaches can be considered. For example, even volume shadowing (with a memory disk as one of the members of the shadow set) becomes a possible acceleration strategy. Bypassing RMS (as opposed to optimizing one's use of RMS) has far lower yield.
If performance is lacking, my first stop after the above would be directly using the file created using FORTRAN OPEN with RMS, bypassing the FORTRAN IO/formatting processing.
I hope that the above is helpful. If I have been unclear, please let me know.
- Bob Gezelter,
http://www.rlgsc.com
