getting around alpha vms P0 memory limit

[Willem Grooters]

I've programmed 'dynamic array' in FORTRAN years ago, using LIB$CREATE_ZONE and LIB$GET_VM, rabging form 1000 to 100.000 elements; both allow for 64 bit addresses.
I'm not sure whether this holds for COBOL, but AFAIK, the language does contain the required types.
If you handle the data seqentially, think of using linked lists, that you can position in P2 space.

[Hein van den Heuvel]

COBOL does not do pointer variables / dereferencing.
When using Global sections or GETVM you have to 'cheat' by calling subroutines to pass the address of a structure or by calling LIB$COPY / MOVC3 functions to copy the data into a working-storage variable, to act on it.

There is little point speculating until David (and we) learn pertinent details.

>> These arrays keep having to be extended and we are getting close to the P0 limit and failures have occurred recently because this limit has been exceeded.

"extended" how ? Source code changes ?
"failures" how ? Error messages ? Link time ? Run time, Application or systems level?
"have occured" when ? all the time ? during SORT ? During data load ...

My impression certainly is a Virtual Address issue, hitting the 1GB P0 limit. Folks just do not tend to make that up... but they could, to hide lack of true understanding. It could still be a simple PAGFILQUO or such problem until we hear pertinent details.

Cheers,
Hein.

[John Gillings]

P0 space limits are a real issue. We see far too many processes hit the wall as we scale up workloads. It's just a reminder that OpenVMS is really a 32 bit OS which has had some 64 bit stuff bolted in the middle.

It's a very clever way of extending the address space, BUT no one has yet found a simple way to automagically make a 32 bit program exploit 64 bits. There's not a lot of incentive for software suppliers to go back over old code and rework it to use 64 bit space, then redo all the necessary testing.

I've attached a modified version of a procedure we use to monitor the P0 consumption of processes. I've removed all the site specific stuff, but without much testing. You will want to update the alarm code to suit your site.

[David Ben-Nathan]

Thanks again for all the help so far. fyi, the arrays keep growing because customer base is increasing and also more and more functionality keeps getting added all the time. I have created a routine today that allocates P2 space calling SYS$_REGION_64 and SYS$_EXPREG_64. Can someone tell me what the system service calls are to actually start populating the area reserved with data and also to read that data back. Someone mentioned LIB$MOVC3 but that doesn't seem to take quadword arguments and I take it that there are routines that should be used with the ones I've used above. Also, reading chptr 10 of the VMS Programming Concepts manual, it says that P2 is part of process private space. Some of the flags on the calls for the routines above specify protection of the region created and I was wondering why that's necessary if it's all part of a process's own private space. Also, there seem to be a number of similar system service calls and it's hard to tell them apart [eg LIB$GET_VM, similar routines for creating Zones etc]. One last question - what are the limits on these routines - presumably at some point they could fail because there is not enough contiguous space available in memory ?

[John Gillings]

David,

See routines LIB$CREATE_VM_ZONE_64,
LIB$GET_VM_64 and other related routines:

$ HELP RTL LIB$ LIB$*VM*64

and, for more detail in the LIBRTL reference.

To move data around, use OTS$MOVE3 or OTS$MOVE5, they handle 64 bit addresses for source and destination arguments, and the length argument can be up to 2GB. Note that the length is passed by immediate value.

Have a look through the various porting manuals for details of handing 64 bit values. Beware, this might look very ugly in Cobol. Maybe consider writing your 64 bit stuff in another language, with better support for pointers, and access it as a procedural interface from Cobol.

[Hoff]

Given the underpinnings of a virtual memory system, it is typically entirely feasible to choose double-map the data as required.

And given I could not locate a readily available source code example using the 64-bit system services to double-map virtual memory between P2 space and P0 space, I wrote one:

http://labs.hoffmanlabs.com/node/1413

This is the "winking" technique that I referenced earlier.

There is certainly some overhead here with the system service mapping, but hauling around large tracts of virtual memory (via movc3 or movc5 or otherwise) is going to have its own and substantial issues around memory paging and around processor overhead.

You can't map the whole of P2 space into P0 space, but you can certainly map parts of P2 space and "wink" your buffers into P0 space as needed.

Using this technique as part of an application-defined programming interface (whether shareable image or otherwise) means you can keep a consistent interface for your code, and the COBOL code can be largely oblivious to the use of 64-bit address space. And you will need to add some code written in a 64-bit capable language into the COBOL application here to deal more directly with P2 space.

[David Ben-Nathan]

Thanks very much for all the replies, in particular to Stephen Hoffman - I have taken a look at the routine you wrote and will see if I can work it into a Cobol routine [or maybe a cobol program could call a C routine to handle accessing the P2 memory. Sorry I couldn't reply sooner but I had not time during the week.