topic Re: Program size 1GB limit in Operating System - OpenVMS

Program size 1GB limit

alphacat — Fri, 14 Oct 2011 15:25:51 GMT

The problem: OpenVMS Linker returns error message

when linking a Fortran program having a large data array:

%LINK-E-INSVIRMEM, insufficient virtual memory for 2110331. pages
for cluster DEFAULT_CLUSTER

Platform: XP1000, 2GB core memory, OpenVMS 7.3,

Fortran 7.1-1 (does not produce any warning or error messages)

pagefile.sys >4,000,000 pagelets, wsmax=3,000,000,

process wsext=3,000,000

Process Quotas:
Account name: ALLUSERS
CPU limit:                       Infinite   Direct I/O limit:        150
Buffered I/O byte count quota:     99808   Buffered I/O limit:     150
Timer queue entry quota:                   10   Open file quota:        100
Paging file quota:                 2996720   Subprocess quota:    10
Default page fault cluster:          64   AST quota:                 248
Enqueue quota:                        2000   Shared file limit:            0
Max detached processes:               0 Max active jobs:         0

Is it the result of 1GB P0 space? How to get out of 1GB limit?

One solution is to split the problem into several parallel processes,

but it is quite undesirable when debugging a new algorithm.

It will be splitted later to run on MPP machines. And now I just

want to use 1.5 GB of my 2 GB core memory.

I have CDs to upgrade to 7.3-1 or to 8.3 hobbyist. Shall it help?

Thanks.

Re: Program size 1GB limit

Hoff — Sun, 02 Oct 2011 14:16:36 GMT

Ok, some basics of computing and of OpenVMS follow.

VMS is a 32-bit virtual memory system, with a segmented addressing model that allows applications to access 64-bit address space on OpenVMS Alpha V7.0 and later, and on OpenVMS I64. It's sign-extended addressing. The 32-bit portion of the available address space is the default, and is further divided into four hunks, P0, P1, S0 and S2 address space; each of one gigabyte.

32-bit applications have access to one gigabyte by default, and part of a second gigabyte with some extra coding.

64-bit addressing requires additional source code changes, or compiler support, and variously both. Fortran is among the compilers with 64-bit addressing support on OpenVMS.

Here is some of the older 64-bit addressing documentation. Go read that, as a start. (IIRC, that documentation has been integrated into the OpenVMS Programming Concepts manual in newer releases. Check there for the latest.)

The process quotas are not particularly related to the use of virtual addressing (nor physical addressing) though you will want to look at the locality of the array references (the access patterns and stride) to avoid heavy page-faulting activity, and at the WSMAX system parameter (the maximum size of the process working set), and I'd also consider moving to private or global sections (Fortran does pointers) and away from data structures declared inline. Put another way, becoming somewhat more familiar with the implementation of virtual memory and paging can avoid severe performance problems, beyond the use of that knowledge when addressing more than the classic 32-bit address space.

A software upgrade is your call. I'd move to OpenVMS V8.3 and a more current Fortran compiler, minimally. Messing around with this stuff isn't worth the effort. I'd probably start looking to upgrade the hardware, too; to an Alpha or Itanium or x86-64 box with at least four gigabytes of physical, and preferably more.

I haven't checked to see of gfortran does 64-bit addressing on a flat-addressed 64-bit system, but that might be worth a look, if you have access to flat-addressed 64-bit addressing that can be available on other operating system platforms. (I'd be very surprised if the available Fortran compilers on gcc and llvm were not capable of 64-bit addressing.)

Re: Program size 1GB limit

alphacat — Fri, 14 Oct 2011 15:27:16 GMT

Thank you for the prompt answer and the attention to my request.

Yes, I agree that another technique qf making data structures could help. Among system services I found something like xxxx_64.

However this would require complete re-designing the whole program of about 100 modules and 30000 source code lines because it is based on our own high-level dynamic memory distribution technique. The only thing I would like is to move the whole common//a(1000...00) to p2 address space. It would be nice if there were platform-dependent language extensions allowing us to indicate the location ovf variables, either in p0 or p2 space. This does not contadict to portability concept because any real implementation of a programming language has a number of platform-dependent features this or that way.

The work-around in my case is to truncate the data set which I use to test the code at the development stage. I have 5 Alphas on 3 sites (including home) and consider OpenVMS as the best environment for program development because of DCL, logical names, error reporting, easy graphics, EDT, and many more nice things. Later the programs are run on Linux-driven MPP machines where debugging is much more difficult because of NFS synchronization problems etc.

Re: Program size 1GB limit

Mike Kier — Mon, 03 Oct 2011 18:28:21 GMT

Have you tried to move some of your larger statically or dynamically allocated structures to 64-bit space?

There are compiler directives to do this. You just have to be careful if they are referenced in any 32-bit system service call, etc.

Here's an example from one of my codes:

!DEC$   Attributes ADDRESS64 ::    Rec_Array
        Record /DePers_MEMMST_str/ Rec_Array(:)
        Target Rec_Array
        Allocatable Rec_Array

FORTRAN

Statements

    Directives

      General_Directives

        ATTRIBUTES

          ADDRESS64 (VMS,WNT)

             Specifies that the data object has a 64-bit address. This property
             can be specified for any variable or dummy argument, including
             ALLOCATABLE and deferred-shape arrays. However, variables with
             this property cannot be data-initialized.

             It can also be specified for COMMON blocks or for variables in a
             COMMON block. If specified for a COMMON block variable, the COMMON
             block implicitly has the ADDRESS64 property.

             ADDRESS64 is not compatible with the AUTOMATIC attribute.

Re: Program size 1GB limit

alphacat — Sun, 09 Oct 2011 19:54:34 GMT

Thanks to All

Hope this will help after upgrade to the latest Fortran version.

Correction to mistype in my previous 2 messages: not SMP, but MPP machines, of course,

which allow splitting of large data across numerous MPP nodes.

Thanks once again

Re: Program size 1GB limit

Dennis Handly — Mon, 10 Oct 2011 02:44:06 GMT

>Correction to mistype in my previous 2 messages:

(You can go back and edit your posts. Under post Options, see Edit Reply.)