Problems allocating "large" arrays in OpenVMS-Fortran

[Jansen_8]

Hi all,

I'm trying to use large arrays in a Fortran program, but it fails giving: %FOR-F-INSVIRMEM, insufficient virtual memory!

PLEASE, can someone point me in the right direction to tune my system in order to avoid the above error.

I'm using : HP Fortran V8.2-104954-50N8G on OpenVMS IA64 V8.4

The code I tried:

program test
!
  integer ( kind = 1 ) , allocatable :: size_array( : , : , : )
  integer ( kind = 4 ) , parameter :: dimens = 1000
  integer ( kind = 4 ) :: gbs
!
  write( * , * ) 'number of GB :'
  read ( * , * ) gbs
  allocate( size_array( gbs * dimens , dimens , dimens ) )
  size_array=4
  read( *, *)
!
end


Compiling running it like:
valeta-jj) f90 test
valeta-jj) link test
valeta-jj) run test
number of GB :
1

valeta-jj) run test
number of GB :
2
%FOR-F-INSVIRMEM, insufficient virtual memory!/  unit !SL  file !AS!/ user PC !
XL
%TRACE-F-TRACEBACK, symbolic stack dump follows
image     module    routine               line      rel PC           abs PC
DEC$FORRTL                                   0 00000000000CFA40 FFFFFFFF84DDFA40
test  TEST  TEST                             9 00000000000002C2 00000000000102C2
                                             0 FFFFFFFF80A190D2 FFFFFFFF80A190D2
DCL                                          0 000000000007D072 000000007AE45072
%TRACE-I-END, end of TRACE stack dump



So if gbs=1 everything is OK, but with gbs=2 it already crashes.

The machine has 64GB RAM and enough paging space which is hardly ever used:
valeta-jj) sh mem
              System Memory Resources on 28-OCT-2013 10:25:14.11

Physical Memory Usage (pages):     Total        Free      In Use    Modified
  Main Memory (63.99GB)          8388560     8195291      192775   494

Extended File Cache  (Time of last reset: 28-OCT-2013 08:30:33.56)
 Allocated (MBytes)            144.65    Maximum size (MBytes) 32767.81
 Free (MBytes)                   0.00    Minimum size (MBytes)     3.12
 In use (MBytes)               144.64    Percentage Read I/Os       96%
 Read hit rate                     81%   Write hit rate        0%
 Read I/O count                 55125    Write I/O count     1942
 Read hit count                 44669    Write hit count        0
 Reads bypassing cache             51    Writes bypassing cache        1
 Files cached open                682    Files cached closed      611
 Vols in Full XFC mode              0    Vols in VIOC Compatible mode       27
 Vols in No Caching mode            0    Vols in Perm. No Caching mode        0

Granularity Hint Regions (pages):   Total        Free      In Use Released
  Execlet code region                4096         597        3499      0
  Execlet data region                2048        1143         905      0
  S0S1 Executive data region         6845           0        6845      0
  S0S1 Resident image code region    8192        5055        3137      0
  S0S1 Resident image data region     512         347         165      0

Slot Usage (slots):                Total        Free    Resident     Swapped
  Process Entry Slots                826         783          43     0
  Balance Set Slots                  824         783          41     0

Dynamic Memory Usage:              Total        Free      In Use     Largest
  Nonpaged Dynamic Memory (MB)     53.17       42.88       10.29 42.60
  Bus Addressable Memory  (KB)    128.00      110.87       17.12 104.00
  Paged Dynamic Memory    (MB)     11.68        5.60        6.07  5.60
  Lock Manager Dyn Memory (MB)      5.88        2.43        3.44

Buffer Object Usage (pages):                  In Use        Peak
  32-bit System Space Windows (S0/S1)              1           1
  64-bit System Space Windows (S2)                 0           0
  Physical pages locked by buffer objects          1           1

Memory Reservations (pages):       Group    Reserved      In Use        Type
  Total (0 bytes reserved)                         0           0

Swap File Usage (8KB pages):                   Index        Free        Size
  DISK$VALETA_29757:[SYSEXE]SWAPFILE.SYS;2
                                                   1        6600  6600

Paging File Usage (8KB pages):                 Index        Free        Size
  DISK$VALETA108:[sysexe]PAGEFILE.SYS;3
                                                 253     4999992 4999992
  DISK$VALETA_29757:[SYSEXE]PAGEFILE.SYS;1
                                                 254     3143423 3143424

  Total size of all paging files: 8143416
  Total committed paging file usage: 17873

Of the physical pages in use, 157170 pages are permanently allocated to OpenVMS.


I already tried to increase my process quota:
UAF> sh joukj

Username: JOUKJ                            Owner:  JOUK JANSEN
Account:  JOUKJ                            UIC:    [300,300] ([JOUKJ])
CLI:      DCL                              Tables: BOLERO$DKA0:[JOUKJ.COM]JJJ.EX
E
Default:  SYS$SYSDEVICE:[JOUKJ.COM]
LGICMD:
Flags:
Primary days:   Mon Tue Wed Thu Fri
Secondary days:                     Sat Sun
No access restrictions
Expiration:            (none)    Pwdminimum:  6   Login Fails:     0
Pwdlifetime:           (none)    Pwdchange:  19-SEP-2011 16:47
Last Login: 28-OCT-2013 10:08 (interactive), 28-OCT-2013 08:52 (non-interactive)
Maxjobs:         0  Fillm:      5000  Bytlm:      2100000
Maxacctjobs:     0  Shrfillm:      0  Pbytlm:           0
Maxdetach:       0  BIOlm:     10000  JTquota:      60000
Prclm:          40  DIOlm:      1024  WSdef:        48000
Prio:            4  ASTlm:      1200  WSquo:       200000
Queprio:         0  TQElm:       400  WSextent:   1000000
CPU:        (none)  Enqlm:      4000  Pgflquo:   72000000


I tried to play with some sysgen-parameters but I did not see any change
SYSGEN>  SHOW WS*
Parameter Name            Current    Default     Min.       Max.   Unit  Dynamic
--------------            -------    -------   -------    -------  ----  -------
WSMAX                     4669536     131072     16384  134217728 Pagelets
 internal value            291846       8192      1024    8388608 Pages
WSINC                      187780       2400         0         -1 Pagelets   D
 internal value             11737        150         0         -1 Pages      D
WSDEC                       93890       4000         0         -1 Pagelets   D
 internal value              5869        250         0         -1 Pages      D
SYSGEN>  SHOW PQL_M*
Parameter Name            Current    Default     Min.       Max.   Unit  Dynamic
--------------            -------    -------   -------    -------  ----  -------
PQL_MASTLM                    100          4         4         -1 Ast      D
PQL_MBIOLM                    100          4         4         -1 I/O      D
PQL_MBYTLM                 128000     128000    128000         -1 Bytes      D
PQL_MCPULM                      0          0         0         -1 10Ms      D
PQL_MDIOLM                    100          4         4         -1 I/O      D
PQL_MFILLM                    100          2         2         -1 Files      D
PQL_MPGFLQUOTA           72000000     512000    512000         -1 Pagelets   D
 internal value           4500000      32000     32000         -1 Pages      D
PQL_MPRCLM                     10          0         0         -1 Processes  D
PQL_MTQELM                      0          0         0         -1 Timers     D
PQL_MWSDEFAULT             291937      16384     16384         -1 Pagelets
 internal value             18247       1024      1024         -1 Pages
PQL_MWSQUOTA               583874      32768     32768         -1 Pagelets   D
 internal value             36493       2048      2048         -1 Pages      D
PQL_MWSEXTENT             4669536      65536     65536         -1 Pagelets   D
 internal value            291846       4096      4096         -1 Pages      D
PQL_MENQLM                    300         64        64         -1 Locks      D
PQL_MJTQUOTA                    0          0         0         -1 Bytes      D



I'm lost, so any help is welcome

                  Regards
                      Jouk

[Hein van den Heuvel]

While OpenVMS is a 64-bit operating system, (All) OpenVMS languages and processes start out in 32 bit mode with 'only' 1 GB of address space (0x3fff.ffff) available for program code and data without jumping though hoops.

See for example: http://h71000.www7.hp.com/doc/82final/5841/5841pro_035.html

Fortan might not be the handiest language to deal with this. Is C an option?

Please check the HP Fortran Userguide and read up on anything and everything releated to:

cDEC$ ATTRIBUTE ADDRESS64

ADDRESS64 Property

Also check the Fortan release notes

And maybe the (old) OpenVMS Alpha Guide to 64-Bit Addressing and VLM Features.

http://h71000.www7.hp.com/doc/72final/6467/6467pro.html

Of all the parameters and quotas you list "Pgflquo:   72000000" is most critical.

The system may also need a large page file when actually using such large array.

Come back with detailed questions as needed after processing all this.

Met vriendelijke sport groetjes,

Hein

[Jansen_8]

C is not an option without porting several 10000 of lines written by others.

I tried the DEC-Attribute as was also suggested in comp.os.vms. That works for AXP. On IA64 however this seems not to work with allocvatable arrays resulting in:

%FOR-F-INVREALLOC, allocatable array is already allocated
%TRACE-F-TRACEBACK, symbolic stack dump follows
image     module    routine               line      rel PC           abs PC
DEC$FORRTL                                   0 00000000000CFB30 FFFFFFFF82DDFB30
test  TEST  TEST                            11 0000000000000352 0000000000010352
                                             0 FFFFFFFF80A190D2 FFFFFFFF80A190D2
DCL                                          0 000000000007D072 000000007AE45072
%TRACE-I-END, end of TRACE stack dump

[abrsvc]

Using the compile/lnk commands as below, I get no errors other than the INSVIRMEM when selecting 2 for GB on an I64 system using fortran version: HP Fortran V8.2-104939

OpenVMS V8.4


Commands: $fort T
$link T
$run T
$ number of GB:
2
{error mesage here}


Dan

[Jansen_8]

Dan,

did you insert the line

!DEC$ ATTRIBUTE ADDRESS64 :: size_array

?

         Jouk

[abrsvc]

When I insert the !DEC, I too get the allocation message.

I haven't had much time to play with this, but I will and report what I find.

Dan

[H.Becker]

>>> I'm trying to use large arrays in a Fortran program

Can you use/did you try a static array? On Alpha and I64 the FORTRAN compiler should generate a (demand zero) PSECT with the ALLOC_64BIT attribute. The linker will create a demand-zero image section/segment for that, preserving the 64bit-ness.  (For older VMS/Alpha systems you need the special linker64.) The image activator will map that image section/segment into P2 space.

[Jansen_8]

Static arrays works, but is not an option in the application I'm working on.

An alternative is using DEC-pointers and LIB$GET_VM_64. This works for 1-dimensional arrays.

Presently I have no ideas how to map a multi-dimensional array on this 1-dimensional strip of memory, without everytime calculating the location from the indices and the dimensions.

                          Regards

                                  Jouk

[Dennis Handly]

>I have no idea how to map a multi-dimensional array on this 1-dimensional strip of memory, without every time calculating the location from the indices and the dimensions.

This is exactly what a compiler must do.

[Volker Halle]

The test program fails on the call to DFOR$ALLOC_ALLOCATABLE64 with %FOR-F-INVREALLOC, allocatable array is already allocated

The same code works o.k. on OpenVMS Alpha (may fail with INSVIRMEM, if PGFLQUOTA is too low).

Cross-reference: https://groups.google.com/forum/?hl=de#!topic/comp.os.vms/6pmLU_ncbfw

Volker.