- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Re: Problems allocating "large" arrays in OpenVMS-...
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-28-2013 03:50 AM
10-28-2013 03:50 AM
Problems allocating "large" arrays in OpenVMS-Fortran
Hi all,
I'm trying to use large arrays in a Fortran program, but it fails giving: %FOR-F-INSVIRMEM, insufficient virtual memory!
PLEASE, can someone point me in the right direction to tune my system in order to avoid the above error.
I'm using : HP Fortran V8.2-104954-50N8G on OpenVMS IA64 V8.4
The code I tried:
program test
!
integer ( kind = 1 ) , allocatable :: size_array( : , : , : )
integer ( kind = 4 ) , parameter :: dimens = 1000
integer ( kind = 4 ) :: gbs
!
write( * , * ) 'number of GB :'
read ( * , * ) gbs
allocate( size_array( gbs * dimens , dimens , dimens ) )
size_array=4
read( *, *)
!
end
Compiling running it like:
valeta-jj) f90 test
valeta-jj) link test
valeta-jj) run test
number of GB :
1
valeta-jj) run test
number of GB :
2
%FOR-F-INSVIRMEM, insufficient virtual memory!/ unit !SL file !AS!/ user PC !
XL
%TRACE-F-TRACEBACK, symbolic stack dump follows
image module routine line rel PC abs PC
DEC$FORRTL 0 00000000000CFA40 FFFFFFFF84DDFA40
test TEST TEST 9 00000000000002C2 00000000000102C2
0 FFFFFFFF80A190D2 FFFFFFFF80A190D2
DCL 0 000000000007D072 000000007AE45072
%TRACE-I-END, end of TRACE stack dump
So if gbs=1 everything is OK, but with gbs=2 it already crashes.
The machine has 64GB RAM and enough paging space which is hardly ever used:
valeta-jj) sh mem
System Memory Resources on 28-OCT-2013 10:25:14.11
Physical Memory Usage (pages): Total Free In Use Modified
Main Memory (63.99GB) 8388560 8195291 192775 494
Extended File Cache (Time of last reset: 28-OCT-2013 08:30:33.56)
Allocated (MBytes) 144.65 Maximum size (MBytes) 32767.81
Free (MBytes) 0.00 Minimum size (MBytes) 3.12
In use (MBytes) 144.64 Percentage Read I/Os 96%
Read hit rate 81% Write hit rate 0%
Read I/O count 55125 Write I/O count 1942
Read hit count 44669 Write hit count 0
Reads bypassing cache 51 Writes bypassing cache 1
Files cached open 682 Files cached closed 611
Vols in Full XFC mode 0 Vols in VIOC Compatible mode 27
Vols in No Caching mode 0 Vols in Perm. No Caching mode 0
Granularity Hint Regions (pages): Total Free In Use Released
Execlet code region 4096 597 3499 0
Execlet data region 2048 1143 905 0
S0S1 Executive data region 6845 0 6845 0
S0S1 Resident image code region 8192 5055 3137 0
S0S1 Resident image data region 512 347 165 0
Slot Usage (slots): Total Free Resident Swapped
Process Entry Slots 826 783 43 0
Balance Set Slots 824 783 41 0
Dynamic Memory Usage: Total Free In Use Largest
Nonpaged Dynamic Memory (MB) 53.17 42.88 10.29 42.60
Bus Addressable Memory (KB) 128.00 110.87 17.12 104.00
Paged Dynamic Memory (MB) 11.68 5.60 6.07 5.60
Lock Manager Dyn Memory (MB) 5.88 2.43 3.44
Buffer Object Usage (pages): In Use Peak
32-bit System Space Windows (S0/S1) 1 1
64-bit System Space Windows (S2) 0 0
Physical pages locked by buffer objects 1 1
Memory Reservations (pages): Group Reserved In Use Type
Total (0 bytes reserved) 0 0
Swap File Usage (8KB pages): Index Free Size
DISK$VALETA_29757:[SYSEXE]SWAPFILE.SYS;2
1 6600 6600
Paging File Usage (8KB pages): Index Free Size
DISK$VALETA108:[sysexe]PAGEFILE.SYS;3
253 4999992 4999992
DISK$VALETA_29757:[SYSEXE]PAGEFILE.SYS;1
254 3143423 3143424
Total size of all paging files: 8143416
Total committed paging file usage: 17873
Of the physical pages in use, 157170 pages are permanently allocated to OpenVMS.
I already tried to increase my process quota:
UAF> sh joukj
Username: JOUKJ Owner: JOUK JANSEN
Account: JOUKJ UIC: [300,300] ([JOUKJ])
CLI: DCL Tables: BOLERO$DKA0:[JOUKJ.COM]JJJ.EX
E
Default: SYS$SYSDEVICE:[JOUKJ.COM]
LGICMD:
Flags:
Primary days: Mon Tue Wed Thu Fri
Secondary days: Sat Sun
No access restrictions
Expiration: (none) Pwdminimum: 6 Login Fails: 0
Pwdlifetime: (none) Pwdchange: 19-SEP-2011 16:47
Last Login: 28-OCT-2013 10:08 (interactive), 28-OCT-2013 08:52 (non-interactive)
Maxjobs: 0 Fillm: 5000 Bytlm: 2100000
Maxacctjobs: 0 Shrfillm: 0 Pbytlm: 0
Maxdetach: 0 BIOlm: 10000 JTquota: 60000
Prclm: 40 DIOlm: 1024 WSdef: 48000
Prio: 4 ASTlm: 1200 WSquo: 200000
Queprio: 0 TQElm: 400 WSextent: 1000000
CPU: (none) Enqlm: 4000 Pgflquo: 72000000
I tried to play with some sysgen-parameters but I did not see any change
SYSGEN> SHOW WS*
Parameter Name Current Default Min. Max. Unit Dynamic
-------------- ------- ------- ------- ------- ---- -------
WSMAX 4669536 131072 16384 134217728 Pagelets
internal value 291846 8192 1024 8388608 Pages
WSINC 187780 2400 0 -1 Pagelets D
internal value 11737 150 0 -1 Pages D
WSDEC 93890 4000 0 -1 Pagelets D
internal value 5869 250 0 -1 Pages D
SYSGEN> SHOW PQL_M*
Parameter Name Current Default Min. Max. Unit Dynamic
-------------- ------- ------- ------- ------- ---- -------
PQL_MASTLM 100 4 4 -1 Ast D
PQL_MBIOLM 100 4 4 -1 I/O D
PQL_MBYTLM 128000 128000 128000 -1 Bytes D
PQL_MCPULM 0 0 0 -1 10Ms D
PQL_MDIOLM 100 4 4 -1 I/O D
PQL_MFILLM 100 2 2 -1 Files D
PQL_MPGFLQUOTA 72000000 512000 512000 -1 Pagelets D
internal value 4500000 32000 32000 -1 Pages D
PQL_MPRCLM 10 0 0 -1 Processes D
PQL_MTQELM 0 0 0 -1 Timers D
PQL_MWSDEFAULT 291937 16384 16384 -1 Pagelets
internal value 18247 1024 1024 -1 Pages
PQL_MWSQUOTA 583874 32768 32768 -1 Pagelets D
internal value 36493 2048 2048 -1 Pages D
PQL_MWSEXTENT 4669536 65536 65536 -1 Pagelets D
internal value 291846 4096 4096 -1 Pages D
PQL_MENQLM 300 64 64 -1 Locks D
PQL_MJTQUOTA 0 0 0 -1 Bytes D
I'm lost, so any help is welcome
Regards
Jouk
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-28-2013 05:29 AM - edited 10-28-2013 05:34 AM
10-28-2013 05:29 AM - edited 10-28-2013 05:34 AM
Re: Problems allocating "large" arrays in OpenVMS-Fortran
While OpenVMS is a 64-bit operating system, (All) OpenVMS languages and processes start out in 32 bit mode with 'only' 1 GB of address space (0x3fff.ffff) available for program code and data without jumping though hoops.
See for example: http://h71000.www7.hp.com/doc/82final/5841/5841pro_035.html
Fortan might not be the handiest language to deal with this. Is C an option?
Please check the HP Fortran Userguide and read up on anything and everything releated to:
cDEC$ ATTRIBUTE ADDRESS64
ADDRESS64 Property
Also check the Fortan release notes
And maybe the (old) OpenVMS Alpha Guide to 64-Bit Addressing and VLM Features.
http://h71000.www7.hp.com/doc/72final/6467/6467pro.html
Of all the parameters and quotas you list "Pgflquo: 72000000" is most critical.
The system may also need a large page file when actually using such large array.
Come back with detailed questions as needed after processing all this.
Met vriendelijke sport groetjes,
Hein
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-28-2013 06:33 AM
10-28-2013 06:33 AM
Re: Problems allocating "large" arrays in OpenVMS-Fortran
C is not an option without porting several 10000 of lines written by others.
I tried the DEC-Attribute as was also suggested in comp.os.vms. That works for AXP. On IA64 however this seems not to work with allocvatable arrays resulting in:
%FOR-F-INVREALLOC, allocatable array is already allocated
%TRACE-F-TRACEBACK, symbolic stack dump follows
image module routine line rel PC abs PC
DEC$FORRTL 0 00000000000CFB30 FFFFFFFF82DDFB30
test TEST TEST 11 0000000000000352 0000000000010352
0 FFFFFFFF80A190D2 FFFFFFFF80A190D2
DCL 0 000000000007D072 000000007AE45072
%TRACE-I-END, end of TRACE stack dump
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-28-2013 07:02 AM
10-28-2013 07:02 AM
Re: Problems allocating "large" arrays in OpenVMS-Fortran
OpenVMS V8.4
Commands: $fort T
$link T
$run T
$ number of GB:
2
{error mesage here}
Dan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-28-2013 07:10 AM
10-28-2013 07:10 AM
Re: Problems allocating "large" arrays in OpenVMS-Fortran
Dan,
did you insert the line
!DEC$ ATTRIBUTE ADDRESS64 :: size_array
?
Jouk
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-28-2013 07:55 AM
10-28-2013 07:55 AM
Re: Problems allocating "large" arrays in OpenVMS-Fortran
I haven't had much time to play with this, but I will and report what I find.
Dan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-29-2013 09:37 AM
10-29-2013 09:37 AM
Re: Problems allocating "large" arrays in OpenVMS-Fortran
>>> I'm trying to use large arrays in a Fortran program
Can you use/did you try a static array? On Alpha and I64 the FORTRAN compiler should generate a (demand zero) PSECT with the ALLOC_64BIT attribute. The linker will create a demand-zero image section/segment for that, preserving the 64bit-ness. (For older VMS/Alpha systems you need the special linker64.) The image activator will map that image section/segment into P2 space.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-30-2013 01:48 AM
10-30-2013 01:48 AM
Re: Problems allocating "large" arrays in OpenVMS-Fortran
Static arrays works, but is not an option in the application I'm working on.
An alternative is using DEC-pointers and LIB$GET_VM_64. This works for 1-dimensional arrays.
Presently I have no ideas how to map a multi-dimensional array on this 1-dimensional strip of memory, without everytime calculating the location from the indices and the dimensions.
Regards
Jouk
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-30-2013 02:27 AM
10-30-2013 02:27 AM
Re: Problems allocating "large" arrays in OpenVMS-Fortran
>I have no idea how to map a multi-dimensional array on this 1-dimensional strip of memory, without every time calculating the location from the indices and the dimensions.
This is exactly what a compiler must do.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-30-2013 02:57 AM
10-30-2013 02:57 AM
Re: Problems allocating "large" arrays in OpenVMS-Fortran
The test program fails on the call to DFOR$ALLOC_ALLOCATABLE64 with %FOR-F-INVREALLOC, allocatable array is already allocated
The same code works o.k. on OpenVMS Alpha (may fail with INSVIRMEM, if PGFLQUOTA is too low).
Cross-reference: https://groups.google.com/forum/?hl=de#!topic/comp.os.vms/6pmLU_ncbfw
Volker.