HP-UX large memory process limits/kernel config

[Joseph Hartvigsen]

I posted this in the workstation forum, but perhaps this is the better audience.

I'd like to be able to run large memory processes, but am currently limited to something just under 1Gb. I assume that some kernel parameter configuration will solve this but when I pick parameters with values of ~1Gb and try to increase them SAM won't let me.

I am running a B2600 with 1280 Mb of physical memory and 2.5Gb swap space.

OS is the full install of HPUX11i-TCOE

I've tried both f90 programs, simple and complex as well as matlab. All
fail when total process memory demand is just under 1Gb.

Below is a simple FORTRAN process to illustrate.

integer n
parameter (n=121600191)
real*8 a(n)
common // a
a(1)=1
a(n)=n
write(6,*)a(1),a(n),a(n)/a(1)
stop
end

The results of the size command on the compiled program is as follows
(perhaps mangled by this posting window width).

size -v mem

Subspace Size Physical Address Virtual Address
$SHLIB_INFO$ 2299 8192 4096
$MILLICODE$ 72 10492 6396
$CODE$ 5296 10568 6472
$LIT$ 240 15864 11768
$UNWIND_START$ 896 16104 12008
$UNWIND_END$ 144 17000 12904
$RECOVER_END$ 4 17144 13048
$PFA_COUNTER$ 8 20480 1073745920
$DATA$ 300 20544 1073745984
$SHORTDATA$ 16 20848 1073746288
$PLT$ 88 20864 1073746304
$DLT$ 16 20952 1073746392
$GLOBAL$ 16 20968 1073746408
$BSS$ 972801576 0 1073746432

Total 972810971

The program runs as expected.
mem
1.0 121600191.0 121600191.

If I increase the value of n from 12160019121
to 12160019122 the program fails.

size -v mem

Subspace Size Physical Address Virtual Address
$SHLIB_INFO$ 2299 8192 4096
$MILLICODE$ 72 10492 6396
$CODE$ 5296 10568 6472
$LIT$ 240 15864 11768
$UNWIND_START$ 896 16104 12008
$UNWIND_END$ 144 17000 12904
$RECOVER_END$ 4 17144 13048
$PFA_COUNTER$ 8 20480 1073745920
$DATA$ 300 20544 1073745984
$SHORTDATA$ 16 20848 1073746288
$PLT$ 88 20864 1073746304
$DLT$ 16 20952 1073746392
$GLOBAL$ 16 20968 1073746408
$BSS$ 972801584 0 1073746432

Total 972810979


mem
*** FORTRAN I/O ERROR 913: OUT OF FREE SPACE
Bus error

At a somewhat larger value it still compiles fine
but won't even load n=122600192

mem
exec(2): could not load a.out

Since up to 4Gb of physical memory is possible in this machine, and
more in higher end systems it seems that HPUX11i would support
processes with more than 1Gb memory.

What is required to run a multi Gb process?

Thanks,

Joe

[Patrick Wallek]

The key question here is are you running a 32-bit program or a 64-bit program?

If you are running a 32-bit program and it hasn't been compiled with appropriate flags to allow more, then you will only be able to use approx. 1 GB of RAM. That is due to 32-bit shared memory limits.

[Joseph Hartvigsen]

Patrick,
That is a good question. The answer in the case of matlab according to their tech support is that matlab is 32 bit. As far as programs created using the f90 compiler - I don't know. I can't see from the man page what is requred to make a 64 bit application. In the SGI IRIX compilers you just add a -64 flag and get a 64 bit executable.

From what I've read even a 32 bit OS should be able to run a process close to 2Gb.

Thanks,

Joe

[Joseph Hartvigsen]

I forgot to list the compile listing. I had thought that the warning about a PA 2.0 object indicated a 64 bit application.

f90 -o mem mem.f
mem.f
main program

9 Lines Compiled
/usr/ccs/bin/ld: (Warning) At least one PA 2.0 object file (mem.o) was detected. The linked output may not run on a PA 1.x system.

[Patrick Wallek]

If you have an 11.0 system have a look at the memory management white paper in /usr/share/doc/mem_mgt.txt

If you are on 11i, and don't have these, then look here:
http://docs.hp.com/cgi-bin/otsearch/getfile?id=/hpux/onlinedocs/os/11i/mem_mgt.html&searchterms=5965-4641&queryid=20030128-153709

[Patrick Wallek]

Nope, PA-RISC 2.0 does not necessarily mean that it is a 64 bit executable. If you do a 'file programname' it should say something about being a 64-bit if it is in fact a 64-bir program.

[Joseph Hartvigsen]

Wow, that white paper on 11i memory will take a while to wade through.

This is what the file command tells me about my test program and matlab.

file mem
mem: PA-RISC2.0 shared executable dynamically linked -not stripped

file matlab
matlab: PA-RISC2.0 shared executable dynamically linked

Apparently both are 32bit.

Does anyone have experience making 64 bit compiles with f90?

Thanks,

Joe

[Mike Stroyan]

If you are linking your own 32-bit program you can get about 1GB of additional data space by linking with a -Wl,-N option to cc, aCC or f90 or a -N option to ld. That starts the data area at a lower address.
You can also increase the available data area by using "chatr +q3p enable a.out". That will change a program to use the third quadrant of its address space for private data instead of shared memory. It could cause trouble for some programs that need to map in shared memory segments that were created by other programs in the third quadrant.
(You can do this to already linked programs. You need write permission on the program file to make this change. The program cannot be running when you try to change the file.)
The "chatr +q4p enable a.out" option can be used to make the fourth quadrant hold private data. That is extreme, interfering with use of shared memory.

[Joseph Hartvigsen]

Mike,
What is the chatr command to get the same effect as linking a program with the -N flag? I assume the shared memory is used for inter-process communication, so I'd rather not mess with that but I would like to go from the 950Mb limit to the 1750Mb limit on a vendor supplied code.

Thanks,

Joe

[Mike Stroyan]

Chatr cannot change a program between -N and not. The -N option moves a program's global data addresses from a base address of 0x40000000 to a base address just above the programs code. That requires relocation of data accesses. Chatr does not have enough information to do that.

[Joseph Hartvigsen]

Mike,

After reading the process management white paper section on EXEC_MAGIC I searched for EXEC_MAGIC in the chatr man page and found a -N option. When I tried to apply this option to the program of interest (matlab), chatr returned an error. I've also tried this on the simple FORTRAN program with similar result.

chatr(error): only SHMEM_MAGIC files can be made EXEC_MAGIC

I also tried chatr +q3p enable matlab , which was sucessful. However when I ran matlab it quit with a "Fatal Java Exception". However, matlab has a flag to disable Java -nojvm and using this flag matlab ran and was able to use more than the 1Gb limit.

chatr +q4p enable matlab also seems to work. The executable loads much faster as explained in the white paper.

Thanks to all who have responded to my 3 threads on this issue. It brings back memories of 20 years ago with the 8088 segmented address space.

Joe

[Paddy_1]

If thats really the 32/64 bit question then compile it with the option
+DAportable
This is at the cost of performance.If you know your target machine is RISC1.1 then add the switch
+DS1.1
Try compiling with these options.

[Mike Stroyan]

"chatr(error): only SHMEM_MAGIC files can be made EXEC_MAGIC"

chatr -N and chatr -M are an odd pair.
If you link with -N, then you can use "chatr -M a.out" to change from EXEC_MAGIC to SHMEM_MAGIC. That makes quadrant 1 hold text and data, while quadrants 2, 3, and 4 hold shared memory segments.
I don't deal with SHMEM_MAGIC users, but I expect it has been largely obsoleted by memory windows.

The only use for chatr -N is to undo a chatr -M.