PA-8600 memory limitation for FEA application?

[Kirk MacLean_2]

Is there an addressable memory limitation on a PA-8600 RISC-processor running at 552MHz available for an application?

An engineer has tried to run two different FEA Finite Element Analysis programs which require more than 4GB of RAM to solve particular problems. On lesser problems the application(s) perform as expected. The machine has 8GB installed but fails to run the application(s). The applications fail due to apparent memory limitation.

The software vendor can successfully run this job on a Sun workstation with 8GB of RAM.

The OS is lli and kernel parameters are correct according to the guide:
"Optimizing, Configuring and Tuning HP-UX 11 for CAE Applications
Revision 5.0 August, 2004 Edited by: John Cowles, Optimization Technology Manager Hewlett-Packard Company"

From a data sheet for the hp visualize c3600 UNIX workstation:

Feature: Maximum main memory up to 2GB Synchronous DRAM
WHAT DOES THIS MEAN?

Benefit: Supplies higher application performance with less disk access; supports interactive work with large 3D models
Advantage: Delivers the largest RAM capacity available on any workstation today

IN ANOTHER AREA OF THE SAME SHEET:
c3600 technical specifications: main memory
Bus bandwidth 2GB/sec
RAM type 120MHz SDRAM
Capacity 512MB-8GB
Memory slots 8


Thank you for your help!

Kirk

[Steven E. Protter]

Shalom Kirk,

On athe 32 bit OS, and you have not indicated your OS, just hinted, there is a concept called memory windows.

This enables your applications to address memory in blocks larger than 2 GB.

It would be helpful to know the following:

as root user
uname -a
model number of the system
swapinfo -tam
Amount of memory in the system.

I also recommend getting the system fully up to date on patches.

SEP

[Jaime Bolanos Rojas.]

Kirk,

Your OS version is extremely important in this question, remember that in 11.11 and before all applications in a system are limited to a total of 1.75GB of share memory.

Memory windows as stated above might be your solution.

Regards,

Jaime.

[Patrick Wallek]

Actually that is not quite true.

It is not that 11.11 and prior are limited to 4GB, it is more that 32-bit applications are limited to 4GB.

As of 11.0, with the advent of the 64-bit OS, large RAM access is possible IF the applications are compiled as 64-bit apps.

[Kirk MacLean_2]

SEP, Jamie & Patrick,

Thank you for your reply; I look forward to awarding points to each of you!

Here is some additional information about this system:

HP-UX caewks04 B.11.11 U 9000/785 2015455360 unlimited-user license

NOTE: I have watched swap while the program tan and it never seemed to be a problem.
# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 2048 0 2048 0% 0 - 1 /dev/vg00/lvol2
dev 2048 0 2048 0% 0 - 1 /dev/vg01/lvol1
reserve - 1081 -1081
total 4096 1081 3015 26% - 0 -

Memory Information:
physical page size = 4096 bytes, logical page size = 4096 bytes
Physical: 8388608 Kbytes, lockable: 7705772 Kbytes, available: 6527700 Kbytes

# file /stand/vmunix
/stand/vmunix: ELF-64 executable object file - PA-RISC 2.0 (LP64)


The application is 64 bit:
# file solver64
solver64: ELF-64 executable object file - PA-RISC 2.0 (LP64)
===========================
Thank you again for helping me solve this!

Kirk

[Jaime Bolanos Rojas.]

Hi Kirk!

All looks like it should be working just fine for you, but it does not.

Your app is 64, the OS supports 64 bit applications, you got 6GB of free memory.

What version of the Finite Element Analysis are you using by the way?

Jaime.

[A. Clay Stephenson · Accepted solution]

In your case, the fundamental problem is not enough swap space. You have 8GiB of physical memory installed but only 4GiB of swap AND pseudoswap is not enabled. This means that you are only allowed a maximum of 4GiB of total process space. The system cannopt reserve space bewyond that limit. You have two alternatives: 1) define more swap space 2) enable pseudoswap (set swapmem_on=1 and rebuild the kernel). Pseudoswap allows the system to pretend that 75% of your memory counts as process reservation space so that now your total virtual address space would be .75 x 8Gib (pseudoswap) + 4Gib (device swap) or 10GiB. Pseudoswap is a way of telling the system that yes, I know I really don't have enough swap but I don't ever expect to need it -- that's why I bought that memory in the first place.

One other common mistake is to set maxssiz (and maxssiz_64bit much too large since the stack and data segment share a common quadrant. If maxssiz_64bit is greater than 64MiB, I would reduce it. Maxssiz needs to be no bigger than 32MiB --- and these limits are extremely generous. Only very poorly written software ever needs stacks any larger than this.

[Kirk MacLean_2]

Jaime,

The applications are the latest releases available from these software companies and should run on HP-UX 11.11. One of the companies is dropping support for HP-UX; I am guessing this is because HP chose to get out of the workstation market.

Thank you,

Kirk

[Kirk MacLean_2]

Clay,

Regarding the swap issue, you didn't mention file system swap as an option. Neither of the two disks had unallocated space but the 36 GB disk's only file system volume had 80% free, so I tried adding file system swap:

# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 2048 0 2048 0% 0 - 1 /dev/vg00/lvol2
dev 2048 0 2048 0% 0 - 1 /dev/vg01/lvol1
localfs 3942 0 3942 0% none 0 1 /1a/paging
reserve - 473 -473
total 8038 473 7565 6% - 0 -

Had the engineer try the test part again but will not know the results till Wednesday. So far the results are encouraging because the memory use is up to 5 GiB. This didn't require a reboot which was a preference at this time.

The second hint you graciously provided was on the kernels maxssize values. Please confirm that I understand you suggestion correctly, below I have included the current kernel value on top with what I am interpreting as your suggested value under:

Maxssiz 100610048 current size
33554432 equals 32 MiB

maxssiz_64bit 268435456 current size
67108864 equals 64 MiB

And Clay, I have to admit my ignorance and was educated by pondering your reference to GiB; wikepedia had a good reference to this with links to standards bodies. This terminology is not new; I saw references back several years.

Greatly appreciate your help!

Kirk

[A. Clay Stephenson]

Yes, your math is quite correct and filesystem swap is fine (because you probably won't be using it anyway). If you do actually begin to page out to a significant degree, the performance hit from swapping of any kind is so bad that I always say that worrying about swap layout is akin to rearranging the deck chairs on the Titanic.

I would reduce your maxssiz_xxxxx values to something very near 32MiB and 64Mib. As I said earlier, only extremely poorly written software ever needs a stack any larger.

I have disciplined myself over the last year or so to use the Ki, Mi, and Gi prefixes when I mean 2^10, 2^20, and 2^30 respectively and to use the standard SI prefixes K, M, and G when I mean 10^3, 10^6, and 10^9 respectively. What I really hate are what I refer to as "marketing gigabytes" in which 2^30 (1GiB) becomes
~1.074 GB so that a disk or memory appears larger.