Out of memory error Hpux 11i

[Jeff Disney]

All,

I have a 32 bit application running on an L2000 server with 16gb of memory, running Hpux 11i. The application is dying at 890mb of memory consumption. The vendor had me set the masxdsiz value to 2gb, it is set as below:

maxdsiz 0X80000000

After setting the larger value the application still crashes. They sent me a little test application as well that runs in a loop and takes 10mb of system memory on each iteration and it is also dying at 890mb.

How do I determine if I need to make any other kernel changes?

What tools can I used to nail this one down?

[Patrick Wallek]

You are hitting a 32-bit memory limit.

Even though you have 16GB of RAM, a 32-bit program will only be able to use up to 4GB, if it is compiled correctly.

A read through the "HP-UX Memory Management Whitepaper" should explain things:
http://docs.hp.com/en/1218/mem_mgt.html

[A. Clay Stephenson]

You have hit the single 1GiB quadrant limit for 32-bit applications. In order to go past this limit, your software vendor must compile and link the code to allow it exceed these limits. The paper Patrick cites will explain this.

Without changing the application, you will be able to get another 60MiB or so by simply DECREASING maxssiz by 60MiB. The data segment and stack segment are allocated from the same 1GiB quadrant so the data segment is is implicitly reduced by maxssiz even if the run-time stack never approaches maxssiz.

[Bill Hassell · Accepted solution]

As mentioned, an ordinary 32 bit program is limited to less than 1000 megs of RAM. If you have the source code, you can recompile the program use EXEC_MAGIC which maps the first and second quadrants together and your program can now obtain about 1800 megs. With the latest patches, you can take that same program and use chatr to enable quadrant 3 and quadrant 4, which will allow your program to obtain up to 3700 megs of RAM.

There is absolutely no reason for any program to crash when it runs out of RAM -- that's just poor code. And any programmer writing old 32 bit code must be well aware of the severe limitations that these small programs have. I have attached a small program which requests RAM (malloc) until the current addressing model runs out of space. The program can request up to 3700 megs of RAM as a 32 bit program and unlimited amounts of RAM when recompiled as a 64 bit program.

[Jeff Disney]

Thanks to all of you for the help on this.

But that brings up the question of how to determine how they have compiled the code, this is vendor code and I am likely going to have to advise them on how to compile it.

[Bill Hassell]

> This is vendor code and I am likely going to have to advise them on how to compile it.

I hope this code is free. I wouldn't pay a nickel for code that is supposed to run on HP-UX and I had to explain to the programmer how to handle memory. When an application is advertised as being compatible or runs on HP-UX, you would rightfully assume the code was tested on an HP-UX box running your version of HP-UX. If this vendor doesn't have such a platform, you might offer to help port the code and test it on one of your boxes -- for a substantial discount off the software price.

[A. Clay Stephenson]

The easiest way to determine the "bitness" of the code is to execute the file command against the executable.

for example, file myapp
if the display is something like "PA-RISC1.1 shared executable dynamically linked" then it is 32-bit PA-RISC code.

If you see something like "ELF-64 executable ..." then you have 64-bit code.

You can also execute chatr against the executable for more detailed infomation.

Man chatr for details.

In any event, woe be unto the vendor that I had to explain any of this to. In fact, his suggestion that you increase maxdsiz to 2GiB while apparently not knowing that because of his compiler/linker options you will still not exceed 1GiB is a big red flag.

[Jeff Disney]

I agree that I should not have to provide them any help, but it is a very important application that is time sensitive and this issue has been around for a bit and it took me a bit to nail it down, thanks again for all the help from all. I have raised the red flags to my management and will see how it shakes out, my guess is that we will stick with the applications regardless, ahh the joy of being the unix/problem resolution guy.

[ckm_1]

Even I am facing a similar issue of this kind. The process(32 bit) core dumps after reaching 2.xx GB of memory.

I enabled the 3rd quadrant memory to be used by this process by "chatr" command but I dont find the process is consuming 2.85GB of memory which is supposed to be, when 3rdquad mem is enabled.

Why is it so?is it like this memory is not accesible to this process if this one is already allocated to some shared libraries or something like that?

Could some one please pur in your thouhgts?

[Bill Hassell]

> Even I am facing a similar issue of this kind. The process(32 bit) core dumps after reaching 2.xx GB of memory.

The program must be rewritten to properly handle memory requests.

> I enabled the 3rd quadrant memory to be used by this process by "chatr" command but I dont find the process is consuming 2.85GB of memory which is supposed to be, when 3rdquad mem is enabled.

Did you see all the other steps (maxdsiz, ulimit, compile with EXEC_MAGIC). Here is a program that will ask for memory up to 3700 megs when compiled (32 bit mode) using the comments in the code.

[ckm_1]

Yeah it worked after enabling fourth quadrant....and the test application helped...thanks to all