Memory management tools for HP-UX?

[Steve Zwach]

I have a process that seems to be leaking memory on the HP-UX B.11.00 platform but I can't seem to find out where. I have the HP ANSI C++ B3910B A.03.31 compiler. I have tried the same code compiled on Sun and Linux but I don't see the leak on either of these platforms using Valgrind and purify. The problem is only apparant on the HP, and the process will grow to about 1 gig in size then it dies since the system doesn't allow processes to be this large.

Does anybody have any ideas on other memory management tools that could be used to find memory leaks specifically on HP's?

Any help is much appreciated.

[A. Clay Stephenson · Accepted solution]

I assume that you are hitting maxdsiz. My first cut at this would be to install Purify on HP-UX (at least an evaluation copy) and see what happens. I assume that you are doing all your memory allocation either via malloc() or new.

[Sundar_7]

I beleive even glance can be used to find the memory leaks. Install the trail version of glance plus if u dont have one, and select the Memory Regions of the process you are interested in. Consistent increase in the Private Data area means the process is allocating memory but failing to deallocate.

[Albert Smith_1]

I would definitly get the measureware suite installed and a atleast a demo of perfview and monitor over a couple of days.

Also run SAR and VMSTAT and IOSTAT to find your memory and io bottle necks.

Glance will give you real time as well as SAR. Perfview will track the data over time and allow it to be graphed and tracked.

[Paddy_1]

As Other people already mentioned the tools I would like to add a little to it.

Memory leak detection is a 2 step process :

(1)Use /usr/sbin/swap -s to check the swap size and try repeatedly over intervals to see
if available swap space keeps getting smaller

(2) ps -lu shows the size of all your processes.The column headed SZ is the size of the process in pages. (The pagesize command will tell you how big that is in Kbytes if you really must know.) Again, repeat the command several times; any program that dynamically allocates memory can be observed growing in size


Finally I would profile the process using a new tool from HP called prospect which could potentially help you.
http://h21007.www2.hp.com/dspp/tech/tech_TechSoftwareDetailPage_IDX/1,,3282,00.html

[Steve Zwach]

So far I have been using glance and perfview, that is how I have determined that the process is leaking. The problem I am having is finding out where in the code is process leaking. This is hard to find since when I run the same process on a Linux or Sun platform it doesn't grow in size.

Clay, you have suggested installing Purify, we have not been able to get that to run on our HP system. We use a combination of both malloc() and new. We did use Purify on the Sun and it proved to find a couple of small areas where we would leak but that wasn't all. Do you have any tips for getting Purify to work on the HP?

One other option we have thought about is updating the compiler since we are using the oldest compiler that Purify supports. Updating the compiler does have risks though, does anyone have any thoughts if a new compiler would help or hurt?

Thanks again.

[Paddy_1]

Insure++, like Purify, is designed to find memory leaks, corruption, and other memory use errors.

You can try the trial version.
http://www.parasoft.com/jsp/products.jsp?/insure/support/index.htm

If the leak is in the code a compiler cannot help though we can always make sure to customise the flags on the HP-UX.

[A. Clay Stephenson]

I suspect that if you update the compiler, Purify will also run. Since you have Purified your code on Sun, I rather doubt that your problem is in your code per se. I strongly suspect that the problem lies in one of the libraries. It also could lie in one of application libraries (perhaps a database library, if applicable). If I were you I would bring aCC up to the latest version and also make sure that you use the latest aCC run-time libraries (PHSS_26945). This patch fixes a number of run-time memory leaks. One method that I use, is to put breakpoints in the code sections I suspect and then examine the process with Glance to see if the process has grown unexpectedly. You can also put probes in to do a system call invoking ps -p PID to examine the vsz and echo'ing the current location (e.g. "Probe 16").

I'm actually betting that the new run-time library will fix you but I would update to at least the A.03.45 compiler as well. I have very seldom been bitten by updating compilers.

I suspect you really appreciated all the hints at helping you determine if you had a memory leak that you already knew you had. I would be interested in knowing if you are hitting maxdsiz or maxssiz. If it's maxssiz then you just might have a real bug in your code. Of course, if you tell me that you have a 1GB stack, I ain't gonna help you no more on account of that's way too big.

[A. Clay Stephenson]

I would also install PHSS_30048 that fixes a memory leak in dynamic library loads.

[Steve Zwach]

Clay, I believe I am hitting the maxdsiz size that is set at about 1 gig and watching my process through glance I can see when it approaches 1 gig in size that is when it cores. When the process starts back up it has a size about 650 meg which is well within expectations.

[Mike Stroyan]

A 1GB limit is likely to come from the way you link your program. The default linker behavior puts the data region at addresses starting at 1GB and extending to about 2GB. Using a compiler option of -Wl,-N will pass the linker a -N option with starts the data area just above the text area. That will allow almost 2GB of data area as long as you don't reach another limit such as maxdsiz, ulimit or available swap space.

The wdb debugger has a simple memory leak checker in it. You can get the most current wdb from http://www.hp.com/go/wdb .

[Paddy_1]

Actually Clay has a point in saying that the runtime library interaction to be suspect.

IBM has a memory interceptor library thats Free and available for download at :
http://www.alphaworks.ibm.com/tech/mil

-Paddy

[Ted Buis]

This is not a fix, but if you have Process Resource Manager you could limit the amount of real memory that the application gets to use, and keep it from totally impacting other applications. Not really a long term solution, but something that might help in the short term.

[Steve Zwach]

We have updated our compiler to the 3.52 version and the linker version is at 11.38 but Purify still core dumps when we try to run. Are there any extra tricks or something in the setup that we need to do to get Purify to run on this HP machine? I was hoping that the new compiler would make it easier to run but that is not the case, and the requirements for Purify don't even list the new version of the compiler so I am wondering if this is supported at all.

[Peter Hug]

Steve, did you manage to get your software working? If yes, what did you end up doing?

[Steve Zwach]

Here is an update. After we updated to the newest compiler 3.52, then we tried to get purify to work and again it failed. Then we called IBM Rational to get their help and they have not been any help yet! First they said that they don't support that version of the compiler so we went to 3.45 which IBM said is the latest version they support and agin we fail in the same way. Right now after almost five weeks they said they think they have a bug have sent the problem to thier support team in India to look at. Apparantly they said it should be a two week turnaround! Long story short we still don't have the problem fixed.

We did find a way to link the process so that it can grow to over a gig but our client doesn't feel that is a solution.

[Lalit Seth]

Hi Steve,

What all flags and technique to used to link the process so that it doesn't do core dump after crossing 1G restriction.

Many Thanks
Lalit

[David Gourley_1]

Just a couple of comments from my experience in this area.

1) Purify doesn't catch all types of memory "leak"; we had an algorithmic problem where memory was being added to an internal (complex) data structure, and not being removed correctly. When our process exited, the memory structure was cleaned up properly, so Purify didn't detect the problem (as we didn't have a classic leak; we had a logic bomb).

2) We've also encountered scenarios where the algorithms in a process were such that the process could grow to > maxdsize (with the inevitable core dump).

3) Where we've had problems with purify, and to address (1) or (2), we've instrumented our code by logging the size of a process's data segment (found by calling pstat_getproc). This can help pin down exactly where in your code a process is growing (which is the first step to working out why it's growing).