Memory leak issue

[Atul Goel]

Attached graph shows the memory data of my binary run for few days. Data is collected using glance. Can any inference be drawn from the data. That is wether there is a memory leak, corruption or fragmentation

[Atul Goel]

Wrong file attached in last post.
Attaching the excel file in zip format

[Kapil Jha]

I am not sure what field are there exactly in the XL sheet.
What exactly is the problem are you running out of memory.
What does glance -m says do you have lots of page fault.
Any other observation.

BR,
Kapil

[Steven E. Protter]

Shalom,

Try the memory leak detector for HP-UX.

http://www.hpux.ws/?p=8

SEP

[Atul Goel]

Hi Kapil,
Actually problem is as observed from the top the Resident memory size of my process is increasing. But,i ncrease is random some time it is stable for 2 hrs some time more then that. I have checked the business logic many time, even tried wdb but unable to find any memory related issue. I am not sure wether actually there is a problem in my code or it is HP related problem? Whether it is memory leak, corruption of memory fragmentation. From the data first i want to be sure what actually is the problem then would like to go for fixing it

The sheet i have attached has the VSS, RSS, DATA VSS, DATA RSS field as observed from glance at interval of 5 minute

[Kapil Jha]

Have u tried the link given above, it would help u.
I have couple of more questions.
Is this graph always going UP, does Resident memory (CODE+DATA) coming down anytime or going up.
If its coming down sometime to its real level then i would say that there is no memory leak and if thats going up and up and up.
It would suggest memory leak.

The graph you shared can lead to anlysis that there is leak but it we willl have to see broad picture over long period of time.

Since when u facing the issue.
Any changes in server??

BR,
Kapil+

[Dennis Handly]

If you have heap fragmentation you should look at the following thread:
http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=1351858

Calling mallopt(3) and adding the values will show you if the amount of free space increase but used space is the same.

>tried wdb but unable to find any memory related issue.

If WDB doesn't report any leaks, it's fragmentation.
What HP-UX version are you using? You may be able to use MallocNextGen.

[Dennis Handly]

>ME: If you have heap fragmentation you should look at the following thread:

Hmm, that was your thread, it would have been helpful to mention that. Is this the same problem?

[Atul Goel]

"Is this graph always going UP, does Resident memory (CODE+DATA) coming down anytime or going up."
>>>>>>>>>>Yes graph is going up


The graph you shared can lead to anlysis that there is leak but it we willl have to see broad picture over long period of time.
>>>>The graph is of data collected over 11 days of run

Any changes in server??
>>>No changes in the server


What HP-UX version are you using?
>>>B.11.23

You may be able to use MallocNextGen
>>>mallocnextgen cannot be used

[Dennis Handly]

>mallocnextgen cannot be used

It would be helpful if you could explain why?

Also, you could install it as an experiment to see if it even solves your problem. This may explain what kind of fragmentation you don't have.

[Atul Goel]

Firstly my application is single threaded. mallocnextgen is useful for multi threaded application
Secondly mallocnextgen is for itanium server but my build is made (or is should say cross compiled )on a pa-risc system where i cannot install this mallocnextgen

Does anybody had a quick look on the data i attached? I would request you to have a quick look. File is not of much size and only a look will give an idea of memory issue to experts.

[Dennis Handly]

>Firstly my application is single threaded. mallocnextgen is useful for multi threaded application

(Last time you said "Yes, it is a multi threaded application."?)

Do you know it won't help? The specs don't say it is only useful for threads.

If it is single threaded, you are allocating memory in a non-optimal manner. And from your other thread, you may need to special case your allocators for various block sizes, 64028, 120 4944, 40, 48, 12, 28, 64.

>where I cannot install this mallocnextgen

You simply install it on your Integrity box and then copy the shlib to your PA box.

[Atul Goel]

"If it is single threaded, you are allocating memory in a non-optimal manner. And from your other thread, you may need to special case your allocators for various block sizes, 64028, 120 4944, 40, 48, 12, 28, 64."

>>>Can you kindly explain this in a bit more detail.

[Dennis Handly]

>>you are allocating memory in a non-optimal manner. you may need to special case your allocators for various block sizes, 64028, 120 4944, 40, 48, 12, 28, 64."

>Can you kindly explain this in a bit more detail?

If you are allocating memory with random sizes and continually malloc and freeing this space, you will eventually fragment the heap.
Assuming the above block sizes, which have lots of allocations, if you get N blocks of 64028 and then you free N - M. You would end up with M blocks of 64028 and N - M freed blocks of 64028.

Suppose you then want to allocate blocks of 4944, those freed larger blocks would be used. Pretty soon those large blocks are broken up into smaller blocks, that can't be used for 64028 requests, so the arena grows.

What you could do is make a special allocator that only serves up 64028 byte blocks and when they are freed, it keeps them on a linked list for later use. That way these 64028 blocks won't get fragmented.

But this means you have to take up the burden of managing these blocks. Can you do a better job, if you know what your application does?

But the simplest way to solve this to see if MallocNextGen helps.