Vitual memory increasing

[Satish Kumar Malisetti]

All,

in my application i used start a process it occupies virtual memory and while processing the files with the same PID it is increasing the virtual memroy size

ex:

i have started process x occupied memory 1 GB
under that i have processed multiple files it is increasing the virtual memory 1.04 GB , i have verifed the open handles before and after processing files is same and the oracle session is same

can any body suggest me to how to know the appended memory ?

[James R. Ferguson · Accepted solution]

Hi:

First, a process that 'malloc()'s memory and then subsequently 'free()'s that memory doesn't see the freed memory returned to the operating system until the process terminates. Instead, the freed memory is returned to the process's "heap" for reuse.

This is by design to reduce the overhead of memory allocation and deallocation and to reduce memory fragmentation.

I'm not sure the 4% increase you see is significant. Memory leaks and heap fragmentation will lead to growth in a process's size.

See this thread, with particular regard to the 'mallinfo(3)' and 'wdb':

http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=1351858

Regards!

...JRF...

[Michael Steele_2]

Hi

To find a memory leak you need a script executed in cron every 20 minutes or so. This command is the meat of the script.

UNIX95=1 ps -ef -o vsz,pid,ppid,state,wchan,args | sort -rn | head 10 >> outfile

You are finding biggest 'vsz' consumers and by taking a report every 20 minutes you are observing any growth.

[Bill Hassell]

You can't fix program x problems with HP-UX changes. The process is growing because of how it was designed. Whether that growth is correct or not requires information from the programmer. As mentioned, a 4% growth is insignificant. When the program grows to 1.5 to 2 GB, you definitely need to contact the programmer and get the code fixed (if necessary). After all, the programmer may tell you that the program is correctly designed to grow to 50 GB under some circumstances.

[Dennis Handly]

>can anybody suggest me to how to know the appended memory?

As JRF suggested, you can look at my replies in that thread.
Ideally, for every file you process, you should come back to some constant state. But if your files are larger and need more heap, you would keep growing.
But if you processed that "large" file first, it should grow to that value right away. (I'm not sure if that experiment is easy to do for you?)
Except if you have heap fragmentation, then the small requests split up the big blocks so nobody can use them.

>Bill: You can't fix program x problems with HP-UX changes.

Some times you can with something like MallocNextGen. Especially with heap fragmentation, it really isn't the program's fault.

>the programmer may tell you that the program is correctly designed to grow

In this case, the program must be remembering some state for each file.

[Satish Kumar Malisetti]

I have tried the with malloc env varibles as posted in above link

export _M_ARENA_OPTS=1:2024
export _M_SBA_OPTS=65536:50:256

but still my virtula memory is increasing after processing file 7MB per file and

i can't test it based on huge file first and small file later coz i can't expect the upstream file sizes

Any suggessions please

[Satish Kumar Malisetti]

Bill ,

I am from the development team itself ,
as per the design virual memory should not increase , if it reaches around 8GB then component/particular process automatically go down

[Dennis Handly]

>but still my virtual memory is increasing after processing file 7MB per file

Then you need to first assume you have a memory leak and use gdb to find them.

>as per the design virtual memory should not increase

Then if you don't have leaks, you have heap fragmentation. Have you tried MallocNextGen?

[Satish Kumar Malisetti]

can i have any links how to use
mallocnextgen and gdb please

[Dennis Handly]

>can I have any links how to use mallocnextgen and gdb

Did you look at all the URLs in thread JRF listed above?