Possible Memory Leak??

[Con O'Kelly]

Hi All

Have a problem with L-3000 11.00 server with 12GB of memory.

After a re-boot the server starts to use increasing amounts of memory. Starts off with about 8GB free and within an hour it's all used and the system starts swapping.

I can't identify the process(es) responsible.
Tried sorting processes by memory usage using Glance and also UNIX95= ps -el command but both don't show any obvious process(es) that are utilising almost 8GB of memory within an hour.

Also process data from Measureware and nothing jumps out. All the additional memory usage is user memory. Buffer cahce is constant around 300-500MB.

The only change was an install of JFS 3.3 last night.

I have sdone multiple re-boots and every time the same thing happens.

Any ideas what could be causing this?
Thanks

Cheers
Con

[Steven E. Protter]

Usually its an application that leaks memory.

You should be able to use glance to identify the process that has growing memory use. Maybe even it will show up on top.

I'd take the following steps:

1) Take a look at any oracle apps and see if there are memory patches
2) Make sure the system itself is up to date on patches. Maybe there is one for Online JFS.

SEP

[Sridhar Bhaskarla]

Hi Con,

If there is any increase in memory usage due to JFS upgrade, I would expect it to be 'system memory' due to vx-inode cache.

Since the increase is in user memory, I would suggest you to look any other changes. Run the following commands periodically and see if you see any increase in the first column for any processes over the time. If so, then those processes may have memory leaks.

UNIX95= ps -e -o 'vsz pid args' |sort -n

-Sri

[bhavin asokan]

hi,

almost same problem was with me.
after i gave the sybase group the privilage of MLOCK it started working fine.

i gave like this.
#setprivgrp sybase12 MLOCK

as per our DBA for oracle also we have to do this.

regds,

[Con O'Kelly]

Hi

Thanks for the replies.

Sri thats exactly the command I was using to try and identify the leak but it didn't show anything. No process was increasing memory usage even though user memory was increasing all the time.

Bhavin - that sounds an interesting idea as we are running Sybase - I'll look into that as that could be the issue.

I'll assign points tomorrow.

Thanks

Cheers
Con

[Ivajlo Yanakiev]

I have same problem.
Patches fix problem.
Apply last patch bundle

[Sridhar Bhaskarla]

Couple more things...

1. Are you sure you have the same number of processes everytime you run that command?.

2. How about 'ipcs -ma'. Do you see massive shared memory segments in it that were not there before?.

3. Can you post the last lines of your glance's 'm' screen?. It shows how the memory is distributed..

MLOCK will

[Sridhar Bhaskarla]

Excuse my impatience to work with the windows. I did something causing it to post the earlier message prematurely.

MLOCK capability will make that particular group to 'lock' memory pages so that they won't get paged out in case of memory pressure. It may improve your application's performance but will not reduce the memory usage.

Patching is always a good idea.

-Sri

PS: 0 points for this post please.

[Eric Antunes]

Con,

I've installed JFS 3.3 on my test system and seem to have no problem. Did you applied patch PHKL_30927 after upgrading to JFS 3.3??

Best Regards,

Eric Antunes

[Con O'Kelly]

Hi

Thanks for the help.

Through trial & error we have identified the application that appears to be causing the issue.
When the app is running, the memory leak occurs. From glance we can see Total Memory utilisation increasing with a corresponding increase in User Memory utilisation. Once the application is shutdown memory utilisation stabilises.

The strange thing is that the app's processes do not show any increase in their memory use. The 3 processes only use about 4MB of memory. I have used glance and the UNIX95= ps -e.... command and memory usage stays the same.
The app is a third party security product not a database.
Shared memory also stays constant.

Any ideas on how we can see what process(es) is using this memory?

Thanks.

Cheers
Con