100% real memory usage: how to find process

[Bill McNAMARA_1]

Hi,
this is what my system looks like this morning:

# swapinfo -tm
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 1024 0 1024 0% 0 - 1 /dev/vg00/lvol2
reserve - 69 -69
memory 322 10 312 3%
total 1346 79 1267 6% - 0 -
#


Great!

But recently, I have been seeing
memory 322 322 0 100%

quite often.
and hence getting plock and mlock errors, apps failing to start and so on.

from top I couldn't find any major memory user. (netscape was largest - and normal)

How can I best find memory looser apps?

Later,
Bill

[Steve Steel]

Hi


Glance is always best but this will give a guide.



/bin/rm /tmp/$PPID 2>/dev/null
UNIX95= ps -e -o ruser,vsz,pid,args > /tmp/$PPID
head -n 1 /tmp/$PPID
tail -n +2 /tmp/$PPID| sort -rnk2
/bin/rm /tmp/$PPID 2>/dev/null

Steve Steel

[Paula J Frazer-Campbell]

Bill

Glance is the tool you need.

If you do not have it install the 60 trial from the core/apps cd set.


Paula

[Deepak Extross]

'top' is another quick tool to indicate the processes that are hogging memory. However, since the output is sorted by CPU usage and not process size, you may have to scroll down a few pages to find the culprit.

[Kenny Chau]

I am not sure it will work or not. You can use ipcs to get information from memory and use ipcrm to remove them. If still can't remove them, reboot is the last step to release memory.

Hope this helps.
Kenny.

[Animesh Chakraborty]

Hi Bill,
Have you tried lsof ?

[Bill McNAMARA_1]

have lsof installed, but forgot to use it...
trying to reproduce now..

How do I use the ipcs.. utils.

Later,
Bill

[Kenny Chau]

I am not sure it will work.

Try to shutdown all the applications in the server and ask all users logout. Only left the root or one user. Then use "ipcs -mo" to display information. Then use ipcrm to remove the process that lock the memory.

Or refer to this link for information:

http://us-support2.external.hp.com/cki/bin/doc.pl/sid=8b98bf4415d2965507/screen=ckiDisplayDocument?docId=200000030335258

Hope this helps.
Kenny.

[Paula J Frazer-Campbell]

Bill

ipcs -moa

OWNER The login name of the owner of the facility entry.
GROUP The group name of the group of the owner of the facility entry.
CREATOR The login name of the creator of the facility entry.
CGROUP The group name of the group of the creator of the facility entry.
CBYTES The number of bytes in messages currently outstanding on the associated message queue.
QNUM The number of messages currently outstanding on the associated message queue.
QBYTES The maximum number of bytes allowed in messages outstanding on the associated message queue.
LSPID The process ID of the last process to send a message to the associated message queue.
LRPID The process ID of the last process to receive a message from the associated message queue.
STIME The time the last msgsnd() message was sent to the associated message queue.
RTIME The time the last msgrcv() message was received from the associated message queue.
CTIME The time when the associated facility entry was created or changed.
NATTCH The number of processes attached to the associated shared memory segment.
SEGSZ The size of the associated shared memory segment.
CPID The process ID of the creating process of the shared memory segment.
LPID The process ID of the last process to attach or detach the shared memory segment.
ATIME The time the last shmat() attach was completed to the associated shared memory segment.
DTIME The time the last shmdt() detach was completed on the associated shared memory segment.
NSEMS The number of semaphores in the set associated with the semaphore entry.
OTIME The time the last semop() semaphore operation was completed on the set associated with the semaphore entry.


HTH

Paula

[Paula J Frazer-Campbell]

Bill

To find oversized user processes, use "ps -el" and note the SZ column.
It gives the process size in units of 4k.

That should help identify processes with big memory consumption. You could then look at their details with glance.

Also look for shared memory usage "ipcs -moa", where SEGSZ is the size in bytes. The owner of the shared memory gives typically hints for the usage.

Paula

[Steve Lewis]

The SZ column in 'ps -el' (l=little L) shows the memory used by all processes.
If there are in-house applications I suggest you use Purify. It is excellent at tracking down memory leaks in applications.
There have been instances of drivers and apps leaking and hogging memory, such as fibrechannel interfaces, which consume vast amounts of RAM to support required throughput.

[Roger Baptiste · Accepted solution]

Using ipcs ->

#ipcs -am >/tmp/x
#vi /tmp/x and remove first three lines of header info

#cat /tmp/x | awk '{ SUM=SUM+$10} END {printf "%d\n ", SUM/1024}'


HTH
raj

[Bill McNAMARA_1]

Kenny's link for those in Europe!

http://europe-support2.external.hp.com/cki/bin/doc.pl/sid=206dcc080568990c2b/screen=ckiDisplayDocument?docId=200000030335258

Later,
Bill