long running process & more memory available

[Vipin Singhal]

Hi Friends,
I have two issues:
1) how could I identify what is that long running process on server?
We may have to kill it after the investigation...

Memory: 190752K (154364K) real, 1082400K (832500K) virtual, 11272628K free Page# 1/19

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND

4 pts/1 22288 jetscan 231 20 4388K 68K run 52233:24 100.08 99.90 sh


2. We are experiencing on out of memory condition on server. I think there should be more memory available there than the 11 GB showing above…

Please suggest what need to do?
Thanks
Vipin

[Geoff Wild]

ps -ef |grep pts/1

to see what is all coming from that user...

For memory - download and compile (with cc - even native one will work) the attached c program called memdetail.

Output like:

# memdetail
Memory Stat total used avail %used
physical 14240.0 14184.9 55.1 100%
active virtual 15570.4 8765.4 6805.0 56%
active real 11761.4 6748.1 5013.3 57%
memory swap 11139.4 1700.9 9438.6 15%
device swap 26528.0 14516.8 12011.2 55%


Rgds...Geoff

[Vipin Singhal]

this is the output:

# ps -ef | grep pts/1
jetscan 22288 1 255 Oct 27 pts/1 52281:28 -sh
root 13675 13510 1 11:56:53 pts/8 0:00 grep pts/1

# man memdetail
No manual entry for memdetail.

# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4096 129 3967 3% 0 - 1 /dev/vg00/lvol2
dev 20480 219 20261 1% 0 - 1 /dev/vg00/swap2
reserve - 386 -386
memory 16351 4841 11510 30%
total 40927 5575 35352 14% - 0 -

Please suggest

[James R. Ferguson · Accepted solution]

Hi Vipin:

It looks like you have an old shell session from October 27 started by "jetscan" originally, but owned by 'init' (pid=1) probably because it was abnormally disconnected. You can kill it and this should reduce your CPU consumption.

Regards!

...JRF...

[Christine Hartman]

Hi Vipin,

Do you have glance installed? That is good tool to view/diagnose things as well.

Also you can do a ps -fp 22288 and that will show you that process' PPID.

For your memory issues....you can check some of your system kernel params that could be taking up more memory than you expect:

dbc_max_pct = set this to something low like 5..the default is 50
also
dbc_min_pct = again set this to something low like 2

[Vipin Singhal]

I have glance and this is the output of glance:

GlancePlus C.03.85.00 12:16:03 grouse ia64 Current Avg High
-----------------------------------------------------------------------------------------------------------------------------
CPU Util SSUU | 4% 4% 4%
Disk Util F FV V | 11% 53% 100%
Mem Util S SUUB B | 31% 31% 31%
Swap Util U UR | 14% 14% 14%
-----------------------------------------------------------------------------------------------------------------------------
PROCESS LIST Users= 11
User CPU Util Cum Disk Thd
Process Name PID PPID Pri Name ( 800% max) CPU IO Rate RSS Cnt
--------------------------------------------------------------------------------
perl 14476 14449 149 jetscan 4.2/ 4.2 0.2 0.4/ 0.4 2.6mb 1
perl 14473 14449 149 jetscan 4.2/ 4.2 0.2 0.6/ 0.6 1.8mb 1
perl 14474 14449 149 jetscan 3.8/ 3.8 0.2 0.4/ 0.4 3.3mb 1
perl 14471 14449 180 jetscan 3.8/ 3.8 0.2 0.0/ 0.0 1.8mb 1
perl 14477 14449 149 jetscan 3.8/ 3.8 0.2 0.8/ 0.8 2.8mb 1
perl 14479 14449 154 jetscan 3.6/ 3.6 0.2 0.2/ 0.2 3.3mb 1
cmcd 39 0 152 root 3.6/ 0.0 4379.1 0.0/ 0.0 128kb 2
vxfsd 57 0 134 root 0.2/ 2.4 305701 5.3/ 1.6 24.9mb 45
scopeux 2129 1 127 root 0.2/ 0.0 3324.6 9.5/ 0.7 9.2mb 1
sshd 14452 1403 133 root 0.0/ 0.0 0.0 2.2/ 3.5 676kb 1
swapper 0 0 127 root 0.0/ 0.0 314.4 1.3/ 0.3 64kb 1
naviagent 4754 1 154 root 0.0/ 0.0 140.0 5.5/ 0.0 1.1mb 1
midaemon 2103 1 -16 root 0.0/ 0.2 28556.6 0.0/ 0.0 38.2mb 2
sshd 14454 1403 128 root 0.0/ 0.0 0.0 1.1/ 1.7 668kb 1


and these are the values of kcweb:
dbc_max_pct 30 30 yes
dbc_min_pct 5 5 yes

[James R. Ferguson]

Hi (again) Vipin:

Glance is your best friend since it "understands" the HP kernel metrics the best.

I don't understand your concern with memory here. You seem to be utilizing only about 30% overall and have good swap space. 'glance' report memory page-outs which is your indiction of memory pressure. I'll bet that you don't have any.

Running a 30% buffer cache maxiumum is probably overly generous given your physical memory, but if it works for you, then it works. Depending on what you are doing, this may benefit you. I'd probably reduce that to no more than 15%.

If you are seeing ENOMEM errors (errno=12; not enough core) you might look at the data stack needs of your processes. The kernel's 'maxdsiz' and 'maxdsiz_64' parameters control this.

Regards!

...JRF...

[Vipin Singhal]

these are the values of kcweb:
dbc_max_pct 30 30 yes
dbc_min_pct 5 5 yes

[Christine Hartman]

Vipin,

are you using SAN attached storage on your system? If so, you should set your dbc_max_pct to something low like 5...that way buffer cache will only use 5% of system memory instead of 30%. Also check the bufpages and nbuf..if they are both set to 0 then you are using dynamic buffer caching. Most SAN attached storage has its own cache...there's no need to have your system do it for you.

You can get more info on these kernel params in SAM...go to the help menu...it should explain it in there. There are probably white papers out there for this too??

[Christine Hartman]

let me clarify....there are "some" instances where you would want your dbc_max_pct to be larger....I believe it's gonna depend on the type/volume of I/O activity on the system. I have yet to work on a system that does benefit from this though. There may be other out there that have experienced an increased benefit from making this setting higher.

[Vipin Singhal]

Thanks