Resource Utilization is very high

Bunti — Tue, 20 Dec 2011 05:48:06 GMT

Hello,

We have one production box which is UP n running from last 910days.

Output of glance is as below --

Most of the resources are being used by one of the application called medpap

any alternative how can we reduce load on this system?

B3692A GlancePlus C.03.86.00 01:03:38 vfauh427 9000/800 Current Avg High

----------------------------------------------------------------------------------------------------

Cpu Util S SN NU U | 99% 99% 99%

Disk Util F F | 96% 91% 100%

Mem Util S SU UB B | 93% 93% 93%

Swap Util U UR R | 64% 61% 64%

----------------------------------------------------------------------------------------------------

PROCESS LIST Users= 5

User CPU Util **bleep** Disk Thd

Process Name PID PPID Pri Name (1000% max) CPU IO Rate RSS Cnt

--------------------------------------------------------------------------------

arm_CV 18223 18146 253 medPap 67.2/45.9 4.7 0.0/ 0.5 6.2mb 1

arm_CV 18233 18166 253 medPap 57.6/38.6 4.0 0.0/ 0.1 7.4mb 1

arm_CV 18395 18350 253 medPap 54.2/54.2 2.8 0.3/ 0.3 6.2mb 1

arm_CV 16625 16564 253 medPap 52.5/50.6 12.9 29.4/26.7 6.3mb 1

arm_CV 16852 16802 253 medPap 50.4/47.9 12.2 53.8/39.6 6.1mb 1

arm_CV 17510 17467 253 medPap 50.0/54.7 8.6 20.9/21.0 6.1mb 1

arm_CV 17896 17856 253 medPap 48.1/52.2 5.4 32.1/31.1 6.0mb 1

oracleMED05 1069 1 149 medP3db 41.2/28.7 64.5 1581/ 762 22.8mb 1

arm_CV 18019 17968 253 medPap 33.6/43.0 4.4 0.0/ 0.3 7.5mb 1

ksh 28410 4682 158 medPap 33.2/34.8 3800.1 0.0/ 0.1 392.2mb 1

ksh 28402 4682 158 medPap 23.7/26.0 2834.1 0.0/ 0.1 372.8mb 1

oracleMED05 16094 1 239 medP3db 22.9/26.4 9.5 251/ 280 22.9mb 1

oracleMED05 6545 1 239 medP3db 22.5/28.9 48.8 274/ 418 22.8mb 1

arm_CV 18536 18488 223 medPap 22.3/22.3 1.2 1.3/ 1.3 7.5mb 1

ftm 4808 1 154 medPap 17.2/17.8 1943.2 24.4/16.4 21.6mb 1

arm_fr 18232 18166 154 medPap 13.5/ 8.8 0.9 17.1/10.9 2.3mb 1

ftcd 4682 1 239 medPap 12.6/14.8 1611.1 0.0/ 0.0 11.4mb 1

arm_fr 18222 18146 154 medPap 11.6/ 8.0 0.8 15.1/10.0 2.4mb 1

arm_fr 18394 18350 154 medPap 9.4/ 9.4 0.5 11.3/11.3 2.9mb 1

sqlplus 18284 28410 149 medPap 7.4/ 7.3 0.4 0.5/ 0.5 3.4mb 1

arm_fr 18018 17968 218 medPap 6.7/ 9.2 1.0 8.4/11.5 2.4mb 1

arm_fr 16624 16564 240 medPap 5.9/ 5.6 1.4 7.3/ 6.9 3.7mb 1

vfauh427[/home/yg09958]$ more /etc/passwd|grep medPap

medPap:*:6130:2130:Mediation application owner:/med/p31100/comptel/medPap:/usr/bin/ksh

vfauh427[/home/yg09958]$ more /etc/passwd| grep medP3db

medP3db:*:7701:2889:Oracle Mediation database owner:/usr/users/medP3db:/usr/bin/ksh

vfauh427[/home/yg09958]$ ps -eaf|grep arm_CV

medPap 432 384 254 00:59:38 ? 0:04 /med/p31100/comptel/medPap/amd/arm/bin/arm_CV -e ICC3 -i 182935

medPap 520 472 255 00:59:38 ? 0:04 /med/p31100/comptel/medPap/amd/arm/bin/arm_CV -e ICC1 -i 182935

yg09958 799 21624 0 00:59:43 pts/8 0:00 grep arm_CV

medPap 349 296 255 00:59:37 ? 0:04 /med/p31100/comptel/medPap/amd/arm/bin/arm_CV -e ICC0 -i 182935

medPap 375 327 252 00:59:37 ? 0:05 /med/p31100/comptel/medPap/amd/arm/bin/arm_CV -e ICC_ADJ1 -i 18

medPap 29211 29177 253 00:59:19 ? 0:19 /med/p31100/comptel/medPap/amd/arm/bin/arm_CV -e ICC2 -i 182935

vfauh427[/home/yg09958]$

vfauh427[/home/yg09958]$ ps -eaf|grep oracleMED05P

yg09958 4484 21624 0 01:00:24 pts/8 0:00 grep oracleMED05P

medPap 4757 1 98 Oct 2 ? 1536:47 oracleMED05P (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))

medP3db 19745 1 0 Nov 15 ? 1:14 oracleMED05P (LOCAL=NO)

medPap 4700 1 0 Oct 2 ? 7:10 oracleMED05P (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))

medP3db 3187 1 0 14:01:51 ? 0:38 oracleMED05P (LOCAL=NO)

medP3db 19986 1 254 00:55:31 ? 1:36 oracleMED05P (LOCAL=NO)

medP3db 26599 1 253 00:58:12 ? 0:42 oracleMED05P (LOCAL=NO)

medP3db 24274 1 0 14:08:10 ? 0:29 oracleMED05P (LOCAL=NO)

medP3db 23570 1 0 16:29:18 ? 0:04 oracleMED05P (LOCAL=NO)

daemon 2278 1869 254 01:00:05 ? 0:03 oracleMED05P (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))

medP3db 18214 1 252 00:55:10 ? 1:41 oracleMED05P (LOCAL=NO)

medP3db 1069 1 254 00:59:53 ? 0:07 oracleMED05P (LOCAL=NO)

medP3db 6416 1 0 Nov 14 ? 1:19 oracleMED05P (LOCAL=NO)

vfauh427[/home/yg09958]$

Re: Resource Utilization is very high

Dennis Handly — Tue, 20 Dec 2011 06:39:42 GMT

>any alternative how can we reduce load on this system?

Why? What kind of complaints are you getting?

Replace the system by more and faster CPUs/memory. Port to an Integrity server.

Reduce the number of medPap's processes.

Re: Resource Utilization is very high

Prakash Pitroda — Wed, 21 Dec 2011 12:22:55 GMT

Get the high Cpu, Memory utilization processes & informed users accordinglly.

For list 5 Highest CPU utilized process.

UNIX95= ps -eo user,pid,pcpu,comm | sort -nrbk3 | head -5

For list 5 highest Memory utilized process.

UNIX95= ps -eo user,pid,vsz,comm | sort -nrbk3 | head -5

Re: Resource Utilization is very high

Bill Hassell — Fri, 23 Dec 2011 08:26:01 GMT

You have used all your CPUs and memory, and your disks are at their limit, based on the snapshot from Glance.

The simple answer is to stop running the medPap application. Then your resources will return to normal.

I assume that this is not an option. You need a lot more processors, and a lot more memory, and probably a faster set of disks. Your machine has reached its limit. I know that sysadmins are often asked to fix overloaded machines with some sort of "go-faster" button but without a major redesign of the application, or reducing the application load, you'll need a much bigger machine.

topic Re: Resource Utilization is very high in Operating System - HP-UX

Resource Utilization is very high

Re: Resource Utilization is very high

Re: Resource Utilization is very high

Re: Resource Utilization is very high