System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

Top Command CPU Idle 0% and NICE value High

 
Nazaf Khan
Frequent Advisor

Top Command CPU Idle 0% and NICE value High

Hi,


We regularly get ESM alers about CPU utilization high in the system and when we try to login and check at that particular time it shows CPU ideal 100% but after puting script for the top command we observed that the CPU utilization goes high with the NICE value as high but who owns this value is not shown in the top and really confused why this happens. It happens in only a minutes and then it comes to normal. Could you pls help in this.


System: yohp15 Thu May 20 13:31:00 2010
Load averages: 3.85, 1.21, 0.72
1051 processes: 1028 sleeping, 22 running, 1 zombie
Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 4.02 0.4% 86.7% 12.8% 0.1% 0.0% 0.0% 0.0% 0.0%
1 3.69 0.7% 86.1% 12.8% 0.4% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 3.85 0.6% 86.4% 12.8% 0.2% 0.0% 0.0% 0.0% 0.0%

Memory: 3125004K (871744K) real, 3858424K (1152240K) virtual, 3338876K free Page# 1/66

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
0 ? 20853 oracle 154 22 29920K 4096K sleep 0:00 31.04 1.51 sqlplus
1 ? 20815 oracle 154 22 30176K 4360K sleep 0:00 15.41 1.47 sqlplus
1 ? 20867 oracle 154 22 29920K 4092K sleep 0:00 30.04 1.46 sqlplus
0 ? 20887 oracle 154 22 30176K 4360K sleep 0:00 30.04 1.46 sqlplus
1 ? 20756 oracle 190 22 29920K 4100K run 0:00 15.15 1.44 sqlplus
0 ? 20817 oracle 190 22 30176K 4360K run 0:00 14.96 1.42 sqlplus
0 ? 15376 oracle 190 22 2756K 264K run 0:00 1.06 1.01 ksh
0 ? 20928 oracle 154 22 29920K 4088K sleep 0:00 0.00 0.83 sqlplus
1 ? 20934 oracle 189 22 29728K 3896K run 0:00 0.00 0.78 sqlplus
0 ? 28017 esm_srv 152 20 44072K 19096K run 10:31 0.66 0.66 agent
1 ? 37 root 152 20 9632K 9632K run 147:09 0.59 0.59 vxfsd
1 ? 2522 root 152 20 220M 25896K run 395:15 0.29 0.29 java
1 ? 1 root 168 20 496K 204K sleep 51:52 0.28 0.28 init
0 ? 10468 root 152 20 25852K 3964K run 0:28 0.21 0.21 adclient
0 ? 2320 root 154 20 4408K 764K sleep 121:20 0.11 0.11 sendmail:
1 ? 2844 root 152 20 25120K 4800K run 8:05 0.10 0.10 vxsvc

System: yohp15 Thu May 20 13:36:30 2010
Load averages: 0.05, 0.64, 0.65
1030 processes: 1013 sleeping, 16 running, 1 zombie
Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 0.05 0.0% 0.0% 0.1% 99.9% 0.0% 0.0% 0.0% 0.0%
1 0.04 0.2% 0.0% 0.0% 99.8% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 0.05 0.1% 0.0% 0.1% 99.8% 0.0% 0.0% 0.0% 0.0%

Memory: 3095532K (811992K) real, 3822244K (1080824K) virtual, 3370468K free Page# 1/65

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
1 ? 37 root 152 20 9632K 9632K run 147:10 0.52 0.52 vxfsd
0 ? 28017 esm_srv 152 20 44072K 19096K run 10:32 0.35 0.35 agent
1 ? 2522 root 152 20 220M 25896K run 395:15 0.27 0.27 java
0 ? 10468 root 152 20 25852K 3964K run 0:28 0.15 0.15 adclient
1 ? 2479 root 152 20 28312K 7348K run 175:44 0.10 0.10 cimserver
1 ? 2844 root 152 20 25120K 4800K run 8:05 0.10 0.10 vxsvc
0 ? 2320 root 154 20 4408K 764K sleep 121:20 0.08 0.08 sendmail:
1 ? 4909 root 152 20 15392K 3016K run 1:35 0.08 0.08 samd
1 ? 10804 oracle 158 22 2756K 268K sleep 68:50 0.07 0.07 ksh
0 ? 25810 oracle 156 20 388M 8240K sleep 7:11 0.06 0.06 ora_mmon_prd445
1 ? 19520 oracle 158 22 2756K 268K sleep 70:09 0.06 0.06 ksh
1 ? 0 root 127 20 32K 0K sleep 9:46 0.04 0.04 swapper
0 ? 572 root 152 20 2240K 456K run 25:19 0.04 0.04 syncer
0 ? 10469 root 152 20 6960K 820K run 0:00 0.04 0.04 cdcwatch
0 ? 26285 oracle 152 20 62160K 6364K run 3:57 0.04 0.04 tnslsnr
0 ? 20 root 147 20 32K 32K sleep 1:56 0.04 0.04 lvmkd
5 REPLIES
Nazaf Khan
Frequent Advisor

Re: Top Command CPU Idle 0% and NICE value High

Also had captured the ouput of Glance too it shows the details as mentioned below.

Cpu Util S SN NU |100% 100% 100%
Disk Util F FV | 20% 27% 42%
Mem Util S SU UB B | 61% 62% 62%
Swap Util U UR R | 25% 25% 25%
------------------------------------------------------------------------------------------------------------------------------------------------------
PROCESS LIST Users= 4
User CPU % Thrd Disk Memory Block
Process Name PID Name ( 200% max) Cnt IOrate RSS/VSS On
--------------------------------------------------------------------------------
sqlplus 21037 oracle 7.3 1 1.8 4.5mb 5.3mb died
sqlplus 21184 oracle 7.3 1 1.3 4.5mb 5.3mb died
sqlplus 21142 oracle 7.3 1 0.0 4.3mb 5.1mb died
sqlplus 21060 oracle 7.3 1 0.0 4.5mb 5.3mb died
sqlplus 21126 oracle 7.1 1 1.3 4.7mb 5.4mb died
sqlplus 21089 oracle 7.1 1 0.0 4.3mb 5.1mb died
sqlplus 21165 oracle 7.1 1 1.8 4.7mb 5.4mb died
sqlplus 21221 oracle 7.1 1 1.3 4.5mb 5.3mb died
sqlplus 21240 oracle 7.1 1 0.0 4.3mb 5.1mb died
sqlplus 21286 oracle 7.1 1 0.0 4.3mb 5.1mb died
sqlplus 21269 oracle 7.1 1 1.3 4.5mb 5.3mb died
sqlplus 21303 oracle 7.1 1 1.3 4.5mb 5.3mb new
sqlplus 21338 oracle 7.1 1 0.4 4.3mb 5.1mb died
sqlplus 21202 oracle 7.1 1 1.3 4.5mb 5.3mb died
sqlplus 21327 oracle 7.1 1 0.4 4.3mb 5.1mb died
sqlplus 21385 oracle 6.9 1 1.3 4.3mb 5.1mb new
sqlplus 21104 oracle 6.9 1 0.0 4.3mb 5.1mb died
sqlplus 21421 oracle 6.9 1 0.0 4.3mb 5.1mb new
sqlplus 21367 oracle 6.9 1 1.3 4.5mb 5.3mb new
sqlplus 21419 oracle 6.7 1 1.3 4.3mb 5.1mb new
sqlplus 21443 oracle 6.7 1 0.0 4.0mb 4.8mb new
sqlplus 21462 oracle 6.5 1 0.0 3.8mb 4.6mb new
sqlplus 21024 oracle 5.8 1 0.0 4.3mb 5.1mb died
sqlplus 21478 oracle 1.3 1 0.0 2.7mb 3.5mb new
ksh 17277 oracle 1.1 1 0.0 268kb 408kb PRI
sqlplus 20996 oracle 0.4 1 1.8 4.5mb 5.3mb died
ora_mmon_prd 4492 oracle 0.0 1 0.0 32.5mb 36.9mb SEM
ora_dbw0_prd 4477 oracle 0.0 1 0.0 32.6mb 36.2mb SEM
ora_smon_prd 4486 oracle 0.0 1 0.0 32.0mb 35.4mb SEM
agent 28017 esm_srv 0.0 7 0.4 29.2mb 38.0mb SLEEP
ora_pmon_prd 4471 oracle 0.0 1 0.0 31.1mb 33.5mb SLEEP
ora_qmnc_prd 2676 oracle 0.0 1 0.0 30.1mb 31.8mb SEM
ora_mman_prd 25796 oracle 0.0 1 0.0 21.3mb 23.6mb SEM
ora_mmon_prd 25810 oracle 0.0 1 0.0 23.6mb 27.0mb SEM
ora_pmon_prd 25792 oracle 0.0 1 0.0 22.4mb 23.8mb SLEEP
ora_smon_prd 25804 oracle 0.0 1 0.0 24.2mb 25.7mb SEM
ora_arc1_prd 25818 oracle 0.0 1 0.0 35.7mb 39.1mb SEM
ora_dbw0_prd 25798 oracle 0.0 1 0.0 24.5mb 25.8mb SEM
ora_ckpt_prd 25802 oracle 0.0 1 0.6 24.3mb 26.5mb SEM
ora_arc0_prd 25816 oracle 0.0 1 0.0 36.7mb 39.1mb SEM
ora_ctwr_prd 2654 oracle 0.0 1 0.0 30.6mb 33.9mb SEM
ora_qmnc_prd 25822 oracle 0.0 1 0.0 21.3mb 23.6mb SEM
java 2522 root 0.0 17 0.0 29.8mb 199.5mb SLEEP
vxfsd 37 root 0.0 42 34.7 9.4mb 9.4mb IO
LRUE-YOHP15_# terval toggle Page 1 of 2
LRUE-YOHP15_# pt Mem Rpt Disk Rpt NextKeys SlctProc Help Exit
Jean-Luc Oudart
Honored Contributor

Re: Top Command CPU Idle 0% and NICE value High

Hi

just noticed the sqlplus processes are not attached to a terminal. just wondering if you have a process that keeps starting (failing) sqlplus sessions. What are the ksh processs pid 19520 and 10804. PPID ?

Regards
Jean-Luc
fiat lux
Nazaf Khan
Frequent Advisor

Re: Top Command CPU Idle 0% and NICE value High

Hi Jean,

Thanks for the reply and help below are the details that you had asked.


LRUE-YOHP15_# ps -ef | grep 19520
oracle 24404 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 25423 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
root 22487 22357 0 10:59:54 ttyp2 0:00 grep 19520
oracle 26218 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 22770 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24089 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 19520 19519 2 Mar 12 ? 74:08 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 25128 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 22485 19520 0 10:59:54 ? 0:00 sleep 1
oracle 23012 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24380 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 22675 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24501 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24338 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24323 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 25771 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 23033 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24587 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 22754 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 23065 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 25909 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 26312 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 22961 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 27474 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 23776 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24364 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24308 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 26600 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24740 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 23655 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 25607 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 26461 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 202 19520 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh

====================

LRUE-YOHP15_# ps -ef | grep 10804
oracle 15899 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 14992 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 14064 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 14146 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 16178 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15175 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 13876 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15392 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15451 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 10804 10803 10 Mar 12 ? 72:45 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15656 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 16199 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15412 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15683 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15482 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15623 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15813 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 13810 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 12394 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 13896 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15590 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 14120 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15193 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15466 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 14759 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 14180 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15427 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 13941 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 14882 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 15060 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 16154 10804 0 Mar 12 ? 0:00 /usr/bin/ksh /home/oracle/apps_scripts/monitor/llycheckupdb.ksh
oracle 24508 10804 2 11:00:25 ? 0:00 sleep 1
root 24574 22357 0 11:00:26 ttyp2 0:00 grep 10804
Suraj K Sankari
Honored Contributor

Re: Top Command CPU Idle 0% and NICE value High

Hi,
As per your post
>>Load averages: 3.85, 1.21, 0.72
I can see cup use is more....
I think you oracle is taking more and more cup..if possable just bounce the Oracle.

Suraj
Jean-Luc Oudart
Honored Contributor

Re: Top Command CPU Idle 0% and NICE value High

Hi again

looks like the script llycheckupdb.ksh does not terminate normally. Is this normal you have so many of them still up and running (Note : from MAR 12 !!).

ps -ef | grep 10804


Regards
Jean-Luc
fiat lux