high cpu utilzation

[Shivkumar]

Dear Sirs,

I am seeing 97% cpu utilization on a server. Someone suggested the the server has been up for 406 days and hence it needs reboot to clear the cpu. I also saw from the below output that java process is taking so much cpu but it is using the normal amount of memory (hence the suspection of memorty leakage may not exist). I have copied the output of "top" command for your kind perusal.

Load averages: 1.35, 1.44, 1.43
370 processes: 261 sleeping, 108 running, 1 zombie
Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 1.14 4.0% 0.0% 84.2% 11.9% 0.0% 0.0% 0.0% 0.0%
1 1.55 2.0% 0.0% 89.2% 8.8% 0.0% 0.0% 0.0% 0.0%
2 0.75 10.8% 0.0% 68.6% 20.6% 0.0% 0.0% 0.0% 0.0%
3 1.95 2.0% 0.0% 92.2% 5.9% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 1.35 4.9% 0.0% 83.3% 11.8% 0.0% 0.0% 0.0% 0.0%

Memory: 3531708K (3258096K) real, 21853896K (7384236K) virtual, 1902632K free Page# 1/42

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
1 ? 21669 weblogic 152 20 1389M 513M run 18474:45 357.77 357.15 java
3 ? 3880 root -16 20 35356K 10612K run 23:19 8.21 8.20 midaemon
2 ? 8955 weblogic 152 20 907M 514M run 254:16 2.45 2.45 java
1 ? 4963 weblogic 152 20 891M 425M run 139:03 2.05 2.04 java
1 ? 20989 weblogic 152 20 1396M 496M run 164:15 1.80 1.80 java
3 ? 16153 weblogic 152 20 760M 480M run 808:42 1.29 1.29 java
0 ? 16848 weblogic 152 20 785M 401M run 279:06 1.22 1.22 java
2 ? 17650 weblogic 152 20 469M 175M run 703:53 0.95 0.95 java

My question is why 97% of cpu is being used ? I believe that rebooting should always be the last option.

Thanks,
Shiv

[Bharat Katkar]

Hi Shiv,
The load average of 1.35 looks ok. Does user complain about performance. Is the system performance slow?

If not i would suggest you to continue as it is.

CPU getting utilsed to it capcity can be a good sign. :)

If possible (not required) i recommend you to take server reboot once in a quarter. Cleanup process/removing logs files etc. are done during this.

Hope that helps.
Regards,

[Torsten.]

Check if you have the current patches installed. See this link for more information:

http://www.hp.com/products1/unix/java/patches/index.html

[B. Hulst · Accepted solution]

Hi,

Java virtual machine have a tendency to hog CPU when they can. Usually it is no problem.

Are you able to identify the function of this (weblogic) java vm?

Regards,
Bob

[Ranjith_5]

Hi Shiva,

It is good to reboot the system to free up some amount of resources. I dont know the reason why you havent even rebooted even once for the last 406 days.

from this I can also assume that you have not applied kernel pachces to this system for a long time,which require reboot.

please look at the paching level of your system.Secondly go through the performance cook book attached and compare the suggested kernel parameters for your system. Its a great doc written by HP R&D engineers and wil help you to many of your performance bottleneck problems.

Wish you good luck.


Regards,
Syam

[Devender Khatana]

Hi,

It is quite clear that the no reboot since 406 days means that the current patch level is quite old. The pacthed should be updated with recent ones.

Also there should be a option to restart the application in such scenarios and reboot should not be required. As I am not familiar with weblogic can not say how to do that. Is not it there?

HTH,
Devender

[Ranjith_5]

Hi Shiva,

Here is one knowledge base doc from HP on

" Determining the Cause of System Performance Problems ". Please find the attachment.


Regards,
Syam

[Arunvijai_4]

Shiv, Java in HP-UX will use some amount of CPU.You have to bear with that, In case of shoot-out, try to upgrade your Java version and download latest from www.hp.com/go/java

-Arun

[Andrew Merritt_2]

> It is good to reboot the system to free up
> some amount of resources. I dont know the
> reason why you havent even rebooted even
> once for the last 406 days.

Possibly because it's an HP-UX system, not a Windows system! Having systems running for over a year without a reboot is not unusual in an enterprise environment. Properly written applications and OS should not need the system to rebooted. If an individual process is causing trouble it should be possible to kill it. Only in exceptional circumstances is a reboot necessary to clear a problem in the kernel. If there is some problem that requires a regular reboot, get onto your HW or SW vendor and get them to fix it!

The only real reason to reboot in normal operation should be to apply a critical kernel patch or make hardware changes.

Andrew

[Tim Sanko]

>It is good to reboot the system to free up some amount of resources. I dont know the reason why you havent even rebooted even once for the last 406 days.

If you have cognos or jdk 1.3 or previous there may be memory issues that require a reboot. This is an OS not a MS POS. (piece of S...)

1. run sar -a to see what the system thinks of itself before you do anything.

2. Check the memory with top to see how much memory is allocated.

3. Run swapinfo. If you are swapping tune it or add RAM.

3. I have found make sure you have more than one GB of RAM free or fix the size of your buffer cache. It can be worse than a straight memory to disk swap when it gets close enough to "Rob Peter to pay Paul"

4. run sar -a to see what the system thinks of itself before you do anything else.



>from this I can also assume that you have not applied kernel pachces to this system for a long time,which require reboot.

Sometimes it is good to let a sleeping dog lie.

>please look at the paching level of your system.Secondly go through the performance cook book attached and compare the suggested kernel parameters for your system. Its a great doc written by HP R&D engineers and wil help you to many of your performance bottleneck problems.

Try what I have suggested first. THen later we can get into real action after we see the sar output...

Tim

[Ted Buis]

It looks to me like PID 21669 is using all the resources. That doesn't seem to me like an OS problem, but more like an application stuck in a loop. You might want to ask weblogic about it, or just start and stop weblogic.