topic Re: system processes consuming high CPU resource. in Operating System - HP-UX

system processes consuming high CPU resource.

Phani_4 — Thu, 01 Apr 2004 13:39:32 GMT

Hi,

I have HP9000/K380 server with three 240 Mhz CPU.

When i looked at the top o/p the system parameter consuming high cpu...it is varying from 0% to 70%...But no one system processes consuming more than 3% of CPU individually..
"vhand" process consuming 3% of CPU even though there is no much Paging.

Can somebody please guide me how to resolve this issue..

Waiting for your prompt reply.

Thanks & Regards,
Phani Varma P.

Re: system processes consuming high CPU resource.

Todd McDaniel_1 — Thu, 01 Apr 2004 14:47:50 GMT

Have you looked at "sar -uM" see what your output is since midnite. it will show you individual cpu util

Also, many times on a small box with few cpu, you will see a "favorite" cpu based on the "shortness" of the path.

Are all or most of these procs on one CPU?

Re: system processes consuming high CPU resource.

Phani_4 — Thu, 01 Apr 2004 15:26:25 GMT

Todd,

Thanks for your response..Please find the attached sar o/p on this server.

One month back server was rebooted with HPMC, & found it is becasue of Cache parity error on one of the CPU..But we have not changed it..server is up & running fine..

Phani

Re: system processes consuming high CPU resource.

Todd McDaniel_1 — Thu, 01 Apr 2004 15:41:25 GMT

Was this for the time period you were concerned about?

Show the whole day of the occurences you are concerned with.

Are these few hours what you are worried about?

Re: system processes consuming high CPU resource.

Todd McDaniel_1 — Thu, 01 Apr 2004 15:43:46 GMT

Also, without knowing how your "normal" day goes and what are your high and low use times it would be hard to diagnose anything.

What you show is not unusual, imho. but it may be for your environment. Just b/c one cpu is at 70% is not a great concern to me.

If you can give a bit of detail on your normal day, it would be great.

Re: system processes consuming high CPU resource.

Phani_4 — Thu, 01 Apr 2004 16:26:21 GMT

Hi Todd,

I am waiting for my user's response on the server's performance in 1 Hr. time...becasue they are facing performance issue in the night time..

If you are available for some time, so that i can send you o/p of below commands.

# sar -uM 2 50
# sar -d 2 50

# vmstat 2 50

Thanks,
Phani

Re: system processes consuming high CPU resource.

Sundar_7 — Thu, 01 Apr 2004 17:01:30 GMT

Hi Phani,

# UNIX95= ps -ef -o pid,ppid,pcpu,args | sort -nbk 3 | tail -10

Post the output

Sundar

Re: system processes consuming high CPU resource.

Phani_4 — Thu, 01 Apr 2004 17:13:32 GMT

Sundar,

Please find the attached doc with o/p which u requested.

-o is not available in hp-ux.

Thanks,
Phani

Re: system processes consuming high CPU resource.

Sundar_7 — Thu, 01 Apr 2004 23:49:06 GMT

Hi Phani,

Are you kidding me ? :-) . I copied and pasted the command line from my test HP-UX box.

# UNIX95= ps -ef -o pid,ppid,pcpu,args | sort -nbk 3 | tail -10

Make sure there is a space after UNIX95=

or try this

# export UNIX95=1
# ps -ef -o pid,ppid,pcpu,args | sort -nbk 3 | tail -10
# unset UNIX95

Sundar

Re: system processes consuming high CPU resource.

Phani_4 — Fri, 02 Apr 2004 08:30:46 GMT

Hi Sundar,

I am not sure which HP-UX OS Version you are using..i didn't find the -o option in 10.20 & 11.11 as well.

# uname -a
HP-UX suboz039 B.11.11 U 9000/800 818622396 unlimited-user license
#
# ps -ef -o pid,ppid,pcpu,args | sort -nbk 3 | tail -10
ps: illegal option -- o
usage: ps [-edaxflP] [-u ulist] [-g glist] [-p plist] [-t tlist] [-R prmgroup]

Thanx,
Phani

Re: system processes consuming high CPU resource.

RAC_1 — Fri, 02 Apr 2004 08:35:34 GMT

UNIX95 variable is used to conform to XPG4.
Just do as follows.

UNIX95= ps -ef -o "all options here"

This will work. Post the output.

Anil

Re: system processes consuming high CPU resource.

Phani_4 — Fri, 02 Apr 2004 08:40:26 GMT

Hi Anil,

Thanx for your information..please find the below o/p

suboz026:/ > UNIX95= ps -ef -o "pid,ppid,pcpu,args"| sort -nbk 3 | tail -10
29669 19130 0 sh -c $GEMMS_ADDON/start_prem_scr
29670 29669 0 /PRDRS_cust_appl/add_on/start_prem_scr /PRDRS_cust_appl/add_
29679 29670 0 runform30x -c sc_appl:vt220 /PRDRS_cust_appl/add_on/prem_scr
29687 29679 0 oraclePRDFC (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))
29800 1 0 /PRDRS_cust_appl/osb/program/killer.sh /PRDRS_cust_appl/osb/
29803 29800 0 sleep 1200
29834 1480 0 telnetd
29835 29834 0 -ksh
29913 1 0 oraclePRDNR (LOCAL=NO)
27488 1 1 oraclePRDNR (LOCAL=NO)

Re: system processes consuming high CPU resource.

doug mielke — Fri, 02 Apr 2004 08:42:01 GMT

I would be a bit concerned if the system % went up suddenly. Don't put too much weight in the devision SAR makes between user and syatem, some conditions seem to show up in catagories that you wouldn't expect.

Since you have sar history on, try comparing a day's worth of sar with some from 2 weeks ago. Disk re- reads, as when a disk is starting to go bad, should show up as w i/o, but the real number to see is the sar -d service time. A network problem could show as nothing more thatn Hhigh system usage. Don't forget to look at name resolution. Changes here can hide themselves pretty well.

Re: system processes consuming high CPU resource.

Phani_4 — Fri, 02 Apr 2004 08:51:49 GMT

Hi Doug,

Please find the attached sar o/p on CPU & disk i/o when the end users experienced performance issues.

Thanx,
Phani

Re: system processes consuming high CPU resource.

RAC_1 — Fri, 02 Apr 2004 08:54:06 GMT

What is total system process usage and what is users and what is idle?

Did you look at swapinfo -mat, if vhand is in play system is paging out. Do get you get any pageouts in vmstat 5 5?

If not you should not worry about it. The point here is if enough CPU is avilable to users or not?

Anil

Re: system processes consuming high CPU resource.

Phani_4 — Fri, 02 Apr 2004 08:58:32 GMT

Hi Anil,

I have just replied to doug's message with all the o/p which you have asked..

Please find the attached o/p in my last message.

Thanx,
Phani

Re: system processes consuming high CPU resource.

RAC_1 — Fri, 02 Apr 2004 09:06:22 GMT

I did not fine anything unusual with sar -uM output. In fact when users usage is high, the idle has gone down. Which is obivous.

But for disk c8t3d3, what is the problem? The % wait time, %serve time, %busy -- everything is high. What applications you run on this machine? What is causing performance problem for disk-c8t3d3. What are your buffer settings- dbc_max_pct and dbc_min_pct?
What is swap size and how it is configured.

Anil

Re: system processes consuming high CPU resource.

Phani_4 — Fri, 02 Apr 2004 10:01:13 GMT

Hi Anil,

What applications you run on this machine?

Running Oracle Databse on this LUN

What is causing performance problem for disk-c8t3d3. What are your buffer settings- dbc_max_pct and dbc_min_pct?

Disk i/o going high because of the Oracle Databse HOT backup ( exp & compress ).

dbc_max_pct 50 - - -
dbc_min_pct 5 - - -

Physical Memory : 1.2 GB

Swap Defined : 1:5 GB

What is swap size and how it is configured.

The existing swap is Primary Swap ( Device Swap on root disk )..no secondary swap configured.

Thanx,
Phani

Re: system processes consuming high CPU resource.

RAC_1 — Fri, 02 Apr 2004 10:21:11 GMT

Disk problem OK. That explains why the disk is so busy.

About your dbc_max_pct and dbc_min_pct. The setting dc_max_pct is too high. In fact it default. That means buffer cache varies from 60mb to 600mb. are you getting it used?

Check sar -b and %rcache. If it is more than 90 % most of the time, you are effectively using it. Otherwise there is no need to set it that high. You can set dbc_max_pct to 10/15 %. This will give more memory to users and less memeory to system. You can it with glance -m

Anil

Re: system processes consuming high CPU resource.

Phani_4 — Mon, 05 Apr 2004 12:52:29 GMT

Hi,

The read cache utilzation is 100% & write cache utilization is on an average 98%..

What i observed was the 'sys' CPU utilization is high, when compared to other servers..

How to find out which system process is consuming high CPU ?

Regards,
Phani