topic Re: System not using all processors. in Operating System - HP-UX

System not using all processors.

Richard Pereira_1 — Thu, 14 Jul 2005 07:36:35 GMT

Hi,

I know this came up sometime ago, but didnt find any relevant threads in my seach.

My new vpar is on 11.11 and doesnt seems to be using all its CPUs. Here is a quick exemple of top.

System: psa1h12 Thu Jul 14 10:36:16 2005
Load averages: 19.65, 16.53, 16.71
493 processes: 451 sleeping, 41 running, 1 zombie
Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 38.18 0.0% 0.0% 1.0% 99.0% 0.0% 0.0% 0.0% 0.0%
1 0.10 1.0% 0.0% 5.0% 94.0% 0.0% 0.0% 0.0% 0.0%
2 0.11 0.2% 0.0% 7.9% 91.9% 0.0% 0.0% 0.0% 0.0%
3 40.19 0.0% 0.0% 0.8% 99.2% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 19.65 0.2% 0.0% 3.6% 96.2% 0.0% 0.0% 0.0% 0.0%

Memory: 5979456K (1745548K) real, 10205560K (3531084K) virtual, 344608K free Page# 1/21

Suggestions?

Re: System not using all processors.

David Child_1 — Thu, 14 Jul 2005 08:06:41 GMT

Are all your cpus bound or are two bound and two available for moving between vpars?

Only bound cpus can be used for I/O so perhaps its related to that.

David

Re: System not using all processors.

Richard Pereira_1 — Thu, 14 Jul 2005 08:08:44 GMT

psa1h11:/:>vparstatus -v -p psa1h12
[Virtual Partition Details]
Name: psa1h12
State: Up
Attributes: Dynamic,Autoboot
Kernel Path: /stand/vmunix
Boot Opts: -lq

[CPU Details]
Min/Max: 2/8
Bound by User [Path]:
Bound by Monitor [Path]: 1.12
1.13
Unbound [Path]: 1.14
1.15

[IO Details]
1.0.1
1.0.2
1.0.4
1.0.6
1.0.1.1.1.8.0 ALTBOOT
1.0.1.1.0.8.0, BOOT

[Memory Details]
Specified [Base /Range]:
(bytes) (MB)
Total Memory (MB): 8192

Re: System not using all processors.

Richard Pereira_1 — Thu, 14 Jul 2005 08:09:44 GMT

i'll have to disagree, i have several other vpars and they dont behave in this way.

Re: System not using all processors.

David Child_1 — Thu, 14 Jul 2005 08:19:13 GMT

Can you run 'top -h' and post a screen of that as well.

Thanks,
David

Re: System not using all processors.

Richard Pereira_1 — Thu, 14 Jul 2005 08:33:37 GMT

sure,

System: psa1h12 Thu Jul 14 11:33:27 2005
Load averages: 13.52, 17.52, 17.91
527 processes: 485 sleeping, 41 running, 1 zombie
Cpu states: (avg)
LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
13.52 26.7% 0.0% 0.0% 73.3% 0.0% 0.0% 0.0% 0.0%

Memory: 6343016K (2621000K) real, 12084852K (4933920K) virtual, 55444K free Page# 1/19

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
1 ? 26907 fsdev4 241 20 238M 163M run 0:41 8.50 8.48 PSAPPSRV
2 ? 26933 fsdev4 154 20 263M 184M sleep 0:52 7.31 7.30 PSAPPSRV
3 ? 3 root 128 20 32K 32K sleep 238:31 4.10 4.10 statdaemon
0 pts/5 12203 cmtst2 168 20 274M 152M sleep 9:26 2.41 2.40 PSPUBDSP
1 pts/0 9250 cmtst2 168 20 11844K 9820K sleep 0:26 0.75 0.75 top
2 ? 26616 fsdev 155 20 143M 71492K sleep 0:42 0.74 0.74 PSAPPSRV
3 ? 26994 fsdev4 155 20 37468K 32928K sleep 0:02 0.74 0.73 JSH
0 ? 1945 root 152 20 457M 93200K run 463:56 0.61 0.61 java
0 ? 26610 fsdev 155 20 141M 76124K sleep 0:44 0.59 0.58 PSAPPSRV
2 ? 1605 root 152 20 252M 8620K run 83:55 0.39 0.39 prm3d
0 ? 29562 rmdmo2 168 20 157M 32852K sleep 1:17 0.39 0.38 PSBRKDSP
2 ? 1581 root -16 20 53064K 14644K run 204:11 0.37 0.37 midaemon
1 ? 40 root 152 20 9632K 9632K run 47:09 0.31 0.31 vxfsd
2 ? 26829 fsdev3 152 20 181M 63384K run 1:07 0.30 0.30 PSPUBDSP
2 pts/5 12171 cmtst2 168 20 157M 37484K sleep 0:10 0.28 0.28 PSBRKDSP
0 pts/3 10430 rmdev2 168 20 159M 35392K sleep 0:16 0.27 0.27 PSBRKDSP
3 ? 29604 rmtst2 168 20 158M 33724K sleep 1:20 0.26 0.26 PSBRKDSP
2 pts/2 12634 root 179 20 11760K 9740K run 0:00 5.01 0.24 top
1 ? 29624 rmtst2 152 20 350M 78412K run 0:18 0.19 0.19 PSPUBHND
0 ? 28893 padev3 152 20 366M 82532K run 0:30 0.18 0.18 PSAPPSRV
2 ? 29614 rmtst2 152 20 350M 68956K run 0:18 0.17 0.17 PSPUBHND
3 ? 28475 hrtst 152 20 180M 34392K run 0:35 0.16 0.16 PSPUBDSP
3 ? 28881 padev3 152 20 370M 85700K run 0:30 0.16 0.16 PSAPPSRV
0 ? 12513 rmtst2 168 20 15616K 2396K sleep 0:00 0.16 0.16 PSWATCHSRV
2 ? 2924 root 152 20 15980K 2720K run 2:48 0.16 0.16 rep_server
3 ? 2925 root 152 20 13412K 1736K run 2:14 0.16 0.16 agdbserver
3 ? 28360 hrdev4 152 20 181M 37296K run 0:34 0.15 0.15 PSPUBDSP
2 ? 28350 hrdev4 168 20 82532K 15564K sleep 0:59 0.14 0.14 PSBRKDSP
3 ? 12461 padev3 168 20 15488K 2680K sleep 0:00 0.13 0.13 PSWATCHSRV

Re: System not using all processors.

David Child_1 — Thu, 14 Jul 2005 08:46:24 GMT

Okay, it is clear that all four cpus are being used. The question is why do two have a very high load average. A couple of things to look at;

'sar 5 5' (see what your %WIO is. )
'sar -d 5 5' (see how your disks are responding)
'vmstat 5 5' (check to see for blocking ('b' column)

If you have glance look through that.

If nothing pops out at you post the outputs from the above commands.

Thanks,
David

Re: System not using all processors.

Richard Pereira_1 — Thu, 14 Jul 2005 10:21:07 GMT

i hope this helps....

psa1h12:/:>sar -d 5 5

HP-UX psa1h12 B.11.11 U 9000/800 07/14/05

13:19:14 device %busy avque r+w/s blks/s avwait avserv
13:19:19 c4t8d0 2.80 0.50 6 59 0.00 4.71
c5t8d0 0.80 0.50 6 71 0.00 1.19
13:19:24 c4t8d0 2.00 0.50 5 35 0.00 4.16
c5t8d0 0.20 0.50 3 28 0.00 0.85
13:19:29 c4t8d0 2.79 0.50 7 76 0.00 3.97
c5t8d0 0.20 0.50 3 38 0.00 0.64
13:19:34 c4t8d0 1.80 0.50 4 34 0.00 4.37
13:19:39 c4t8d0 2.60 0.50 4 39 0.00 6.65

Average c4t8d0 2.40 0.50 5 49 0.00 4.65
Average c5t8d0 0.24 0.50 2 28 0.00 0.96
psa1h12:/:>vmstat 5 5
procs memory page
faults cpu
r b w avm free re at pi po fr de sr in
sy cs us sy id
31 1 0 1158844 17848 20 5 6 4 0 0 21 1074
3230 350 2 1 97
46 0 0 1158746 17783 43 40 1 0 0 0 0 1089
6848 406 2 4 94
46 0 0 1158746 17783 13 12 1 0 0 0 0 1072 3102 402 0 2 97
26 1 0 1160741 17796 67 12 0 0 0 0 0 1067 17646 445 7 3 89
26 1 0 1160741 17795 26 3 0 0 0 0 0 1109 7909 407 2 3 96

Re: System not using all processors.

Richard Pereira_1 — Thu, 14 Jul 2005 10:22:11 GMT

btw, this system has no san drives, just uses internal

Re: System not using all processors.

David Child_1 — Thu, 14 Jul 2005 11:37:05 GMT

Sorry to keep asking for more info. Basically two of your procs have many processes queued and the other two don't, but all are being used. As I already mentioned, only bound cpus can do I/O so it looks like there is a queue of processes waiting to use these two cpus to do I/O. The other cpus are handling other requests.

That being said, the real question is why is there a queue. As you can see from top, none of the cpus are very busy.

Take a look at;
swapinfo -tam
sar 5 5 (to see what your %wio is)

David

Re: System not using all processors.

Richard Pereira_1 — Thu, 14 Jul 2005 12:02:08 GMT

No worries, ill send anything that can help.

psa1h12:/:>sar 5 5

HP-UX psa1h12 B.11.11 U 9000/800 07/14/05

14:59:56 %usr %sys %wio %idle
15:00:01 23 5 2 70
15:00:06 22 6 0 72
15:00:11 35 6 2 56
15:00:16 28 6 1 66
15:00:21 17 5 0 78

Average 25 6 1 68
psa1h12:/:>swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 16384 2422 13962 15% 0 - 1 /dev/vg00/lvol2
reserve - 8318 -8318
memory 6487 1228 5259 19%
total 22871 11968 10903 52% - 0 -

I'm at a loss bcs the loads are high, but my impression is its not doing much swapping or disk i/o.

Re: System not using all processors.

David Child_1 — Thu, 14 Jul 2005 12:24:01 GMT

Richard,

Overall it doesn't look too bad, but it does appear that you are swapping (15%);

dev 16384 2422 13962 15% 0 - 1 /dev/vg00/lvol2

(plus quite a few page outs in vmstat)

and that will only use the bound cpus. However, it does look like the rest of the system isn't doing too bad. Not much %wio, etc. I think if you can find a way to reduce the memory usage (or add in more memory from a less-used vPar) you will see those cpu load averages go down.

David

Re: System not using all processors.

Richard Pereira_1 — Wed, 20 Jul 2005 08:12:14 GMT

thanks for the suggestions...

Re: System not using all processors.

Paul Brear_1 — Wed, 20 Jul 2005 23:32:54 GMT

forgive my ignorance with vpars but does this not mean two unbound cpus currently ?
bound at 1.12 , 1.13
unbound at 1.14 , 1.15

CPU Details]
Min/Max: 2/8
Bound by User [Path]:
Bound by Monitor [Path]: 1.12
1.13
Unbound [Path]: 1.14
1.15