CPU load for processes and for the overall system in top

[Arthur Luimes_2]

It seems that there is a difference how top handles the CPU load from processes and the overall CPU load on the system. See for instance the following top output:
System: hp_ux1 Tue Jun 29 17:23:34 2004
Load averages: 0.31, 0.27, 0.21
98 processes: 91 sleeping, 7 running
Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 0.12 0.0% 0.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0%
1 0.51 0.2% 0.0% 0.0% 99.8% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 0.31 0.0% 0.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0%

Memory: 157448K (56400K) real, 98528K (35128K) virtual, 16080K free Page# 1/3

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
0 ? 33 root 152 20 1856K 0K run 25:25 1.13 1.13 vxfsd
1 ? 22283 nfuse 152 20 14872K 7756K run 3:42 0.94 0.94 httpd
1 ? 22281 nfuse 152 20 14616K 7372K run 3:38 0.59 0.59 httpd
1 ? 22282 nfuse 152 20 14744K 7564K run 3:38 0.51 0.51 httpd
1 ? 22272 nfuse 152 20 258M 58084K run 2:31 0.49 0.48 java
0 ? 22218 tss 152 20 189M 46188K run 2:37 0.31 0.30 native

Here, the CPU is 100% idle, so 0% active. However, individual processes have non-zero CPU load, like PID 33 (1.13%), PID 22283 (0.93%), etc. Adding up all the CPU loads of the individual process does not give me the overall CPU load.

Am I misunderstanding something?

[Mel Burslan]

together with not being sure about the internals of top/glance/etc tools, I think you are facing the fact that your processes introduce negligably small loads onto your cpu which do not add up to even 1% even though their individual load percentage representations say the otherwise.

[Arthur Luimes_2]

Mel,

Does this mean that what is shown in the "%CPU" column is inacurate for small CPU load?

Thanks,
Arthur

[Mel Burslan]

I think that is correct. top tool is never meant to be an accurate system performance tool. it is a helper to the sysadmin. I think, the better way to monitor the actual performance is via measureware/glance which as you know cost a pretty penny after the 30/60/90 day trial period.

I think what you see as the difference in numbers on top is how the load gets calculated per cpu per process and on the whole system. God only knows what syscalls are responsible from those calculations. I am not a system or kernel programmer myself and I can not tell this to you but all I know is this. The kernel was not written for a fine and detailed floating point arithmetic calculations.

[Bill Hassell · Accepted solution]

The only way to collect and display accurate CPU usage is for you to be able to read the data faster than 100 times per second. Modern operating systems run millions of instructions per second and HP-UX will be context switching (changing the running programs) dozens of times per second. Since top is just another program, there is very little likelihood that top can sample the current state of the processes and not get switched off (context switched) or be able to gather the data fast enough to eliminate the meaningless discrepancies at low usage.

If you extend the reporting period to 10 or 20 seconds, the differences will likely average out as long as the system's activity remains constant. Even if nothing is running in HP-UX, there are dozens of tasks going on (daemons monitoring the LAN, time of day updates, cron tasks, etc). Most of these run very quickly (a few milliseconds) so are not important to the overall system load.

[Shine_5]

Hello Arthur,

In some instances top(1) may report inaccurate values due
to the use and design of the underlying pstat_getproc() kernel interface.
For instance, it shows:
# top
System: Fri Mar 21 03:11:12 2003
Load averages: 0.02, 0.02, 0.01
82 processes: 79 sleeping, 3 running
Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 0.00 0.0% 0.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0%
1 0.00 0.2% 0.0% 0.7% 99.1% 0.0% 0.0% 0.0% 0.0%
2 0.02 0.0% 0.0% 0.2% 99.8% 0.0% 0.0% 0.0% 0.0%
3 0.05 0.0% 0.0% 0.5% 99.5% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 0.02 0.1% 0.0% 0.4% 99.5% 0.0% 0.0% 0.0% 0.0%

Memory: 26996K (9896K) real, 28876K (15708K) virtual, 682716K free Page#
1/10

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU
COMMAND
1 pts/ta 28326 user1 152 20 276K 280K run 0:23 89.88 89.72
sample <<<<<
1 ? 34 root 152 20 0K 0K run 0:53 0.88 0.88
vxfsd
1 pts/tb 28328 user1 178 20 436K 368K run 0:00 0.73 0.31
top
3 ? 296 root 154 20 32K 96K sleep 5:31 0.22 0.22
syncer
1 ? 12 root -32 20 0K 0K sleep 3:05 0.15 0.15
ttisr
3 ? 797 root 154 20 1464K 1352K sleep 3:10 0.14 0.14
mib2agt
2 ? 3 root 128 20 0K 0K sleep 3:49 0.13 0.13
statdaem
1 ? 1094 root 154 10 624K 816K sleep 2:48 0.12 0.12
psmctd
3 pts/tb 28310 user1 168 20 2960K 432K sleep 0:00 0.04 0.04
csh
--
Inflated percentage CPU utilisation values (pst_pctcpu) are calculated
for processes' that create threads which terminate and are not
immediately reaped. The man page for pstat(2) alludes to this:

WARNINGS
Some parts of the program status may not get updated when a process
becomes a zombie. An example is that the cpu percentage is not
updated because the process is not expected to be scheduled to run
after entering the zombie state.


I hope this will clear your query.

Thanks

Shine

[Arthur Luimes_2]

Mel, Bill, Shine,

Thnaks a lot for your feedback. I will now take the figures top presents as a guideline, and no longer as an absolute measure.

Arthur

[Ted Buis]

Glance is more accurate on HP-UX as it uses unique hp-ux instrumentation that hp designed into hp-ux. Some metrix may come from the standard unix methods. I think this is address in the book "HP-UX Tuning and Performance" by Robert F. Sauers, but I'll have to look and see if there are more specifics on load in glance. I remember xload got a mention in the book.