cpu utilization over 100% by using top

[natis]

when I run top command, the information below.

Load averages: 1.16, 1.23, 1.24
399 processes: 319 sleeping, 79 running, 1 zombie
Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 0.98 0.3% 0.0% 83.9% 15.7% 0.0% 0.0% 0.0% 0.0%
1 1.36 0.0% 0.0% 83.9% 16.1% 0.0% 0.0% 0.0% 0.0%
2 1.06 18.7% 0.0% 81.3% 0.0% 0.0% 0.0% 0.0% 0.0%
3 1.12 27.0% 0.0% 46.0% 27.0% 0.0% 0.0% 0.0% 0.0%
4 1.16 31.1% 0.0% 68.9% 0.0% 0.0% 0.0% 0.0% 0.0%
5 1.36 0.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0% 0.0%
6 1.21 21.9% 0.0% 78.1% 0.0% 0.0% 0.0% 0.0% 0.0%
7 1.28 49.9% 0.0% 50.1% 0.0% 0.0% 0.0% 0.0% 0.0%
8 1.17 15.5% 0.0% 68.7% 15.7% 0.0% 0.0% 0.0% 0.0%
9 1.43 40.6% 0.0% 59.4% 0.0% 0.0% 0.0% 0.0% 0.0%
10 1.21 31.1% 0.0% 37.8% 31.1% 0.0% 0.0% 0.0% 0.0%
11 0.77 31.1% 0.0% 58.4% 10.5% 0.0% 0.0% 0.0% 0.0%
12 1.06 0.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0% 0.0%
13 0.98 0.2% 0.0% 68.9% 30.9% 0.0% 0.0% 0.0% 0.0%
14 1.09 37.5% 0.0% 62.5% 0.0% 0.0% 0.0% 0.0% 0.0%
15 1.25 99.3% 0.0% 0.7% 0.0% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 1.16 25.6% 0.0% 65.1% 9.3% 0.0% 0.0% 0.0% 0.0%

Memory: 7611912K (6611264K) real, 8268816K (7132432K) virtual, 2878952K free Page# 1/20

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
9 ? 18925 mnadm 241 20 27736K 10212K run 114:09 133.31 133.08 onpload
6 ? 19022 mnadm 241 20 27736K 10188K run 113:51 118.20 118.00 onpload
13 ? 7571 mnadm 187 20 1076M 10212K run 17:47 114.18 113.98 onpload
8 ? 13605 mnadm 187 20 1076M 10164K run 127:39 109.88 109.69 onpload
15 ? 18862 mnadm 241 20 1076M 10188K run 71:29 106.94 106.76 onpload
5 ? 18710 mnadm 187 20 1076M 10164K run 106:36 106.90 106.71 onpload
7 ? 18843 mnadm 186 20 1076M 10164K run 106:48 103.52 103.34 onpload
12 ? 18708 mnadm 187 20 1076M 10164K run 109:07 102.09 101.91 onpload
0 ? 18898 mnadm 187 20 1076M 10164K run 96:34 101.45 101.28 onpload
14 ? 25692 mnadm 186 20 1076M 10216K run 207:01 97.70 97.53 onpload
2 ? 18841 mnadm 241 20 1076M 10188K run 65:17 88.40 88.24 onpload
8 ? 19083 mnadm 156 20 1076M 10192K run 58:17 79.36 79.22 onpload
3 ? 13602 mnadm 241 20 1076M 10192K run 37:09 78.44 78.30 onpload
10 ? 7469 mnadm 241 20 1076M 10240K run 8:46 76.40 76.27 onpload
10 ? 25689 mnadm 241 20 1076M 10228K run 51:42 75.90 75.77 onpload
3 ? 23727 mnadm 149 20 1080M 10248K sleep 64:19 67.75 67.63 onpload
4 ? 19029 mnadm 241 20 1076M 10184K run 41:41 67.35 67.23 onpload
1 ? 28718 mnadm 182 20 1080M 10196K run 168:47 50.88 50.79 onpload
14 ? 24713 mnadm 154 20 1080M 10272K sleep 61:34 49.04 48.96 onpload
2 ? 28750 mnadm 154 20 1080M 10216K sleep 164:26 34.01 33.95 onpload


Can cpu run over 100%?
How it get cpu resource when it run over 100%?

[Matti_Kurkela]

If a process has several threads (multi-threading), each thread can be running on a different CPU. If there are enough CPUs, each individual thread can get up to 100% CPU utilization. This makes the per-process CPU utilization go over 100%.

By the way, your the largest per-CPU percentages seem to be in the SYS column. By the totals, your 16-CPU machine is doing 65.1 % system-internal housekeeping and only 25.6 % "real work". This suggests you may have a performance bottleneck somewhere.

Run "vmstat" for a while and watch for the number of page-outs. If the number of page-outs is large and stays that way, your machine seems to be thrashing because you don't have enough physical memory (RAM): the CPU power is wasted in moving things in and out of swap repeatedly. This can make the system perform very badly.

If you cannot get more RAM, changing the load profile might help. If you're loading data to the system, see if you can split the load into parts you can run one at a time. If this allows you to avoid thrashing, this may actually reduce the total time required to load all the data.

MK