Poor Performance

[Cristina Sunshine]

Hello All,

HP Server rp3440
1 CPU
6GB Memory
Progress Databases

We are currently having a serious performance issue here.
I am a DBA and I have checked all the Database parameters and they are okay.
I am very concerned about the load average of the TOP Output, the value is very high.

I would like to know if High Load Average could impact on the performance ??

Any commands to get statistics information will be greatly appreciated.

Kind Regards,

Cris

[Jaime Bolanos Rojas.]

Cristina!

For memory issues, check with dmesg, swapinfo -tam, for disks issues, check with sar -d, same for the cpu, algo glance is a great tool for performance issues.

Regards,

Jaime.

[Steven E. Protter]

Shalom,

By itself a high load average only indicates a lot of processes are waiting for cpu. It is not by itself the indication of a performance problem.

http://www.hpux.ws/system.perf.sh

A good set of background free tools for getting a handle on performance problems.

SEP

[Bill Hassell]

There's not very much you can do until you identify the processes that are consuming the resources. A high load average is normal if you have dozens of processes all trying to run at the same time (typical for a busy database) and just one CPU. You have a lot of memory which is good, but with just one CPU, you'll have to limit the number of users or simultaneous DB access points. The rp3440 is a good little machine but without a second processor (max allowed for this model), it will always be single-threaded. That means that everything from kernel to drivers to Progress code to filesystem access will all go through the one processor. The load average (from top or uptime) is a rough indication of how many processors you really need. A load of 1.0 is normal for your machine, 2.0 is busy and anything more means the system will be very slow.

Also check syslog.log for any error messages.

[Cristina Sunshine]

HEllo Everyone!!

Thanks a lot for everyone who has responsed.

Which file in HP-UX would show me how many CPUs and Memory I have?

I ran sar -d and here is the output:

08:00:02 device %busy avque r+w/s blks/s avwait avserv
08:15:01 c2t0d0 16.72 7.42 73 1541 11.08 7.48
c2t1d0 10.68 11.03 43 985 13.94 8.96
c4t10d0 7.17 2.31 19 281 5.88 8.89
c3t11d0 3.58 4.39 8 134 12.38 12.75
c4t12d0 2.29 0.52 8 120 0.06 3.39
c3t13d0 0.52 0.65 1 18 0.36 6.91
c4t8d0 14.96 2.04 335 7913 2.76 1.71
c3t9d0 9.66 10.75 47 3320 15.84 9.24
c4t14d0 3.00 3.42 22 1148 5.85 4.51
c3t15d0 2.43 3.58 20 1048 4.95 3.91
c3t2d0 5.87 0.50 179 12906 0.00 0.66
c7t0d0 1.32 4.24 4 82 6.02 5.61
c7t1d0 0.53 8.35 2 42 9.64 6.42
c7t2d0 0.88 0.50 2 35 0.01 5.14
c7t3d0 0.58 0.50 1 22 0.01 5.13
c7t4d0 0.19 0.55 0 8 0.13 4.73
c7t5d0 0.16 0.56 0 6 0.10 5.29
c7t8d0 0.08 0.87 0 5 0.46 5.31
c7t9d0 0.05 1.03 0 3 0.51 5.69
c7t10d0 1.25 0.88 3 53 0.74 5.18
c7t11d0 0.63 1.07 2 34 0.65 4.42
c7t12d0 0.03 0.50 0 1 0.00 6.45
08:30:00 c2t0d0 65.19 4.92 292 5934 7.31 6.68
c2t1d0 41.59 6.87 177 4136 8.80 7.64
c4t10d0 5.71 2.12 13 190 6.00 11.05
c3t11d0 3.11 3.32 7 92 9.58 14.09
c4t12d0 0.69 0.50 3 37 0.00 3.12
c3t13d0 0.34 0.50 1 7 0.00 6.44
c4t8d0 0.20 0.50 1 7 0.00 6.36
c3t9d0 0.14 0.50 0 3 0.00 11.37
c4t14d0 2.11 223.48 10 240 165.15 10.00
c3t15d0 1.75 173.37 9 231 117.61 9.35
c7t0d0 1.80 1.18 6 133 1.38 5.17
c7t1d0 0.85 1.67 3 88 1.72 4.90
c7t2d0 0.67 0.55 1 18 0.17 6.69
c7t3d0 0.48 0.56 1 15 0.11 5.52
c7t4d0 0.14 0.55 0 4 0.35 5.56
c7t5d0 0.13 0.51 0 3 0.10 5.32
c7t8d0 0.08 1.09 0 5 1.47 5.86
c7t9d0 0.05 1.21 0 3 1.38 7.36
c7t10d0 1.03 1.09 3 49 1.12 5.45
c7t11d0 0.49 1.45 2 29 1.15 4.75
c7t12d0 0.00 0.50 0 0 0.00 7.25
08:45:01 c2t0d0 29.81 5.93 173 5688 9.16 5.21
c2t1d0 19.68 8.70 102 3734 12.21 6.18
c4t10d0 4.30 0.78 9 124 1.27 9.35
c3t11d0 2.23 1.02 4 61 2.07 9.91
c4t12d0 0.49 0.50 1 8 0.00 7.93
c3t13d0 0.29 0.50 1 6 0.00 5.99
c4t8d0 0.31 0.94 1 12 1.39 8.33
c3t9d0 0.22 0.90 1 9 1.18 8.11
c4t14d0 1.78 17.09 6 135 24.76 12.05
c3t15d0 1.46 19.21 5 126 27.78 12.13
c7t0d0 1.07 1.11 2 45 1.35 8.00
c7t1d0 0.55 1.33 2 31 1.17 6.12
c7t2d0 0.78 0.51 2 24 0.05 6.57
c7t3d0 0.51 0.50 1 20 0.02 5.12
c7t4d0 0.15 0.76 0 4 0.82 5.46
c7t5d0 0.13 0.79 0 4 0.65 5.55
c7t8d0 0.14 3.00 1 8 3.81 6.77
c7t9d0 0.09 3.70 0 6 4.13 7.26
c7t10d0 0.99 1.45 2 28 1.97 8.14
c7t11d0 0.45 1.77 1 22 1.62 6.03
09:00:01 c2t0d0 10.86 10.07 29 346 20.84 14.26
c2t1d0 7.32 10.60 24 301 18.06 12.34
c4t10d0 5.03 7.77 10 116 23.16 17.49
c3t11d0 3.50 9.59 8 89 27.89 17.83
c4t12d0 0.58 0.50 1 11 0.03 8.76
c3t13d0 0.33 0.50 1 6 0.02 7.39
c4t8d0 2.14 0.52 10 208 0.06 2.66
c3t9d0 0.32 0.57 1 56 0.26 5.59
c4t14d0 1.81 6.73 5 140 13.37 13.36
c3t15d0 1.50 6.88 5 135 11.92 12.71
c7t0d0 1.17 1.72 3 53 2.58 7.75
c7t1d0 0.60 2.30 2 36 2.67 7.10
c7t2d0 0.71 0.51 2 24 0.05 6.61
c7t3d0 0.50 0.51 1 19 0.02 5.50
c7t4d0 0.12 0.97 0 3 0.70 5.58
c7t5d0 0.12 0.99 0 3 0.83 5.79
c7t8d0 0.14 2.07 1 10 3.13 7.37
c7t9d0 0.10 2.36 0 8 4.18 8.69
c7t10d0 0.89 1.23 2 28 1.50 7.46
c7t11d0 0.47 1.44 1 22 1.16 5.53
09:15:01 c2t0d0 11.02 36.16 29 317 49.84 13.93
c2t1d0 7.31 42.85 23 282 53.72 12.05
c4t10d0 6.79 3.43 14 198 9.76 11.13
c3t11d0 3.09 5.96 7 94 17.41 14.13
c4t12d0 0.85 0.50 2 36 0.01 4.83
c3t13d0 0.39 0.50 1 12 0.03 6.36
c4t8d0 0.27 0.59 1 13 0.38 5.85
c3t9d0 0.12 0.70 0 5 0.53 7.85
c4t14d0 3.34 280.59 24 457 209.19 7.74
c3t15d0 2.69 240.82 23 451 153.33 6.52
c7t0d0 1.04 2.78 2 38 4.36 9.16
c7t1d0 0.57 3.41 2 31 3.79 6.99
c7t2d0 0.84 0.51 2 25 0.07 6.60
c7t3d0 0.58 0.51 1 19 0.02 5.37
c7t4d0 0.14 1.13 0 3 1.15 6.12
c7t5d0 0.13 1.15 0 3 1.13 5.99
c7t8d0 0.11 3.33 0 7 4.33 7.63
c7t9d0 0.08 3.69 0 6 3.42 6.98
c7t10d0 1.20 1.26 2 34 1.53 7.64
c7t11d0 0.61 1.32 2 29 1.09 5.12
09:30:00 c2t0d0 7.88 5.10 17 141 12.97 13.44
c2t1d0 5.02 5.54 13 114 11.61 11.25
c4t10d0 4.05 5.68 9 164 15.96 13.07
c3t11d0 2.53 6.77 7 141 17.65 12.12
c4t12d0 0.57 0.50 1 12 0.00 7.58
c3t13d0 0.32 0.50 1 7 0.00 6.75
c4t8d0 0.18 0.59 0 7 0.40 7.77
c3t9d0 0.11 0.60 0 5 0.48 9.06
c4t14d0 1.32 7.22 3 58 13.99 13.05
c3t15d0 1.12 7.79 3 56 13.54 12.63
c7t0d0 0.85 1.35 2 32 2.04 8.59
c7t1d0 0.50 1.47 1 25 1.89 7.01
c7t2d0 0.67 0.92 2 25 0.99 6.36
c7t3d0 0.48 1.00 1 20 0.70 5.48
c7t4d0 0.12 0.77 0 3 0.42 5.56
c7t5d0 0.11 0.78 0 3 0.45 5.21
c7t8d0 0.14 1.36 0 7 2.09 8.03
c7t9d0 0.10 1.41 0 6 1.50 7.13
c7t10d0 0.79 1.17 2 28 1.37 7.80
c7t11d0 0.41 1.35 1 20 1.20 5.65
c7t12d0 0.17 3.70 3 154 1.13 3.89
09:45:00 c2t0d0 13.44 33.83 41 557 36.47 12.22
c2t1d0 8.76 38.15 33 479 35.19 11.01
c4t10d0 6.39 0.73 14 252 1.04 7.47
c3t11d0 2.21 0.97 5 103 1.67 9.13
c4t12d0 1.58 0.50 5 130 0.03 3.86
c3t13d0 0.52 0.51 2 55 0.09 5.87
c4t8d0 0.40 0.50 3 69 0.03 2.72
c3t9d0 0.16 0.50 1 41 0.03 3.31
c4t14d0 1.98 13.52 7 128 26.05 13.00
c3t15d0 1.65 14.51 6 121 22.87 12.51
c7t0d0 0.89 1.45 2 41 2.32 7.72
c7t1d0 0.48 1.76 1 28 2.16 6.01
c7t2d0 0.76 0.50 2 20 0.02 6.46
c7t3d0 0.55 0.50 1 17 0.01 5.29
c7t4d0 0.14 0.74 0 3 0.31 5.41
c7t5d0 0.13 0.75 0 3 0.40 5.34
c7t8d0 0.15 3.99 1 9 5.71 7.86
c7t9d0 0.11 4.42 0 8 5.32 7.59
c7t10d0 1.38 1.31 3 50 1.67 6.38
c7t11d0 0.56 1.85 2 29 1.98 5.51
c7t12d0 0.00 0.50 0 0 0.00 5.88
10:00:01 c2t0d0 12.82 12.09 40 606 15.86 11.27
c2t1d0 8.21 16.67 27 344 18.35 11.87
c4t10d0 8.72 2.13 39 1341 5.48 6.16
c3t11d0 5.12 3.09 23 875 7.99 7.83
c4t12d0 7.05 0.50 38 1317 0.00 2.23
c3t13d0 1.68 0.50 12 667 0.01 2.92
c4t8d0 0.96 0.50 4 69 0.00 2.80
c3t9d0 0.14 0.50 0 13 0.00 5.79
c4t14d0 1.86 11.02 6 117 20.59 11.95
c3t15d0 1.41 13.49 5 97 20.35 12.53
c7t0d0 0.81 3.20 2 34 4.44 8.35
c7t1d0 0.48 3.80 1 26 3.99 6.97
c7t2d0 0.94 0.51 2 26 0.04 6.21
c7t3d0 0.68 0.51 2 23 0.03 4.89
c7t4d0 0.12 0.60 0 3 0.41 5.82
c7t5d0 0.12 0.61 0 3 0.50 5.77
c7t8d0 0.06 0.72 0 2 0.62 6.08
c7t9d0 0.04 0.81 0 2 0.59 6.10
c7t10d0 1.48 1.08 4 58 1.20 5.60
c7t11d0 0.62 1.44 2 32 1.44 5.09
10:15:00 c2t0d0 12.77 11.00 36 366 15.54 11.27
c2t1d0 7.81 14.53 24 271 16.68 11.32
c4t10d0 11.96 1.08 65 2351 1.92 3.95
c3t11d0 5.72 1.43 37 1649 2.84 4.14
c4t12d0 1.80 0.50 12 475 0.00 2.08
c3t13d0 0.88 0.50 6 311 0.00 2.78
c4t8d0 5.76 5.01 33 1675 8.00 7.61
c3t9d0 4.17 6.23 24 1590 8.09 7.79
c4t14d0 4.37 114.03 29 966 74.97 5.96
c3t15d0 3.68 142.01 26 955 114.68 6.15
c7t0d0 1.02 1.62 2 52 2.34 7.73
c7t1d0 0.54 2.01 2 38 2.34 6.30
c7t2d0 0.84 0.52 2 20 0.05 6.29
c7t3d0 0.61 0.51 2 18 0.02 5.00
c7t4d0 0.15 0.50 0 4 0.00 5.05
c7t5d0 0.14 0.50 0 3 0.00 4.99
c7t8d0 0.06 1.07 0 3 1.76 8.51
c7t9d0 0.05 1.14 0 3 1.27 7.13
c7t10d0 1.09 2.64 2 33 4.05 8.36
c7t11d0 0.54 2.91 2 26 3.17 6.09
c7t12d0 0.00 0.50 0 0 0.00 3.10
10:30:00 c2t0d0 10.14 12.98 28 302 25.06 11.42
c2t1d0 6.46 14.55 20 235 25.95 11.55
c4t10d0 6.91 1.42 20 386 3.42 7.99
c3t11d0 3.40 2.41 9 186 6.62 11.39
c4t12d0 14.83 0.50 94 2730 0.01 2.50
c3t13d0 5.27 0.50 42 1503 0.01 3.06
c4t8d0 0.44 0.50 1 18 0.00 4.59
c3t9d0 0.12 0.50 0 4 0.00 6.95
c4t14d0 3.35 302.70 25 467 199.75 6.78
c7t5d0 0.14 0.50 0 3 0.00 4.99
c7t8d0 0.06 1.07 0 3 1.76 8.51
c7t9d0 0.05 1.14 0 3 1.27 7.13
c7t10d0 1.09 2.64 2 33 4.05 8.36
c7t11d0 0.54 2.91 2 26 3.17 6.09
c7t12d0 0.00 0.50 0 0 0.00 3.10
10:30:00 c2t0d0 10.14 12.98 28 302 25.06 11.42
c2t1d0 6.46 14.55 20 235 25.95 11.55
c4t10d0 6.91 1.42 20 386 3.42 7.99
c3t11d0 3.40 2.41 9 186 6.62 11.39
c4t12d0 14.83 0.50 94 2730 0.01 2.50
c3t13d0 5.27 0.50 42 1503 0.01 3.06
c4t8d0 0.44 0.50 1 18 0.00 4.59
c3t9d0 0.12 0.50 0 4 0.00 6.95
c4t14d0 3.35 302.70 25 467 199.75 6.78
c3t15d0 2.93 282.66 23 442 197.91 7.10
c7t0d0 1.12 1.44 2 40 2.12 8.45
c7t1d0 0.72 1.50 2 35 1.40 6.17
c7t2d0 0.75 0.53 2 30 0.10 5.77
c7t3d0 0.51 0.53 1 23 0.07 4.91
c7t4d0 0.14 1.23 0 3 1.00 5.43
c7t5d0 0.14 1.27 0 3 1.37 5.69
c7t8d0 0.06 3.20 0 3 3.38 7.36
c7t9d0 0.05 3.51 0 3 3.72 6.98
c7t10d0 0.91 1.07 2 27 1.16 7.21
c7t11d0 0.47 1.20 1 21 1.14 5.47
10:45:00 c2t0d0 14.87 13.57 41 480 38.09 13.71
c2t1d0 10.15 15.01 32 339 39.20 13.08
c4t10d0 4.96 0.84 11 264 1.61 8.64
c3t11d0 2.65 0.97 5 146 1.97 10.83
c4t12d0 1.91 0.50 3 51 0.00 7.32
c3t13d0 0.60 0.50 1 10 0.00 15.31
c4t8d0 2.75 0.50 10 208 0.00 3.13
c3t9d0 0.39 0.50 1 55 0.01 6.76
c4t14d0 3.21 129.44 21 363 92.96 6.80
c3t15d0 2.83 156.61 15 277 113.33 8.90
c7t0d0 0.79 2.87 2 26 5.19 9.22
c7t1d0 0.54 3.12 1 24 4.14 7.11
c7t2d0 0.66 0.55 1 16 0.20 7.05
c7t3d0 0.49 0.55 1 14 0.09 5.33
c7t4d0 0.13 0.88 0 3 0.57 5.00
c7t5d0 0.12 0.92 0 3 0.68 5.27
c7t8d0 0.04 1.41 0 2 2.42 8.52
c7t9d0 0.04 1.42 0 2 1.95 7.74
c7t10d0 1.08 2.02 2 28 2.43 8.10
c7t11d0 0.57 2.27 2 24 2.29 5.73

Average c2t0d0 18.68 9.53 73 1480 14.71 8.37
Average c2t1d0 12.09 12.96 47 1020 17.44 9.05
Average c4t10d0 6.54 2.06 20 515 5.22 7.65
Average c3t11d0 3.38 3.18 11 325 8.36 9.17
Average c4t12d0 2.97 0.50 15 448 0.01 2.70
Average c3t13d0 1.01 0.51 6 237 0.02 3.44
Average c4t8d0 2.58 2.18 36 926 2.99 2.32
Average c3t9d0 1.41 8.55 7 463 12.21 8.56
Average c4t14d0 2.56 145.05 14 383 104.12 7.66
Average c3t15d0 2.13 143.87 13 358 103.39 7.59
Average c3t2d0 0.53 0.50 16 1171 0.00 0.66
Average c7t0d0 1.08 2.08 3 52 3.01 7.33
Average c7t1d0 0.58 2.71 2 37 3.06 6.33
Average c7t2d0 0.77 0.55 2 24 0.15 6.28
Average c7t3d0 0.54 0.56 1 19 0.09 5.21
Average c7t4d0 0.14 0.78 0 4 0.51 5.38
Average c7t5d0 0.13 0.80 0 3 0.56 5.47
Average c7t8d0 0.10 2.30 0 6 3.10 7.26
Average c7t9d0 0.07 2.64 0 5 3.12 7.36
Average c7t10d0 1.10 1.34 2 38 1.64 6.79
Average c7t11d0 0.53 1.64 2 26 1.53 5.35
Average c7t12d0 0.02 3.60 0 14 1.10 3.96

Where should I look to see whether we have a disk or a CPU bottleneck?

Many Thanks!!!
Kind Regards,

Cris

[Peter Nikitka]

Hi,

the report of CPU-activity is provided by
sar -u

mfG Peter

[Hein van den Heuvel]

This is a relatively new system right?
Was it sized correctly, or perhaps undersized?
Do you have something to compare against, that is.. was this an upgrade from a prior PA-risc solution perhaps?

You write 1 CPU. That's the PA 8900 right? 800 Mhz or 1 Ghz?
1 core enabled only?
Maybe it is time to activate a second core or even add a second cpu, as there is room to grow in the box.

Personnaly I do not like single core servers ever. 2 cores seems a much better deal to me!

Just eyeballing what you shown sofar, I notice a good amount of memory and disks which may very well need a whole cpu core to drive.

Hope this help some,

Hein van den Heuvel
HvdH Performance Consulting.