topic Re: Is the disk performance bottleneck? in Operating System - HP-UX

Is the disk performance bottleneck?

Ryan Gu — Wed, 31 Jan 2007 03:36:16 GMT

Many end customs said the system response is slow. So I checked the system. And find these.

CPU:
LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
4.88 28.7% 0.0% 8.9% 62.4% 0.0% 0.0% 0.0% 0.0%

Memory: 44160K (29544K) real, 615912K (44932K) virtual, 9528K free
Total Memory: 1GB

# sar -d 2 100

HP-UX a500 B.11.00 U 9000/800 01/31/07

10:36:16 device %busy avque r+w/s blks/s avwait avserv
10:36:36 c1t15d0 48.00 0.66 132 12224 5.90 19.64
c3t15d0 42.50 0.58 172 15848 5.53 10.97
10:36:38 c1t15d0 65.00 0.58 218 18471 5.55 16.79
c3t15d0 60.50 0.54 225 20729 5.07 13.63
10:36:40 c1t15d0 94.50 0.52 274 27362 5.59 18.62
c3t15d0 90.50 0.51 330 33212 5.13 12.60
10:36:42 c1t15d0 98.50 0.65 277 26501 6.44 21.26
c3t15d0 92.50 0.57 352 32979 5.84 13.51
10:36:44 c1t15d0 60.00 0.54 198 16419 5.79 11.84
c3t15d0 57.00 0.52 207 16778 5.58 10.17
10:37:16 c1t15d0 73.50 0.56 236 21520 5.74 15.73
c3t15d0 69.00 0.53 254 24000 5.74 12.66
10:37:18 c1t15d0 99.00 0.60 256 25006 6.01 23.86
c3t15d0 96.50 0.53 351 34352 5.29 13.43
10:37:20 c1t15d0 99.00 0.63 272 25649 5.82 21.43
c3t15d0 94.50 0.55 337 32079 5.36 14.47
10:37:22 c1t15d0 87.00 0.76 243 21523 6.65 19.29
c3t15d0 84.00 0.66 264 23164 5.76 15.34
10:37:24 c1t15d0 26.50 0.51 98 7475 5.61 8.11
c3t15d0 25.00 0.51 78 7104 5.63 10.53
10:37:26 c1t15d0 1.50 0.50 2 34 5.07 8.13
c3t15d0 1.00 0.50 1 16 7.58 6.23
10:37:28 c1t15d0 0.50 0.50 3 43 1.25 6.12
c3t15d0 0.50 0.50 2 40 1.48 6.14
10:37:30 c1t15d0 2.50 0.50 5 63 3.25 6.71
c3t15d0 1.50 0.50 2 40 3.73 7.57

Is the disk performance bottleneck?

BTW, the disks are all internal.

Re: Is the disk performance bottleneck?

Mridul Shrivastava — Wed, 31 Jan 2007 03:43:43 GMT

Performance issues are a little bit tricky to handle ;)

Here CPU, Memory and disk all seems to be ok.. sar o/p is quite normal as there were disk i/o going on that time as u can see when %busy is high its high for both the disk...

How customer says system performance is slow? is it slow at any particular time or with any perticular utility/report/application..

It could be because of network, I understand all users are remote so that could also be a issue...

you need to collect these data at the time of normal and degraded performance then compare to narrow down the issue

Re: Is the disk performance bottleneck?

Ryan Gu — Wed, 31 Jan 2007 03:49:39 GMT

I find all disks %busy is near by 100%.This machine is running a Oracle DB.

Which data should I collect to find the bottleneck?

BTW, does the output of top '9528K free' means the free memory is 9528k(9.5MB)?

Re: Is the disk performance bottleneck?

Ganesh Balraman — Wed, 31 Jan 2007 03:50:01 GMT

Dear Sir,

It does not seems to be issue with present H/w Availability , might problem with other parameters like(apps or db or network or kernel parms).

Regds
B.Ganesh

Re: Is the disk performance bottleneck?

Oviwan — Wed, 31 Jan 2007 04:00:36 GMT

Hey

We have just the same problem. the column avwait is also high.

perhaps there are some new JOBs in Oracle who reads and writes a lot?

Regards

Re: Is the disk performance bottleneck?

Frank de Vries — Wed, 31 Jan 2007 04:07:19 GMT

Jason

Indeed the Disks are the slowest in the chain, but that does not necessary mean they are always by definition the culprit in contention issues.

When you run Oracle there are a few other issues to consider:

Disk i/o related issues:
1) Are their other heavy queries running (or runaway queries that hog up resources)
2) Is there an Online backup running ?
Lock issues:
3) Are there issues with record or table locking (i.e. few users trying to access the same stuff)
4) Is Oracle wel configured for OLTP ,
sufficient buffer cache, redologs big enough
are checkpoints wel balanced etc..
5) What changed lateley ? Server side, pc -side, application side etc .

This is just a quick checklist, but this topic is an art on its own :)

A possible test you can do:)
Is to run in in a dosbox on the pc a sqlplus session a typical query the users use (behind the interface) and measure response time yourself.
Then run this locally on the server and check
if the difference is significant or does response vary then you know something is not right.

good luck

Re: Is the disk performance bottleneck?

Mridul Shrivastava — Wed, 31 Jan 2007 04:09:33 GMT

I would suggest to check the kernel parameters, are they tuned as per the oracle requirement.
Is this setup was working fine earlier or this is a new setup...
Normally its not recommended to keep the database on the local disks.
can u please post the OS version and kernel parameters value , system currently have..

Re: Is the disk performance bottleneck?

Steven E. Protter — Wed, 31 Jan 2007 04:21:59 GMT

Shalom,

No disk is 100% busy.

Commonly if there are disk bottleneck's a disk will have 100% busy and long avwait numbers, larger than displayed in this report.

You disks are utilized with several in the 90% range as far as utilization.

You may wish to look at other issues concerning processes waiting for i/o.

A process waiting for I/O can do nothing else util it receives that I/O. This situation can be caused by poorly written software.

http://www.hpux.ws/system.perf.sh

SEP

Re: Is the disk performance bottleneck?

Gerardo Mora — Thu, 01 Feb 2007 11:04:33 GMT

Hi,

Could you post the output of
> swapinfo -ta
> ioscan -fn