Disk Bottle neck

[George_14]

I've a disk bottleneck issue.
I've attached the result of sar -d.
When I used Glance, on the system snap shot I'm getting the following msg
SYMPTOM Disk_Bottleneck IS 84.7%

I've created another swap space on the Volume group. CPU,Mem etc usage is okay.
How can I resolve this issue.

[A. Clay Stephenson]

Hi,

The next step is to do a sar -b and let's look at buffer cache hit rates. What is your mmain application? If it is Oracle, are you using raw I/O?

Clay

[Jim Turner]

George,

Here is some general advice to avoid disk bottlenecks:

1. Stripe heavily used LV's across spindles which are fed by different SCSI controllers which are, in turn, fed by different bus adapters.

2. If you are using any form of array with two array controllers, make sure you choose your primary and alternate PV paths for your VG's such that the I/O (during normal ops) is spread evenly across both array controllers.

3. For database applications, it's a good idea to keep these types of data on completely separate spindles to reduce the liklihood of thrashing: data, index, archive log, redo log, roll-back segment.

4. Never forget your six P's: Prior Planning Precludes Piss-Poor Performance.

Cheers,
Jim

[George_14]

Oracle is the main application. One more thing I've noticed is in the vxfs option Full integrity for allmeta and user data(lowest performance) is selected. I tried cahnging it to Full integrity for critical metadata, but it comes to the previous selection.I've attached sar -b result.

[Vibert Hart]

From the sar stats it seems as if load balancing is on of the issues. You may need to relocate some of your datafile from disks that are showing high activity with the objective to spread the i/o. I am assuming you know how to rename(relocate Oracle datafiles)

You may also want to add another I/O card with good transfer rates say 80MPps and up.

Vibert

[A. Clay Stephenson]

Without knowing your Oracle disk organization and the amount of memory, it's difficult to make precise suggestions. I do note that you have high block read rates but low logical read rates indicating that you are using raw/io or mincache=direct,convosync=direct vxfs mount options. In that case, you need to increase the buffers in the SGA and decrease the amount of UNIX buffers. If you are using dynamic buffer cache, I would turn it off by setting bufpages to a non-zero value.
Assuming you have a decentg amount of memory,
I would start at about bufpages=25600 (100MB); I would certainly not exceed 320MB. In your case the memory is more precious in the SGA so you can increase buffers there.

Hope this helps, Clay

[George_14]

How do I Increase the bufpages?

[James R. Ferguson]

Hi George:

'bufpages' is a kernel parameter. You can (should) use SAM to adjust it. Go to SAM -> Kernel Configuration. SAM has built-in help for each kernel tunable, but you can get an equally good overview and/or browse safely, here:

http://docs.hp.com/hpux/onlinedocs/os/KCparams.OverviewAll.html

In part, you will note from this document that:

"If bufpages is zero at system boot time, the system allocates two pages for every buffer header defined by nbuf. If bufpages and nbuf are both zero, the system enables dynamic buffer cache allocation and allocates a percentage of available memory not less than dbc_min_pct nor more than dbc_max_pct , depending on system needs at any given time."

...JRF...

[Vincent Fleming]

George, You don't mention what you've got for disk hardware. If it's a RAID array, you may be able to add cache memory or more disks to help performance, or upgrade to a newer array.

If it's not a disk array, you might consider using one.

[Volker Borowski]

Hello George,

You have four busy disky, four controllers, but the busy disks are all attached to one controller.
If you can not move the disks, may be you can reorganize some lvols vice-versa, so you make better use of the controllers.

Now I hope the controllers are the same, because if the big one is a fast-wide and the others are SE, forget it.

Volker