FC10 performance

[Kirill Cherkashin]

Hello all,
we have a problem with performance of oracle database on HP-UX 11i,fc10 with 6x36Gb disks in mirror. CPU and memory utilization is ok. However, sar -d output shows that %busy always very close to 100%. I do believe that we have wrong distribution of oracle files among our filesystems(redo logs together with datafiles,etc). So,could someone give some ideas how to properly put oracle files?
listing of files and sar output included.

[Keely Jackson]

Hi

I don't know much about oracle but we tend to split data and logs in separate volume groups and it works fine.

We had problems similar to yours where cpu and memory were fine but sar disk utilisation was showing 100%. This turned out to to be a patch problem, so something else to check is that you have all the latest patches installed.

Cheers
Keely

[harry d brown jr]

One, as stated, check your patches.

Using "sar" to make career ending decisions is not a tool I would use. Get (purchase) or use glance/measureware (if you already own it) to determine what your potential IO issues are.

TO determine if you need to "re-distribute" some Oracle files, one would need:

(1) bdf
(2) vgdisplay -v vg*
(3)ioscan -fn
(4) a drawing or explanation on how the host is connected to the FC10, and what kind of host we are talking about, the kind of Fibre Channel card being used, the slot it's in, the patch levels, firmware revisions, ...
(5) transaction volumes
(6) release of Oracle

live free or die
harry

[Kirill Cherkashin]

Harry,
we have glance installed,but i'd rather prefer to run sar because i'm using ssh client which doesn't support hpterm emulation. anyway, glance shows along with sar that we have 100% disk bottleneck on /oradata02 volume.
Sorry, I forgot to mention. this's L3000 box with 4Gb ram,2x550MHz cpu,oracle 8.1.7. All another things are standard stuff,two fiber channel cards,8 FC disks evenly distributed between adapters,more or less recent patch and firmware level,etc.
bdf and ioscan -fnC disk output included.

[harry d brown jr]

Can you post a

vgdisplay -v vgora

and according to your ioscan, you ONLY have ONE Fibre Channel cable to your FC10.

And it says you have EIGHT (8) disks in the FC10 on 0/10/0. Two are 18GB drives and 6 are 36GB drives.

On 0/12/0 it shows you have TWO disks, each 36GB's. It appears that the device on 0/12/0 is another FC10??

When you said the disks were balanced between Fibre Channel adapters, what did you mean?? Because any Fibre Channel port on the FC10 can see any disk on the FC10. It doesn't matter if you are using the Primary ports or the Secondary. I have one FC10 with THREE different systems attached to it - two using the primary ports and one using one of the secondary ports.

Are you using striping? - can you post lvdisplay's of th lv's in vgora ??

live free or die
harry

[Keely Jackson]

Hi

Check you have these fibre channel patches installed otherwise you can get problems.

PHKL_23939 PHKL_25769 PHKL_25938

Keely

[Kirill Cherkashin]

sorry, i lost couple of lines for bdf output. cut&paste problem.;)
no,we don't use striping at the moment. but we are going to do it. lvm or veritas should be determined in the nearest future.
also, i wrote wrong word balancing between different fiber adapters. in fact there are just alternate links.
anyway,primary links really evenly distributes between adapters.;-)

[harry d brown jr]

Kirill,

What I see is that you have two fibre channel adapters into a single FC10.

The bottle necks are as follows:

(1) a single FC10
(2) a single bus on the FC10 (can't change that)
(3) mirroring is occuring within same FC10, doubling the IO rate!!

Other questions:
(1) What are the 18GB disks used for??
(2) If they are in use, could their IO rates be interfering with the IO rates of the other disks?

Potential solutions:
(1) stripe - but first need to upgrade VxFS to latest version.
(2) another FC10 with two more fibre channel cards - mirror between two FC10's
(3) fall back to 18GB drives, which I think are faster than the 36GB drives - plus increases the number of spindles
(4) va7400??

live free or die
harry

[John Palmer]

Rather than look first to a hardware solution, I'd be checking why the database is doing so much I/O. I'd guess from your filenames that you've got a 20Gb tablespace USR? on /oradata02 whereas the indexes are on /oradata03.

Has this database ever performed well? Is it new or has it been migrated to the present hardware? I suspect that you have some very unfriendly SQL that's doing full table scans - get your DBA to check.

Disk striping will help but sorting out the SQL and/or adding appropriate indexes will be much more beneficial.

I'd also recommend glance, the character based version works perfectly well with vt100 type emulation.

Regards,
John

[Dave Chamberlin]

Given that you only have three spindles - the distribution looks ok - you have at least separated index datafiles from table datafiles and your redo logs are spread apart (so you wont have contention during log switches). But you really want several smaller disks - not a few larger disks. Are you running in archive mode? If so there is bound to be some contention if they are being written on the same device as datafiles. It depends somewhat on how many transactions your database is doing. Most DBA handbooks have some useful information on this subject.