topic Re: Mirror UX performance issue in Operating System - HP-UX

Mirror UX performance issue

john leavens_1 — Mon, 25 Apr 2005 23:19:56 GMT

I am running HP-UX 11.23 on a rx5670 server. The 2 internal disk are setup as RAID 1 using MirrorUX. When the disks are under load sar -d shows one disk has twice as many blks/s and the corresponding wait and service times twice as high long.

All volumes are mirrored with the exception of the swap volume. There is no paging occuring.

I have attached the output from sar -d.

Any ideas as to why this is?

Thanks

John Leavens

Re: Mirror UX performance issue

Steven E. Protter — Mon, 25 Apr 2005 23:26:10 GMT

What are the disk types?

If they are the same. Where are they on the scsi chain?

If you mirror across unlike disks you can cause performance issues.

You might also want to make sure that cables and terminations are good on the disk giving you trouble.

SEP

Re: Mirror UX performance issue

Trond Haugen — Tue, 26 Apr 2005 00:24:46 GMT

First of all; do mirror your swap!
If your swap volume goes bad you system is dead. Sure there are no data to preserve on the swap area. But you need the redundancy in case of a disk failure.

Do you see you problem during read or write (or both)?
How is you schedule set up(parallel or sequential)?
Is mirror_write_cache on or off?

Regards,
Trond

Re: Mirror UX performance issue

john leavens_1 — Tue, 26 Apr 2005 00:29:00 GMT

The disk are the the same, 73G 15K I believe.
They on the internal Ultra160 controllers.
They may be on different controllers, though.

Is there some way to tell where they are in the scsi chain from the hardware path?

See the attached ioscan.

Thanks,

John Leavens

Re: Mirror UX performance issue

Steven E. Protter — Tue, 26 Apr 2005 00:39:44 GMT

Based on your update, you should not be having this problem.

I suggest cstm, mstm or xstm and do a full check on the hardware.

I've heard varying opinions about mirroring swap. I did it in my last position and on my personal systems, but others believe its a bad idea.

I'd look for some kind of hardware issue causing this.

SEP

Re: Mirror UX performance issue

Naveej.K.A — Tue, 26 Apr 2005 00:45:06 GMT

Hi John,

They are on the same controller

0/0/2/0.0.0 and 0/0/2/0.2.0 confirms this.

Regards,
Naveej

Re: Mirror UX performance issue

john leavens_1 — Tue, 26 Apr 2005 00:51:53 GMT

Trond,

Thanks for the feedback on the swap.

The scheduling policy is set to parallel.

Write Cache is set on.

Mirror Consistency is set off.

John Leavens

Re: Mirror UX performance issue

Trond Haugen — Tue, 26 Apr 2005 01:43:19 GMT

The only reason I could think of would have been the system paging to that disk. But as you have no paging that can't be the problem.

If the system is a ftp server you may want to try switching the mirror write cache off. It could improve write speed. But may impact read.

Looking at your sar output I would expect the two disks in the mirror to have the same amount of r+w. But if you are doing maily reads it MAy be that one disk is "answering" more frequently than the other. (I'm on thin ice here) Could also be a HW issue as SEP points out.

Regards,
Trond

Re: Mirror UX performance issue

john leavens_1 — Tue, 26 Apr 2005 01:50:36 GMT

This vgroup contain the root file system and Oracle database redo logs.

The Oracle redo logs are creating most of the I/O. Most of the activity would be sequential writes to the same files. In Oracle I see high log_file_sync waits indicating the that the system is waiting to write to disk. That is the original problem that uncovered this issue.

JL

Re: Mirror UX performance issue

Duncan Edmonstone — Tue, 26 Apr 2005 02:17:24 GMT

Well it certainly isn't good practice to put application data in the root volume group - but that aside, if you still have the same problem when your swap lvol is mirrored then check that no other lvols are not mirrored.

Out of interest does vmstat show many page-outs occuring at the same time?

Do you have glance installed on this box? It will give a much better view of performance than sar does.

HTH

Duncan

Re: Mirror UX performance issue

Thayanidhi — Tue, 26 Apr 2005 02:21:12 GMT

Yes,
Both are in same controller instance 0(c0).

Also you can list all the initiators(SCSI controllers)using ioscan -fnC ext_bus.

from ioscan -fnC disk list all starting c0, are connected to one SCSI controller.

Regds
Tt