topic Re: Why one mirror disk is much busier than another one? in Operating System - HP-UX

Why one mirror disk is much busier than another one?

okyour — Thu, 14 Sep 2006 03:51:23 GMT

very stange,just see below the sar result. they are root disk and got mirrored.why c2t1d0 avserv so high?

15:03:19 c2t1d0 38.00 0.50 2 24 0.49 53.49
c3t0d0 1.00 0.50 2 24 1.03 12.07
15:03:20
15:03:21
15:03:22
15:03:23 c2t1d0 24.00 0.50 1 2 0.97 248.05
c3t0d0 1.00 0.50 1 2 0.96 9.89
15:03:24 c2t1d0 12.00 0.50 1 8 1.05 127.17
15:03:25
15:03:26
15:03:27 c2t1d0 10.00 0.50 1 8 1.08 99.67
c3t0d0 4.00 0.50 1 8 1.08 41.66
15:03:28
15:03:29 c2t1d0 16.16 0.50 1 8 1.06 164.48
c3t0d0 4.04 0.50 1 8 1.07 43.70
15:03:30
15:03:31 c2t1d0 24.00 0.50 5 48 1.00 185.45
c3t0d0 6.00 0.50 5 48 1.02 54.20

Re: Why one mirror disk is much busier than another one?

Ralf Seefeldt — Thu, 14 Sep 2006 03:59:05 GMT

Hello okyour,

2 questions:

1) is there SWAP-space on c2t1d0 , which is used extensively?

2) Are c2t1d0 and c3t1d0 of different age which may result in different speed or even c2t1d0 may have some defects, which require several reads or writes on some pionts. Does 'dd if=/dev/rdsk/c2t1d0 of=/dev/null bs=1024k' work corectly?

Bye
Ralf

Re: Why one mirror disk is much busier than another one?

Jaime Bolanos Rojas. — Thu, 14 Sep 2006 06:08:17 GMT

Okyour,

Other thing that you might want to thing about is the in lvm we mirror logical volumes not hard disks, maybe you also got some extra extra vg's on the other drive that might mike the drive works more than the other.

Regards,

Jaime.

Re: Why one mirror disk is much busier than another one?

Tim D Fulford — Thu, 14 Sep 2006 07:45:50 GMT

Hi

in parallel with the above...

1 - even if everything is mirrored exactly on each disk, and the disks are exactly the same. When mirrored disk pair are not too busy (say < 60% util). Then generally the primary disks services ALL reads and writes, the mirror ONLY does the writes. As the system gets busier you will see the mirror disk getting a fairer share of the load.
2 - You have not fully mirrored all LVs on the primary disk
3 - All LVs are mirrored, but are mirrored in a different order.
4 - The disks are not of theb same specification
5 - One disks is failing.

From my point of view I think your disk are in trouble as the service times are enourmous (99-248ms for c2t1d0 & 10-54ms for c3t0d0).

Check your syslog.log

Regards

Tim

Re: Why one mirror disk is much busier than another one?

Ralf Seefeldt — Thu, 14 Sep 2006 08:13:46 GMT

Hi again,

if you have LVM, please check vgdisplay -v . You get information whether or not all LVOLs are in sync. Some may be stale.

Bye
Ralf

Re: Why one mirror disk is much busier than another one?

Mark A. Smith — Thu, 14 Sep 2006 08:32:28 GMT

Make sure that there are no stale extents on the mirrored root drive, if there are any then the mirrored disk will look different when writing, I have had that happen in the past, so everyday I run the following script to check them and it sends the results to root's mail. Adjust the script for the LV's on your particular system.

#!/bin/sh
echo "" > /tmp/vg00_stales
echo "To merge, run: \n
vgsync /dev/vg00 to merge the whole volume group (vg)
or
lvsync /dev/vg00/lvolx (where 'x' is the logical volume number) \n
These CAN be run while the system is up and running. \n
If you run either of the above commands, then manually run '/ones/check_vg00_sync'
and check to see if there are still any stale extents, and if they persist, place
a hardware service call to HP immediately and advise then that we have a root disk
going bad." > /tmp/vg00_stales
echo "" >> /tmp/vg00_stales
echo "If properly synched, there should be nothing
below this line and NO ACTION is required:
-----------------------------------------" >> /tmp/vg00_stales
echo "" >> /tmp/vg00_stales
lvdisplay -v /dev/vg00/lvol1 | grep stale >> /tmp/vg00_stales
lvdisplay -v /dev/vg00/lvol2 | grep stale >> /tmp/vg00_stales
lvdisplay -v /dev/vg00/lvol3 | grep stale >> /tmp/vg00_stales
lvdisplay -v /dev/vg00/lvol4 | grep stale >> /tmp/vg00_stales
lvdisplay -v /dev/vg00/lvol5 | grep stale >> /tmp/vg00_stales
lvdisplay -v /dev/vg00/lvol6 | grep stale >> /tmp/vg00_stales
lvdisplay -v /dev/vg00/lvol7 | grep stale >> /tmp/vg00_stales
lvdisplay -v /dev/vg00/lvol8 | grep stale >> /tmp/vg00_stales
cat /tmp/vg00_stales | mailx -s "Checking merge status of root drives (vg00)" root@yourdomain.com

Re: Why one mirror disk is much busier than another one?

Bill Hassell — Thu, 14 Sep 2006 08:36:58 GMT

Actually, this is quite normal. The only time you will see both disks busy at the same time is when you are writing. In that case, the new record(s) must be written to all mirrors at the same time. Reading only needs one copy of the data so the primary disk will be the first choice. Once the primary disk gets overly busy with reads, the secondary may be used in parallel (exact rules are HP-UX version dependent).

Re: Why one mirror disk is much busier than another one?

Tim D Fulford — Thu, 14 Sep 2006 09:31:53 GMT

I think I'll underline my previous answer... and point out a more salient point

** YOUR ROOT DISK IS FAULTY AND COULD FAIL SOON **

Service times of 200ms should not exist on a good disk.

Tim

Re: Why one mirror disk is much busier than another one?

okyour — Thu, 14 Sep 2006 23:34:06 GMT

Thanks for all of your kindly reply. After I replace the disk,now all seems ok now