topic Re: Resync after failure problem in Operating System - HP-UX

Resync after failure problem

Igor I. Shulz — Sun, 15 Dec 2002 11:31:04 GMT

I have /dev/vg00 on /dev/dsk/c0t6d0 /dev/dsk/c2t6d0 with 8 lvolumes, each is mirrored by HP MirrorDisk/UX. I phisically detach one of the disks (/dev/dsk/c2t6d0). This member of vg00 goes to "unavailable". Now I attach the disk back, run ioscan, to see if it is CLAIMED (it is), and run vgsync /dev/vg00. But it fails:

# vgsync /dev/vg00
vgsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
vgsync: Couldn't resynchronize logical volume "/dev/vg00/lvol1".
vgsync: Couldn't resynchronize volume group "/dev/vg00".

But if I than reboot server, this pv goes online and resync procedure is completed as needed.

My question: how I can force to resync mirrored vg after returning broken disk to working state without rebooting?

Re: Resync after failure problem

Eugeny Brychkov — Sun, 15 Dec 2002 11:37:57 GMT

Igor,
after running ioscan try
vgchange -a y /dev/vg00
You know vg00 can not be deactivated, but can be reactivated
Hope this will help
Eugeny

Re: Resync after failure problem

Robert-Jan Goossens — Sun, 15 Dec 2002 11:51:03 GMT

Hi,

Here is copy of doc KBAN00000347 covers all possibilities, mirrored boot disk, non-mirrored boot disk, mirrored data disk, non-mirrored data disk, etc.

http://forums.itrc.hp.com/cm/components/FileAttachment/0,,0x677db47b9a27d6118ff40090279cd0f9,00.txt

Robert-Jan.

Re: Resync after failure problem

Igor I. Shulz — Sun, 15 Dec 2002 12:50:21 GMT

1. vgchange -a y /dev/vg00 doesn't help:

# vgchange -a y /dev/vg00
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c2t6d0":
Unknown error
Volume group "/dev/vg00" has been successfully changed.
#

2. KBAN00000347 is very usefull document but it didn't help me much: it stated that "reboot may not be necessary in case of hot-swap disk" (it's my case). And for resync I should run
# vgcfgrestore -n /dev/vg00 /dev/rdsk/cNdNtN
# vgchange -a y /dev/vg00

but it doesn't work:

# ioscan -fnC disk
Class I H/W Path Driver S/W State H/W Type Description
===========================================================================
disk 0 0/0/0/2/0.6.0 sdisk CLAIMED DEVICE HP 36.4GST336752LC
/dev/dsk/c0t6d0 /dev/rdsk/c0t6d0
disk 2 0/0/0/3/0.6.0 sdisk CLAIMED DEVICE HP 36.4GST336752LC
/dev/dsk/c2t6d0 /dev/rdsk/c2t6d0
................
# vgcfgrestore -n /dev/vg00 /dev/rdsk/c2t6d0
vgcfgrestore: IO error on Physical Volume device /dev/rdsk/c2t6d0: #
# vgchange -a y /dev/vg00
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c2t6d0":
Unknown error
Volume group "/dev/vg00" has been successfully changed.
#

But if I reboot server this disk is attached to VG00 and synced properly :-(

Re: Resync after failure problem

Robert-Jan Goossens — Sun, 15 Dec 2002 16:11:36 GMT

Take a look at next document

http://www4.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000063238152

Robert-Jan.

Re: Resync after failure problem

Rajeev Shukla — Mon, 16 Dec 2002 05:30:01 GMT

Does it always complain about /dev/vg00/lvol1.
Can you try to sync other logical volumes using "lvsync" command. If other logical volumes get synced and only /dev/vg00/lvol1 has problem, i sucpect the Disk. That PE might be bad and gets stale. Try this with other disk then.

Rajeev

Re: Resync after failure problem

T G Manikandan — Mon, 16 Dec 2002 05:55:09 GMT

What does diskinfo on that disk give

First do a
#diskinfo /dev/rdsk/c2t6d0

to check the disk is working fine returning the proper number of bytes.

Re: Resync after failure problem

T G Manikandan — Mon, 16 Dec 2002 06:05:56 GMT

What is the output of
pvdisplay -v /dev/vg00/c2t6d0

You should get the logical volumes that were under this disk.
Also
a lvdisplay of the logical volumes which were under c2t6d0.

lvdisplay -v /dev/vg00/lvol..

REvert

Re: Resync after failure problem

Igor I. Shulz — Mon, 16 Dec 2002 11:29:54 GMT

To Rajeev Shukla:
lvsync for other volumes doesn't work too:

# lvsync /dev/vg00/lvol3
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol3".
#

To T G Manikandan:

It seems disk works fine:
# diskinfo /dev/rdsk/c2t6d0
SCSI describe of /dev/rdsk/c2t6d0:
vendor: HP 36.4G
product id: ST336752LC
type: direct access
size: 35566480 Kbytes
bytes per sector: 512

pvdisplay shows:

# pvdisplay /dev/dsk/c2t6d0
--- Physical volumes ---
PV Name /dev/dsk/c2t6d0
VG Name /dev/vg00
PV Status unavailable
Allocatable yes
VGDA 2
Cur LV 9
PE Size (Mbytes) 8
Total PE 4340
Free PE 2602
Allocated PE 1738
Stale PE 79
IO Timeout (Seconds) default
Autoswitch On

# lvdisplay -v /dev/vg00/lvol1
--- Logical volumes ---
LV Name /dev/vg00/lvol1
....
Mirror copies 1
..
--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c0t6d0 38 38
/dev/dsk/c2t6d0 38 38

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 /dev/dsk/c0t6d0 00000 current /dev/dsk/c2t6d0 00000 stale
00001 /dev/dsk/c0t6d0 00001 current /dev/dsk/c2t6d0 00001 stale
........

Re: Resync after failure problem

Rajeev Shukla — Mon, 16 Dec 2002 11:41:38 GMT

Hi Iqor,
I see both the disks on a different SCSI bus. Can you confirm about the SCSI termination on the other bus of whose disk shows stale.
It could be issues with bad SCSI terminator or no termination.

Re: Resync after failure problem

Igor I. Shulz — Mon, 16 Dec 2002 11:52:25 GMT

Hmm. These are pre-installed by HP internal disks in our just delivered (a week ago) new rp8400 box. Is it possible that such famous and reliable company installed new disks in new system inproperly? Anyway I don't know how I can make sure that termination is set correctly. Besides - this is only problem concerning disk system that I have now on the system.

Re: Resync after failure problem

Rajeev Shukla — Mon, 16 Dec 2002 12:08:34 GMT

Though the disks are internal the buses will be terminated externally. Try swapping the terminator and see if the problem persistists with the same disk or goes to another disk which can confirm about the terminator.

Re: Resync after failure problem

harry d brown jr — Mon, 16 Dec 2002 13:42:08 GMT

Igor,

How about posting the output of

vgdisplay -v vg00

And you don't have a termination issue. You probably need to "force" the resync, but first I want to see the output from above.

live free or die
harry

Re: Resync after failure problem

Armin Feller — Mon, 16 Dec 2002 14:22:24 GMT

Hi,

perhaps there are entries in the bad block area. Is the 'bad block' relocation switched on for the lvols (lvdisplay)?

Please check if you can read from the disk without errors:

dd if=/dev/dsk/c2t6d0 of=/dev/null bs=1k

If the 'dd' fails, perhaps you have to re-add this disk to vg00.

Regards,
Armin

Re: Resync after failure problem

Igor I. Shulz — Wed, 25 Dec 2002 08:27:11 GMT

Wow!!!

This is an answer from Great Guru. Problem is gone!

Hello Igor,

Please see below reply on your problem with mirror disks.

For your case may to try solve problem by two ways:

1. to try spin-up inserted disk with next command (diskinfo, scsictl,
pvcreate), then vgsync.

but better

2. install next patch PHKL_28096.

Best regards,
Sergei