HPUX 10.0 Disk issue

[MIDAS]

Apologies - quite new to this so forgive me...

I have inherited a UNIX HP9000 system running HPUX 10.0 to support.

A mirrored disk failed and was promptly replaced. I can see the disk in /dev/rdsk but using the vgsync command I cannot resync it - I get the following:-

grvan01[] # vgsync /dev/vg01
Resynchronized logical volume "/dev/vg01/lvol1".
Resynchronized logical volume "/dev/vg01/lvol2".
vgsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
vgsync: Couldn't resynchronize logical volume "/dev/vg01/lvol4".
vgsync: Couldn't resynchronize volume group "/dev/vg01".

lvol4 represents a 6GB volume, made up from 2 disks - a 2GB disk and a 4GB disk which was originally mirrored onto like for like (a 2GB and 4GB disk).

It was the 2GB of the mirrored portion that failed.

This is the extent of what I have done:-

grvan01[lvmconf] # vgcfgrestore -n /dev/vg01 /dev/rdsk/c1t4d0
vgcfgrestore: Couldn't query physical volume "/dev/dsk/c1t4d0":
The specified path does not correspond to physical volume attached to
this volume group
Volume Group configuration has been restored to /dev/rdsk/c1t4d0

grvan01[lvmconf] # vgchange -a y /dev/vg01
Volume group "/dev/vg01" has been successfully changed.

grvan01[lvmconf] # lvsync /dev/vg01/lvol4
lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg01/lvol4".

grvan01[lvmconf] # vgsync /dev/vg01
Resynchronized logical volume "/dev/vg01/lvol1".
Resynchronized logical volume "/dev/vg01/lvol2".
vgsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
vgsync: Couldn't resynchronize logical volume "/dev/vg01/lvol4".
vgsync: Couldn't resynchronize volume group "/dev/vg01".

Can someone please help me out here...? Any suggestions greatly recieved - but bear in mind this is a critical system and I have nothing dev wise to try commands on first :-)

[Paula J Frazer-Campbell]

Hi

Did you split the mirror before repalcing the disk?

Paula

[Paula J Frazer-Campbell]

HI

This will help:-


http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=219549


Paula

[Robert-Jan Goossens]

Hi,

Take a look at this doc,

Document description: Removing a "ghost disk"
Document id: DFQKBCA00000489

http://www5.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000062687480

Regards,
Robert-Jan

[MIDAS]

Paula,

No - I did not remove the mirror before replacing the drive....

Is there a way I can do this in its current configuration?

Thanks
-Darren

[Robert-Jan Goossens]

Darren,

Can you open the link I posted ?

[MIDAS]

Robert,

It doesnt appear to be a ghosted disk.... thanks anyway :-)

My concern is at the moment vg01/lvol4 is not mirrored...

lvol4 is a 6gb volume consisting of a 2GB disk and a 4GB disk. The resultant volume is 85% utilised.

The mirror also consists of a 2gb and a 4gb disk - it was the 2gb disk that failed....

The disk has been replaced (c1t4d0) - you can see from above that the vgcfgrestore complains at first it couldnt query the volume, then reports that the Volume group configuration has been restored....

I am assuming that I need to fully break the mirror for lvol4, but am uncortable doing this becuase I'm unsure of the next steps...

If I break the mirror, will I need to extend the "mirrored" 2gb and 4gb disks to form a 6gb volume that I can use for the mirror...

Apologies if this doesn't make much sense to you!

Thanks
_Darren

[Robert-Jan Goossens]

Hi Darren,

no problem and yes it makes sense to me. (:-)

Maybe the word ghost is not correct. but the doc I mentioned does give you the steps you need. The disk has been changed without the Knowledge off LVM.

well here are the steps,

For every lvol in VG01 which has a copy on the failed disk do

# lvreduce -m 0 -k /dev/vg01/lvol? /dev/dsk/cxtydz

# mv /etc/lvmtab /etc/lvmtab.old

# vgscan â v

# vgchange -a y vg01

# vgreduce -f vg01 /dev/dsk/cxtydz

add the new disk
# vgextend vg01 /dev/dsk/cxtydz

# lvextend -m 1 /dev/vg01/lvol? /dev/dsk/cxtydz

Kind regards,
Robert-

[MIDAS]

Thanks Robert,

But I'm still concerned about it!! :-)

lvol4 is 6GB which is made up of the following disks:-
c4t10d0 (2gb)
c1t12d0 (4gb)

and the 'mirrored' disks are as follows:-
c1t4d0 (2gb) (replaced)
c0t12d0 (4gb)

Im assuming that the fact that lvol4 is extended over 2 disks is irrelavant when talking about mirroring (?), following your commands I only need

# lvreduce -m 0 -k /dev/vg01/lvol4 /dev/dsk/c1t4d0 (to take the 'failed' disk out of lvol4 - but won't I also need to take out c0t12d0 too?)

# mv /etc/lvmtab /etc/lvmtab.old (move lvmtab to keep a safe copy!)

# vgscan -a -v (scan hardware to recreate lvmtab)

# vgchange -a y vg01 (reactivate the vg01 group with read/write access)

# vgreduce -f vg01 /dev/dsk/c1t4d0 (take failed disk out of vg01)

# vgextend vg01 /dev/dsk/c1t4d0 (add it back in again)

# lvextend -m 1 /dev/vg01/lvol4 /dev/dsk/c1t4d0 (use c1t4d0 to mirror lvol4 - but what about c1t12d0??)

Can you see my confusion here (once again - sorry for being a newbie). Its the fact that lvol4 is extended over 2 disks thats stumping me, and 2 additional disks are used for the mirror - do I only need to perform the above operations on c1t4d0 or do I need to perform them on c0t12d0 also??).

Since this is a live system I will need to book in "down time" should the above cause any disruption... Should I need to do this or can it be done whilst users are using it??

Many Thanks for your help
-Darren

Thanks
-Darren

[Robert-Jan Goossens]

Hi Darren,

See the "confusion", in my opinion you should remirror the complete logical volume (both disks) before LVM will understand what happened.

About the downtime. If you can get some downtime I would create a good backup off the filesystem before I would start this procedure. Normaly this can be done while users are working but as you stated "this is a critical server" I would take NO change at all. Furthermore as this is a version of HPUX that is out of support, I would not risk this server without a good backup.

Kind regards and good luck.

Robert-Jan

[Robert-Jan Goossens]

Hi Darren,

Could you give me an update ?

Robert-Jan

[MIDAS]

Hi robert,

I swapped out the affected disk with a new identical disk.

I attempted to resync vg01 and all seemingly was going well.... then towards the end I got the familiar 'vgsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error'

Looking at the stale extent, it appears in exactly the same block as the last disk:-

00506 /dev/dsk/c4t10d0 00506 current /dev/dsk/c1t4d0 00506 current
00507 /dev/dsk/c4t10d0 00507 current /dev/dsk/c1t4d0 00507 current
00508 /dev/dsk/c1t12d0 00000 current /dev/dsk/c0t12d0 00000 current
00509 /dev/dsk/c1t12d0 00001 current /dev/dsk/c0t12d0 00001 current
00510 /dev/dsk/c1t12d0 00002 current /dev/dsk/c0t12d0 00002 current
00511 /dev/dsk/c1t12d0 00003 current /dev/dsk/c0t12d0 00003 current
00512 /dev/dsk/c1t12d0 00004 current /dev/dsk/c0t12d0 00004 current
00513 /dev/dsk/c1t12d0 00005 current /dev/dsk/c0t12d0 00005 stale
00514 /dev/dsk/c1t12d0 00006 current /dev/dsk/c0t12d0 00006 current

You'll notice that its the c0t12d0 disk block 0005 that is reporting as stale.

A disk replacement hasn't sorted it out, do you have any suggestions?

Thanks
-Darren

[Robert-Jan Goossens]

Hi Darren,

Still problems ?

http://www5.itrc.hp.com/service/cki/docDisplay.do?hpweb_printable=true&docLocale=en_US&docId=200000062687420

Date: 4/25/01
Document description: Troubleshooting Stale Mirrored Extents
Document id: DFQKBCA00000210

Robert-Jan

[Mic V.]

Darren,

Apologies, you sound like you know what you're doing, but you did say this was new.

HP-UX 10.0 is no longer supported; in fact, no HP-UX 10-anything is. I'm sure you're stuck on 10 if you haven't upgraded by now, but I just wanted to let you know.

I know it means HP won't make any more software fixes. I'm not sure what it means in terms of them helping you with questions.

--Mic