Replacing a mirrored disk in the root volume group

[Stefan Schulz]

Hi everybody,

Disk_em showed me a faulty disk on my RP3440. This disk is the mirror disk of the root volume group. Please help me to confirm that I do have the correct steps in mind to replace the disk. English is not my native language so I am not 100% confident that I did understand everything correct.

I do have the document â When Good Disks Go Badâ from HP. With the mentioned methods I confirmed that the disk is faulty, that the disk is mirrored and that the disk is in the root volume group (see attached textfile).

Also I donâ t have LVM OLR available, so I have to do it the hard way.

The faulty disk is on /dev/dsk/c2t1d0

These are the steps that I plan to perform

1. remove the mirror copy of all logical volumes with
lvreduce â m 0 â A n /dev/vg00/lvol1 /dev/dsk/c2t1d0
lvreduce â m 0 â A n /dev/vg00/lvol2 /dev/dsk/c2t1d0
lvreduce â m 0 â A n /dev/vg00/lvol3 /dev/dsk/c2t1d0
lvreduce â m 0 â A n /dev/vg00/lvol4 /dev/dsk/c2t1d0
lvreduce â m 0 â A n /dev/vg00/lvol5 /dev/dsk/c2t1d0
lvreduce â m 0 â A n /dev/vg00/lvol6 /dev/dsk/c2t1d0
lvreduce â m 0 â A n /dev/vg00/lvol7 /dev/dsk/c2t1d0
lvreduce â m 0 â A n /dev/vg00/lvol8 /dev/dsk/c2t1d0
lvreduce â m 0 â A n /dev/vg00/lvol9 /dev/dsk/c2t1d0

2. remove the disk from the volume group
vgreduce vg00 /dev/dsk/c2t1d0

But this might fail as the disk is unavailable so I might have to do the following

Vgreduce â f vg00
Mv /etc/lvmtab /etc/lvmtab.save
Vgscan â v

But I am not sure if I have to perform these fist two steps as â When Good Disks Go Badâ says at the end of page 17:

â ¢ If pvdisplay shows PV status as unavailable, or if pvdisplay fails to print the status, use ioscan to determine if the disk can be accessed at all. If ioscan reports the disk status as NO_HW on all its hardware paths, then you can simply remove the disk. If ioscan shows any other status, halt LVM access to the disk by deactivating the volume group.

And ioscan does show â NO_HWâ for /dev/dsk/c2t1d0

3. replacing the disk.
As this disk is merely hot plugable I will shut down the system. The faulty disk with the H/W path 0/1/1/0.1.0 is the one marked as â 1â on the harddisk tray. Really hope this is correct!

4. initialize disk for LVM

vgcfgrestore â n vg00 /dev/dsk/c2t1d0

5. reenabling LVM access to the disk

vgchange â a y vg00

6. set up the mirror again (Appendix A)

insf â e â H 0/1/1/0.1.0
pvcreate â B /der/rdsk/c2t1d0
vgextend /dev/vg00 /dev/dsk/c2t1d0
mkboot /dev/rdsk/c2t1d0
mkboot â a â hpux â lqâ /dev/rdsk/c2t1d0

set up the mirrors for each logical volume:

lvextend â m 1 /dev/vg00/lvol1 /dev/dsk/c2t1d0
lvextend â m 1 /dev/vg00/lvol2 /dev/dsk/c2t1d0
lvextend â m 1 /dev/vg00/lvol3 /dev/dsk/c2t1d0
lvextend â m 1 /dev/vg00/lvol4 /dev/dsk/c2t1d0
lvextend â m 1 /dev/vg00/lvol5 /dev/dsk/c2t1d0
lvextend â m 1 /dev/vg00/lvol6 /dev/dsk/c2t1d0
lvextend â m 1 /dev/vg00/lvol7 /dev/dsk/c2t1d0
lvextend â m 1 /dev/vg00/lvol8 /dev/dsk/c2t1d0
lvextend â m 1 /dev/vg00/lvol9 /dev/dsk/c2t1d0

update the root volume group information

lvlnboot â R /dev/vg00

checking if it is recognized as a mirrored boot disk

lvlnboot â v

So far the planned procedure. At the moment I am not quite sure about step 3, but I can do this with the â try and errorâ method.

But more important is that step 4 vgcfgrestore and 5 seem to conflict with step 6. I am pretty confused as the Gude is not clear for me at this points. Do I have to perform step 4, 5 and 6 or do I have to perform only step 6 as the faulty disk IS part of the root volume group?

Could someone please help me here?

Kind regards

Stefan

[RobinKing · Accepted solution]

Hi Stefan

I agree with you about steps 4,5 and 6. I've just checked the HP LVM manual, and once the disk has been replaced you should just be able to do the "vgcfgrestore -n vg00 /dev/dsk/c2t1d0", then the mkboot, then activate vg00 and then run "vgsync vg00"

[Torsten.]

Reducing a failed disk from a VG may be some difficult.
IMHO this is really not needed (based on my experience).

Is it showing NO_HW?
Is "unavailable" in vgdisplay?

Rip it out!
Check again with ioscan and vgdisplay.

Insert the new disk, vgcfgrestore the config, vgchange -a y vg00, make it bootable (mkboot) and sync the VG. You are done.

[Stefan Schulz]

Hi again,

so you both say that i have do all the steps 4 to 6. And as the disk is shown in ioascan with NO_HW that i dont have to remove it from the VG (as it is allready dead).

Thanks for clearing things up for me.

Kind Regards

Stefan

BTW: sorry for those special characters in the text. Most of them ment to be a "-". But i wrote this on a textprocesser for the spellchecking and copy & paste didnt work as expected.

[Torsten.]

These are the steps:

3. replacing the disk
4. initialize disk for LVM

vgcfgrestore -n vg00 /dev/dsk/c2t1d0

5. reenabling LVM access to the disk

vgchange -a y vg00

6. set up the mirror again (Appendix A)

mkboot -l /dev/rdsk/c2t1d0
mkboot -a "hpux -lq" /dev/rdsk/c2t1d0


vgsync vg00

lvlnboot -r /dev/vg00/lvol3
lvlnboot -b /dev/vg00/lvol1
lvlnboot -s /dev/vg00/lvol2
lvlnboot -d /dev/vg00/lvol2

lvlnboot -v

DON'T DO ANY create or extend commands!

[Stefan Schulz]

Hi again,

i have recieved my replacement disk and will install it end of this or beginning of next week. When i have the necessary timewindow available.

Thanks a lot for your help.

Kind Regards

Stefan

[Stefan Schulz]

Hi,

i have replaced the faulty disk and everything worked like it should. The new disk was recognized immediately and i have been able to restore the configuration and sync the volume group.

Everything without any problems.

Only i now have a red LED on the disk i didn't replace which makes me nervous.

Allthough syslog and stm tell me that everything is just fine, this red LED wont go out. I also cleared SL and CL in the MP but this red LED is still there.

This is on the disk which remained in the system (not the one i replaced) and the LED is constantly red (not flashing).

The only hint i could find was to power down the RP3440 completely, unplug the powercables for some time an then replug and restart everything.

The questions i would like to ask are:

- Do i have to worry about the red light or can i trust stm and syslog?

- How do i get rid of this red LED without powering down everything?

Thanks in advance

Regards

Stefan

[Torsten.]

Is the disk in question in the middle slot?

In this case the "light pipe" of the disk container may be the reason. It is pressing on the red LED - no need to worry, if the disk is working.

Call your HP support, they will give you the solution.

[Stefan Schulz]

No, the disk in question is the lower one in the slot labled "0".

The disk is working just fine, but even if the red light doesn't mean anything i would like to get rid of it. At least i would like to avoid dikussing the status of my server. (You know windows admins against Unix admins :-( )

Thanks

Stefan