lvextend error when trying to mirror boot disk...

[Bryan D. Quinn]

I have replaced a boot disk mirror that had a stale extent. Everything is fine until I try to mirror everything back. On the first lvol I get the following error message:

lvextend: Couldn't re-synchronize stale partitions of the logical volume: Device offline/Powerfailed

When I lvdisplay the lvol that gets this message, the very first extent is stale on the mirror. All of the other extents on this lvol and all of the other lvols in VG00 are synced. So, I replaced the disk again thinking that I just got a bad refurb disk. Well what do you know, two more disks later and I still have the same problem. Any ideas on what could be causing this problem? My system is running fine on the boot disk and I am getting no errors from that disk, but could it be causing the problem somehow?

I know I can Ignite this box and that will probably take care of it, but I would rather figure out what is causing this issue. Any ideas would be greatly appreciated.

Thanks,
Bryan Quinn
UNIX Support Specialist
Cooper Power Tools - Lexington, SC

[Victor BERRIDGE]

Hi Bryan,
What box is it? how are the disks connected (separate controller or share the same)

Do you see anything is syslog.log (like scsi reset...)


All the best
Victor

[Miguel Hernández]

To my ocurrio something similar:

The problem did not have the replaced disc, the problem had the disc "good". I only needed to remove and to return it to put. But with the machine power off.

greetings

[Marvin Strong]

what type of system is it?

Here is a longshot.

Did you run pvchange -a y {device}

[Torsten.]

Did you get the stale extend on your original disk???

[Bill Hassell]

Start by testing the replacement disk with some reads:

dd if=/dev/rdsk/-whatever- of=/dev/null bs=512k

If this works OK, then the disk is alive and readable. Now did you use vgcfgrestore to initialize the new disk? You can't just stick in a new disk and run lvsync. The disk must be part of the volume group and needs VGID and other LVM written to the front end. Once you activate vg00 with the new disk, you'll need to run mkboot and then do vgsync or lvsync.

[Torsten.]

Be aware, if your vgcfgrestore returns

vgcfgrestore -n vg00 /dev/rdsk/c0t6d0
vgcfgrestore: Cannot restore Physical Volume "/dev/dsk/c0t6d0",
Detach the PV or detactivate the VG, before restoring the PV.


there are the new LVM patches in place and you have to use

pvchange -a y/n

see

http://docs.hp.com/en/5991-1236/When_Good_Disks_Go_Bad.pdf

(This is mandatory for every admin ;-)

[Bryan D. Quinn]

The system is a V-2600. The disks are in an A3312 enclosure. The mirror disk appears to be fine. There are no stale extents on the good (boot disk). The stale extent only shows on the mirror. I have done a dd to the mirror and it returns fine, no errors. I am not getting any errors in syslog.

Bill,
Here is the procedure that I used to replace the disk.

1. lvreduce -m 0 /dev/vg00/lvol# /dev/dsk/c0t11d0 (did this for all of the lvols)

2. lvdisplay -v /dev/vg00/lvol# (to verify all mirrors were broke)

3. pvdisplay -v /dev/dsk/c0t11d0 (to verify that there are no extents on this disk before removing it)

4. Pull out the old disk and replace with the new disk.

5. ioscan -fnC disk (to verify the new disk is visible.

6. diskinfo /dev/rdsk/c0t11d0 (to verify new diskinfo)

7. pvcreate -B /dev/rdsk/c0t11d0

8. vgextend /dev/vg00 /dev/dsk/c0t11d0

9. mkboot /dev/rdsk/c0t11d0

10. mkboot -a "hpux -lq" /dev/rdsk/c0t11d0

11. lifcp /dev/rdsk/c0t11d0:AUTO-

12. pvdisplay -v /dev/dsk/c0t11d0

13. lvextend -m 1 /dev/vg00/lvol# (to remirror each lvol)

Did I miss something? I have used this procedure a couple of times before without any issues...

-Bryan

[Bill Hassell]

Yes, your procedure will work OK. For boot disks, using vgcfgrestore saves a step or two, but unmirroring and removing the disk from the vg works just as well.

[Bryan D. Quinn]

I have just talked with a CE and they stated that they have 'spent the night' at a customer site for this very reason. Turned out to be a bad boot disk that was causing the issue. Here was the suggestion.

1. Go ahead and try to boot off of the alternate (mirror) of the boot disk, eventhough it shows one stale extent. If this works and no stale extents show up on the alternate that I booted off of, then replace the primary and re-mirror.

2. Replace the primary and Ignite the box.

So, we'll see what happens.

I will post an update with the scenario that solved the problem.

-Bryan

[Bryan D. Quinn]

Ok, here is an update.

I chose option 1 and booted off of the alternate. The boot was successful. The only problem is that the first extent on the alternate (which I booted off of) is still showing stale. Therefore, I get an error message everytime I try to reduce the mirror off of the primary boot disk. The error says that it can not reduce the mirror because the remaining disk has a stale extent. So, now I am stumped as what to do next.

Should I just throw in the towel and Ignite or is there some way to break that mirror. I read somewhere about the lvunstale tool used by HP engineers. Has anyone used this tool? If so, would it help in this situation?

Thanks,
-Bryan

[Siju Vadakkan]

check for bad blocks on both the disks...

echo 2400?20X | adb

[IT_2007]

Do you have stale extent in lvol1. Can you post first 30 lines of lvdisplay -v /dev/vg00/lvol1 ??

It seems your 1st logical extent on primary disk is bad. You can do this way to fix it.

1. Boot off with alt disk.
2. Split all logical volumes ( don't use lvreduce)
3. Boot of again using alt boot disk and check for any stale extents on lvol1. If you don't see any, then replace your bad primary disk.
4. restore logical volumes on to new disk.

[Arunkumar.B]

Hi Bryan,


The same scenario i faced issue with my superdome.

Stale shown in the secondry.Primary shows good but the problem was the primary disk.After replacing the primary disk things went fine.

My suggestion,

Remove the Primary Disk & boot using the Alternate disk.If it booted properly.Try to reduce using lvreduce using pvkey.
lvreduce -m 0 -A n -k /dev/vg00/lvol1 1
It should work.Otherwise there is one more tool i got from HP-Australia call lvunstale which will make ur stale showing in the Alternate as current.For that u have to boot the machine in maintanence mode using Alternate disk.

1)vgexport /dev/vg00
2)vgimport -v /dev/vg00 /dev/dsk/cxtydz(Alternae disk)
3)Run the lvunstale script for the particular lvol which will turn your Stale to current. Then Normal Procedure.


Finally Nothing Helps Restore thru Ignite.


Cheers
Arunkumar.B