topic Re: what-if hpux11.11 mirror disk is out and want to lvreduce? in Operating System - HP-UX

what-if hpux11.11 mirror disk is out and want to lvreduce?

Steve Post — Tue, 20 Jan 2009 16:14:32 GMT

This is a what-if question:
I have a junk computer with a mirrored boot disk.
primary /dev/dsk/c0t3d0
mirror /dev/dsk/c0t4d0
hpux 11.11
OLD K570 HP9000.

I walk up and yank out the mirror disk.
So disk /dev/dsk/c0t4d0 is dead in my experiment.
In advance, I ran lvdisplay -k -v /dev/vg00/lvol8 (and the rest of them).
So I know the k-number for c0t4d0 is 1. And the primary disk k-number is 0.

In this scenario, I would call HP hardware support. I would expect a spare disk in a few hours. But, what if I wanted to instead remove the dead disk from vg00? Is is possible? Is this a bad idea?

In WHEN-GOOD-DISKS-GO-BAD, I am told to run this....
lvreduce -A n -k -m 0 /dev/vg00/lvol8 1

This bombs out because of something with lvlnboot. lvmtab gets goofed up. And I restore from my ignite tape.

I would give the specific errors. But I don't want to waste another 2 hours restoring a dead computer from my ignite tape.

Maybe I should just run vgreduce -A N -f /dev/vg00?
Maybe I should reboot first?
Or should I go to single user mode without rebooting?
Or go to single user mode after rebooting?
Which way should this lab rat go?

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Shoghi Martinez G. — Tue, 20 Jan 2009 16:19:35 GMT

Just let it there, when the new spare disk arrives, initialice it, make it bootable, and resync.

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Steve Post — Tue, 20 Jan 2009 16:21:56 GMT

I agree with that. But the subject of this is what-if the disk is out and I WANT to lvreduce?

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Shoghi Martinez G. — Tue, 20 Jan 2009 16:25:15 GMT

Use vgreduce!

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Steve Post — Tue, 20 Jan 2009 16:27:29 GMT

Ah...but very near future WIZARD (!!!)

I thought you could not run vgreduce because the logical volumes are still mirrored.

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Avinash20 — Tue, 20 Jan 2009 16:28:14 GMT

Yes, you could pull out the disk and wait for the new disk to come(check if it is hot-swappable), unless you like to remove the disk from vg and do the mirroing again !!

Instead just do a plug in the disk vgcfgrestore on disk, vgsync

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Steve Post — Tue, 20 Jan 2009 16:38:36 GMT

Avinash,
But once AGAIN....
I agree with that. But the subject of this is what-if the disk is out and I WANT to lvreduce?

....BEEP.

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Shoghi Martinez G. — Tue, 20 Jan 2009 17:03:16 GMT

I think the missing command here is lvlnboot -R, after you have remove all lv mirrors and vgreduce vg00.

BTW: (almost wizard! I thought my points was under 850.)

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Steve Post — Tue, 20 Jan 2009 17:06:29 GMT

yep. That's what I did before. It DESTROYED it.

points will be coming by the way....

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Shoghi Martinez G. — Tue, 20 Jan 2009 17:13:40 GMT

Ok man I got it, and have you tried the lvreduce/vgreduce, and reboot without executing anything else?

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Steven E. Protter — Tue, 20 Jan 2009 17:26:01 GMT

Shalom,

You can always break the mirror and then do the work. That was how I avoided confusion when working with RAID 1 mirror sets.

SEP

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

sujit kumar singh — Tue, 20 Jan 2009 17:43:29 GMT

Hello Sir,

the best approach and sounding one that i had to do when i had come across that was .... but u need to do an overall mirroring again ... but tell u what this is more clean..

1) do not reboot the system.
2)for all the LVs on the failed disk , do an lvredue like

#lvreduce -m 0 -k /dev/vg00/lvolN 0

assuming that PV Key of the failed disk as u have observed with the #lvdisplay -v -k /dev/vg00/lvol1 etc O/P is 0.

once done with all this lvreduce with all the LVs on VG vg00,

do the following:

#vgreduce -f /dev/vg00

do an

#lvlnboot -R
#lvlnboot -v

put in the new disk and do a mirroring all over.

starting from::::

#pvcreate -B
#mkboot -a "hpux /stand/vmunix -lq" ....
#vgextend /dev/vg00 ....
#lvextend -m 1 /dev/vg00/lvolN /dev/dsk/cxtydz

that is the new disk.
once this is over for all the LVs in vg00,

do an
#vi /stand/bootconf

check that the entry of the new disk exists here if not put that as

l /dev/dsk/cxtydz

#lvlnboot -R
#lvlnboot -v

#setboot just to make sure that u have the alternate boot path as set.

Regards
Sujit

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Richard Pereira_1 — Tue, 20 Jan 2009 19:21:50 GMT

Make sure you have a good OS backup.

Dont reboot.

Restore the LVM configuration/headers onto the new disk from your backup

of the LVM configuration:

# vgcfgrestore -n [volume group name] /dev/rdsk/cXtYdZ

Reactivate the volume group (VG) so that the new disk can be attached,
resync should also occur.

# vgchange -a y [volume group name]

Make disk bootable

# mkboot /dev/rdsk/cXtYdZ

# mkboot -a "hpux (;0)/stand/vmunix -lq" /dev/rdsk/cXtYdZ

# lvlnboot -R

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Steve Post — Tue, 20 Jan 2009 20:17:48 GMT

Ok Sujit....it sounds great...I tried it until I'm ready to put the disk back in. But it brings up two questions (at the bottom).

lvreduce -A n -m 0 -k /dev/vg00/lvol8 1
Logical volume "/dev/vg00/lvol8" has been successfully reduced.
lvlnboot: Logical Volume has no extents.
lvlnboot: Logical Volume has no extents.
lvlnboot: Logical Volume has no extents.
It has been hanging here for 8 minutes so far. This a 256meg filesystem.
Did I screw it up?

(two minutes after that) The command prompt came back! It just took a while.
On to the rest of them.....
I see I have lvol5 with only 10 extents. Yet it take the same time.
It must be trying to do something with lvlnboot, then timing out after about 12 minutes.
This is SLOW, slower than ignite recovery. But it seems safer too. And people could keep working while I do this.
Jan 20 13:02:01 junk LVM[13852]: lvreduce -A n -m 0 -k /dev/vg00/lvol8 1
Jan 20 13:11:06 junk LVM[14013]: lvreduce -A n -m 0 -k /dev/vg00/lvol7 1
Jan 20 13:29:01 junk LVM[14542]: lvreduce -A n -m 0 -k /dev/vg00/lvol6 1
Jan 20 13:37:49 junk LVM[14807]: lvreduce -A n -m 0 -k /dev/vg00/lvol5 1
Jan 20 13:46:44 junk LVM[14962]: lvreduce -A n -m 0 -k /dev/vg00/lvol4 1
Jan 20 13:55:32 junk LVM[15232]: lvreduce -A n -m 0 -k /dev/vg00/lvol3 1
Jan 20 14:04:55 junk LVM[15503]: lvreduce -A n -m 0 -k /dev/vg00/lvol2 1
Jan 20 14:14:55 junk LVM[15503]: lvreduce -A n -m 0 -k /dev/vg00/lvol1 1
and this part was done at 14:24
vgreduce -f /dev/vg00
done at 14:26

lvlnboot -R
I got lots of....
lvlnboot: Warning: couldn't query all of the physical volumes.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c0t4d0":
vgcfgbackup: /etc/lvmtab is out of date with the running kernel:Kernel indicates 1 disks for "/dev/vg00"; /etc/lvmtab has 2 disks.
Cannot proceed with backup.

So I rebooted the junk box with the disk missing at 3pm.
Same error with lvlnboot -R

This brings up two questions.
1. why won't the box just give it up and realize that c0t4d0 is gone already?
2. why did it take 10 to 12 minutes per logical volume on lvreduce?

And Richard: for the third time now. Good advice. But not the question. Read the subject line.

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Steve Post — Tue, 20 Jan 2009 20:37:02 GMT

I got my own answer to number 1.
1. why won't the box just give it up and realize that c0t4d0 is gone already?
Answer: because file /etc/lvmtab says that c0t4d0 is still around.
Here's what I think is the fix:
vgscan -a -p -v
cp -p /etc/lvmtab /root/lvmtab.justincase
rm /etc/lvmtab
vgscan -a -v

Scan of Physical Volumes Complete.
*** LVMTAB has been created successfully.
*** If PV links are configured in the system.
*** Do the following to resync information on disk.
*** #1. vgchange -a y
*** #2. lvlnboot -R

vgchange -a y vg00
vgchange -a y vg01
vgchange -a y vg02
lvlnboot -R
lvlnboot -v
Now lvlnboot has no complaints. I don't think I needed that reboot.

Number 2? The delay in lvreduce? I can't remove lvmtab could I? Let's face it. I would still have to unmirror a missing disk. I should try removing lvmtab before I run lvreduce? At worse, I have to use my ignite tape that runs in 90 minutes. I don't know if I should try it.

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

sujit kumar singh — Wed, 21 Jan 2009 07:24:44 GMT

hi,

as far as removing lvmtab followed by a vgscan -a -v is concerned u can do that that will create an upto date /etc/lvmtab with the vg definitions.

the other part is that why u need to thinkof the Ignite restoartion as ur system seems to be OK and even if u do an ignite restore u will need to redo the mirroring part.

also here from u can proceed like putting in the new disk, pvcreate -B on that , do an mkboot and add to the VG vg00, do the mirroring of all the LVs and do an

#lvlnboot -R
#lvlnboot

update the entries in /stand/rootconf and do that setboot

Regards
sujit

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

sujit kumar singh — Wed, 21 Jan 2009 07:30:52 GMT

hi

in the previous post i meant /stand/bootconf

apologize for that

regards
sujit

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Steve Post — Wed, 21 Jan 2009 17:04:11 GMT

Thanks for the help. Now some of these I posted 0 points to. I just didn't want points unassigned....(RICHARD....)

And even though guys gave me answers to the wrong question, I DO prefer to leave it all alone. In a normal scenario, I would be putting in a new disk in 5 hours.

These discussions here were if the dead disk was somehow pulling down the last, remaining boot disk. If THAT was the case, would want to cut off that dead weight from the balloon before it crashed.

Thanks again.

now just in case, anybody wants to add more, I left the thread open. I thought I left it open. OH OK. It's open again NOW.

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

Steve Post — Wed, 21 Jan 2009 20:23:56 GMT

I got the answer to #2.
The question was:
Why is it that it takes 10 to 15 minutes to run lvreduce on each and every logical volume?

The answer is:
The computer is wasting 10 to 15 minutes looking for a DEAD disk because it is still listed in /etc/lvmtab. So update lvmtab FIRST. You'll wait the 10 to 15 minutes for that. Then the lvmreduce commands take seconds instead of minutes.

So, at the point where the disk is dead and we are trying to remove it.
a. cp -p /etc/lvmtab /etc/lvmtab.savethis
b. vgscan -v
c. vgchange -a y
d. lvmreduce -A n -k -m 0 /dev/vg00 1
(where 1 is the physical key number that I can see from this command that I ran DAYS before: vgcfgrestore -l -v -n /dev/vg00)

In any case I would avoid this. I would still rather leave vg00 alone, and get the new disk in as soon as possible.

Re: what-if hpux11.11 mirror disk is out and want to lvreduce?

TechC — Thu, 14 Feb 2013 05:56:10 GMT

Hey Steve,

I am having the same issue/scenario you detailed here..

I have a failed root mirror disk. I ran lvreduce -m 0 -A n -k /dev/vg00/lvol1 - 9

It took 9 minutes or so for each reduce.. I came across your post afterwards.. Can you layout your procedure so that I can avoid losing data?

Should I vgreduce the bad disk now or just hot plug the replacemnent? After reading your thread not at all sure what to do.

Thanks