Mirror with stale extend on good drive

[Dennis Boyd]

PROBLEM --
MirrorDisk/UX with stale extend on lvol1 (/stand)...
0001 /dev/dsk/c0t6d0 0001 current /dev/dsk/c0t5d0 0001 stale

I replaced a disk with the stale extend (c0t5d0).
It still shows stale on the new disk.

I assume that means that my problem is on the other disk.
I guess this proves that...
dd if=/dev/rdsk/c0t6d0 of=/dev/null bs=64k
dd read error: I/O error
143+0 records in
143+0 records out

QUESTIONS --
Can I reduce/remove the bad disk even with the good disk reporting
a stale extend?
Any other thoughts/ideas/solutions?

MORE DETAILS --
I do not have a current tape for this box :-(
I tried to make a recovery tape but I keep getting a i/o error.

I tried to look for a bad file using fbackup...
fbackup -i /stand -f - -v >/dev/null
This did not report trouble reading any of the files.

SYSTEM DETAILS --
9000/856/E55 HP-UX 10.20
Jamaca drive unit with 2 drives (no internal drives).
/dev/vg00 /dev/dsk/c0t6d0 /dev/dsk/c0t5d0
/dev/vg00/lvol1 /stand
/dev/vg00/lvol2
/dev/vg00/lvol3 /
/dev/vg00/lvol4 /home
/dev/vg00/lvol5 /opt
/dev/vg00/lvol6 /tmp
/dev/vg00/lvol7 /usr
/dev/vg00/lvol8 /var

Regards

[Patrick Wallek]

What steps did you do after you replaced the drive?

You must go through all appropriate steps to get the VG config onto the new drive and resynchronize the mirrors.

[Stefan Farrelly]

yes, you can remove c6t0d0 and replace it even though c0t5d0 is showing a stale. All the stale means is that its out of sync - data is still there its just not update with the data on c0t6d0 (due to the bad block/IO error). The dd proves you have a problem with c0t06d0 so go ahead and replace. If its your boot disk ensure you follow the exact procedures for replacing a boot disk to ensure its still bootable afterwards!

[G. Vrijhoeven]

Hi,

Yes you should be able to remove the failed disk. You can use the lvreduce command. If it does not work use the -k option ( man lvreduce)

Gideon

[Todd McDaniel_1]

Dennis,

You must execute a few commands to get the stale extents to dissapear...

on a root volume you would do the following:

The pvcreate may fail b/c your disk device would most likely still be defined in lvmtab.

c4t6d0 is just a representation of your disk...

pvcreate -f /dev/rdsk/c4t6d0
pvcreate -B /dev/rdsk/c4t6d0
mkboot /dev/rdsk/c4t6d0
mkboot -a "hpux -lq" /dev/rdsk/c4t6d0
lifcp /dev/dsk/c4t6d0:AUTO
mkboot -b updatediaglif -p ISL -p AUTO -p HPUX -p LABEL /dev/rdsk/c4t6d0
mkboot -b /usr/sbin/diag/lif/updatediaglif -p ISL -p AUTO -p HPUX -p LABEL /dev
/rdsk/c4t6d0
vgcfgrestore -n /dev/vg00 /dev/rdsk/c4t6d0
vgchange -a y /dev/vg00
vgsync /dev/vg00

[Todd McDaniel_1]

One more thing.

If you already removed a disk and replaced it.. you cannot remove the other disk before you fix the one you just replaced.

You must have at least 1 good disk.

[Steven E. Protter]

Yes,

attaching a more complete mirror/boot procedure, in case you run into issues.

insf -e

pvcreate -B /dev/rdsk/c1t0d0 #use real disk

mkboot -l /dev/rdsk/c1t0d0
mkboot -a "hpux -lq (;0)/stand/vmunix" /dev/rdsk/c1t0d0 # use real disk


# mkboot -b /usr/sbin/diag/lif/updatediaglif -p ISL -p AUTO -p HPUX -p PAD -p LABEL /dev/rdsk/c?t?d?

If you are running 64-bit OS:

# mkboot -b /usr/sbin/diag/lif/updatediaglif2 -p ISL -p AUTO -p HPUX -p PAD -p LABEL /dev/rdsk/c?t?d?


vgextend /dev/vg00 /dev/dsk/c1t0d0 # same thing
lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c1t0d0

# real disk. repeat for other lvols

lvlnboot -r /dev/vg00/lvol3 # root fs /
lvlnboot -s /dev/vg00/lvol2 #swap
lvlnboot -d /dev/vg00/lvol2 #swap/dump
lvlnboot -b /dev/vg00/lvol1
lvlnboot -R
lvlnboot -v
setboot
setboot -a 52.1.0 # second disk

SEP

[Sridhar Bhaskarla]

Hi,

Try following Todd's procedure and see if it helps. vgcfgrestore followed by vgchange -a y is the key here.

It is possible to remove the disk. But you have to reduce the mirrors on the logical volumes after you did a vgcfgrestore and vgchange -a y before vgsync.

To remove the disk out of lvmtab after the disk is physically replaced do

#ioscan -f
#diskinfo /dev/rdsk/c0t5d0 (verify the disk)
#vgcfgrestore -n vg00 /dev/rdsk/c0t5d0
#vgchange -a y vg00
#lvreduce -m 1 /dev/vg00/lvol1 /dev/dsk/c0t5d0
repeat the above command for all the logical volumes
#pvdisplay -v /dev/dsk/c0t5d0
make sure no logical volumes are listed in there
#vgreduce vg00 /dev/dsk/c0t5d0

Then you can follow fresh mirror adding procedure.

-Sri

[Dennis Boyd]

Wow. Thanks for all of the great info.

c0t5d0 = good
c0t6d0 = bad

This is a remote site so I had to ship another disk. I should have someone onsite with the disk on Monday.

In the mean time, I was going to remove the bad disk from the mirror and try to make a recovery tape. I was planning on ...
for i in /dev/vg00/lvol*
do
lvreduce -m 0 $i /dev/dsk/c0t6d0
done
vgreduce /dev/vg00 /dev/dsk/c0t6d0
and then of course fix my setboot.

When I was doing some pre checks, i found someting that doesn't look right.

# lifcp /dev/rdsk/c0t5d0:AUTO -
hpux -lq (;0)/stand/vmunix
** looks good

# lvlnboot -v
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
/dev/dsk/c0t6d0 (56/48.6.0) -- Boot Disk
/dev/dsk/c0t5d0 (56/48.5.0) -- Boot Disk
No PV Name Logical Volume configured
Root: lvol3 on: /dev/dsk/c0t6d0
/dev/dsk/c0t5d0
Swap: lvol2 on: /dev/dsk/c0t6d0
/dev/dsk/c0t5d0
Dump: lvol2 on: /dev/dsk/c0t6d0, 0

** where is lvol1? why no pv name?

this is what i am used to seeing...
# lvlnboot -v
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
/dev/dsk/c0t6d0 (56/52.6.0) -- Boot Disk
/dev/dsk/c0t5d0 (56/52.5.0) -- Boot Disk
PV Name: lvol1 on: /dev/dsk/c0t6d0
/dev/dsk/c0t5d0
Root: lvol3 on: /dev/dsk/c0t6d0
/dev/dsk/c0t5d0
Swap: lvol2 on: /dev/dsk/c0t6d0
/dev/dsk/c0t5d0
Dump: lvol2 on: /dev/dsk/c0t6d0, 0

[Sridhar Bhaskarla]

Well, let's be bit cautious here as there are a number of posts with various suggestions.

c0t5d0 is stale as per your original thread.
If you replace it with a good disk and if you want to remove c0t6d0, then you need to make sure that the logical volumes get sync'ed without any issues. For that use the procedure outlined to mirror the disk. vgcfgrestore will only restore the LVM headers not the data. 'vgsync' is the command that synchronizes the data.

So, before removing c0t6d0, make sure you get the data moved to c0t5d0.

-Sri

[Sridhar Bhaskarla]

I missed another question in your last message.

//** where is lvol1? why no pv name? //

When you did mkboot on both the disks during the mirror restore process, you might have missed update the BDRA.

So, you would do

lvlnboot -b /dev/vg00/lvol1
lvlnboot -s /dev/vg00/lvol2
lvlnboot -r /dev/vg00/lvol3
lvlnboot -d /dev/vg00/lvol2
lvlnboot -R
lvlnboot -v ( to verify )

-Sri