How to replace a bad concatenated disk which is in mirror

[Anil_32]

Hi Gurus,

I have a bad disk which is concatenated mirror one. That is two 9 GB drives are concatenated and is mirrored with another two concatenatd disks. the total volume size is of around 18 GB. One of the concatenated disk is showing as stale now. I have to replace it without loosing the data.

Pls find below the lvdisplay o/ps.

lvdisplay -v /dev/vg02/lvol1 |pg
--- Logical volumes ---
LV Name /dev/vg02/lvol1
VG Name /dev/vg02
LV Permission read/write
LV Status available/stale
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 17360
Current LE 4340
Allocated PE 8680
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation strict
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c0t0d0 2170 2170
/dev/dsk/c2t1d0 2170 2170
/dev/dsk/c2t6d0 2170 2170

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 ??? 00000 stale /dev/dsk/c2t1d0 00000 current
00001 ??? 00001 stale /dev/dsk/c2t1d0 00001 current
00002 ??? 00002 stale /dev/dsk/c2t1d0 00002 current
00003 ??? 00003 stale /dev/dsk/c2t1d0 00003 current
00004 ??? 00004 stale /dev/dsk/c2t1d0 00004 current
00005 ??? 00005 stale /dev/dsk/c2t1d0 00005 current
00006 ??? 00006 stale /dev/dsk/c2t1d0 00006 current
00007 ??? 00007 stale /dev/dsk/c2t1d0 00007 current
00008 ??? 00008 stale /dev/dsk/c2t1d0 00008 current
00009 ??? 00009 stale /dev/dsk/c2t1d0 00009 current
00010 ??? 00010 stale /dev/dsk/c2t1d0 00010 current
00011 ??? 00011 stale /dev/dsk/c2t1d0 00011 current
00012 ??? 00012 stale /dev/dsk/c2t1d0 00012 current
00013 ??? 00013 stale /dev/dsk/c2t1d0 00013 current
00014 ??? 00014 stale /dev/dsk/c2t1d0 00014 current
00015 ??? 00015 stale /dev/dsk/c2t1d0 00015 current
00016 ??? 00016 stale /dev/dsk/c2t1d0 00016 current
00017 ??? 00017 stale /dev/dsk/c2t1d0 00017 current
00018 ??? 00018 stale /dev/dsk/c2t1d0 00018 current
00019 ??? 00019 stale /dev/dsk/c2t1d0 00019 current
00020 ??? 00020 stale /dev/dsk/c2t1d0 00020 current
00021 ??? 00021 stale /dev/dsk/c2t1d0 00021 current
00022 ??? 00022 stale /dev/dsk/c2t1d0 00022 current
00023 ??? 00023 stale /dev/dsk/c2t1d0 00023 current
00024 ??? 00024 stale /dev/dsk/c2t1d0 00024 current

.
.
.
.

02168 ??? 02168 stale /dev/dsk/c2t1d0 02168 current
02169 ??? 02169 stale /dev/dsk/c2t1d0 02169 current
02170 /dev/dsk/c0t0d0 00000 current /dev/dsk/c2t6d0 00000 current
02171 /dev/dsk/c0t0d0 00001 current /dev/dsk/c2t6d0 00001 current
02172 /dev/dsk/c0t0d0 00002 current /dev/dsk/c2t6d0 00002 current
02173 /dev/dsk/c0t0d0 00003 current /dev/dsk/c2t6d0 00003 current
02174 /dev/dsk/c0t0d0 00004 current /dev/dsk/c2t6d0 00004 current
02175 /dev/dsk/c0t0d0 00005 current /dev/dsk/c2t6d0 00005 current
02176 /dev/dsk/c0t0d0 00006 current /dev/dsk/c2t6d0 00006 current
02177 /dev/dsk/c0t0d0 00007 current /dev/dsk/c2t6d0 00007 current
02178 /dev/dsk/c0t0d0 00008 current /dev/dsk/c2t6d0 00008 current
02179 /dev/dsk/c0t0d0 00009 current /dev/dsk/c2t6d0 00009 current
02180 /dev/dsk/c0t0d0 00010 current /dev/dsk/c2t6d0 00010 current
02181 /dev/dsk/c0t0d0 00011 current /dev/dsk/c2t6d0 00011 current
02182 /dev/dsk/c0t0d0 00012 current /dev/dsk/c2t6d0 00012 current


vgdisplay -v /dev/vg02
--- Volume groups ---
VG Name /dev/vg02
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 16
Cur PV 4
Act PV 3
Max PE per PV 2170
VGDA 6
PE Size (Mbytes) 4
Total PE 6510
Alloc PE 6510
Free PE 0
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg02/lvol1
LV Status available/stale
LV Size (Mbytes) 17360
Current LE 4340
Allocated PE 8680
Used PV 3


--- Physical volumes ---
PV Name /dev/dsk/c0t0d0
PV Status available
Total PE 2170
Free PE 0
Autoswitch On

PV Name /dev/dsk/c2t1d0
PV Status available
Total PE 2170
Free PE 0
Autoswitch On

PV Name /dev/dsk/c2t6d0
PV Status available
Total PE 2170
Free PE 0
Autoswitch On



Here is the /etc/lvm/vg02.conf.old file output:
/dev/vg02
/dev/rdsk/c0t5d0
/dev/rdsk/c0t0d0
/dev/rdsk/c2t1d0
/dev/rdsk/c2t6d0
so the failed disk here is "c0t5d0".

Thanks in advance.
Anil.

[Pete Randall]

Here's an excerpt from the Disk Recovery Cookbook that covers your situation:


"Shutdown the system, have the customer engineer replace the faulty disk, and
then boot the system. You can boot the system into either single or
multiuser mode, depending on whether you need to provide access to your
users while the recovery procedure is being performed. For file systems that
have mirror copies on the replaced disk, the file system can be used by the
users during recovery. For file systems that didn't have a mirror, and that
resided on the replaced disk, you will have to deny access by the users. Do
this by unmounting the relevant file systems ( they will possibly not be
mounted as part of the bootup sequence anyway ).

[Step 2.2]

Restore the LVM configuration/headers onto the replaced disk from your
backup of the LVM configuration:

# vgcfgrestore -n /dev/rdsk/c0tXd0

where X is the Logical unit number of the disk that has been replaced. For
our example:

# vgcfgrestore -n /dev/vg00 /dev/rdsk/c0t2d0

NOTE: You must have performed the command vgcfgbackup to save off the
headers prior to the disk failure ( Refer to Appendix A ).

[Step 2.3]

Reactivate the volume group so that the new disk can be attached, since it
wasn't configured in at boot time. This will also resync any mirrors that
resided on the faulty disk.

# vgchange -a y

For our example, the volume group vg00 will already be activated, but it
will not know of the replaced disk. Therefore, this step is still required
so that LVM will now know that the disk is again available and the resync
will occur:

# vgchange -a y /dev/vg00

[Step 2.4]

For any file systems on the faulty disk that didn't have mirror copies, you
will have to rebuild the file systems and restore the data. Follow the steps
1.4 and 1.5 in chapter 1 for guidance here.

[Step 2.5]

If you booted your system into single-user mode in step 2.1, reboot your
system now and allow it to boot into multiuser mode. If you were already in
multi-user mode, then no further action is required.

At this stage, your system should be fully functioning."




Pete

[Steven E. Protter]

Couple things I like to do before I start.

1) Ignite backup. Gets the config, make_tape_recovery. Have databases down for that so they can be restored.
2) lvdisplay and other disk reports.

You chould be able to do the following.

lvreduce /dev/vg02 /dev/dsk/c0t5d0

shut system or replace hot swap disk.

pvcreate /dev/rdsk/c0t5d0

vgextend /dev/vg02 /dev/dsk/c0t5d0

If you lvreduce before replacing the disk, you may be able to avoid blowing away /etc/lvmtab and regenerating it with vgscan.

SEP

[Anil_32]

Hi SEP,

How about vgreduce before replacing the bad disk ???? Does this step is required ??

Thanks ,
Anil.