mirroring vg00 with /opt spread across 2 disk

[Dave Wright]

My system, running 11.11 has a failed root mirror. The system has 4 disk, 2 OS, and 2 mirror. I have replaced the failed disk and extended vg00 onto it. My problem is that /opt is extended across 2 disks with 2500 PEs on each, all other lv's are on the first disk.

LV PE Disk Disk
-----------------------------------
lvol1 - 75 1 3
lvol2 - 512 1 3
lvol3 - 75 1 3
lvol4 - 2500 1 3
lvol4 - 2500 2 4
lvol5 - 125 1 3
lvol6 - 500 1 3
lvol7 - 500 1 1
lvol8 - 512 2 4

Is there a way I can I extend the first 2500 PEs of lvol4 to the 1st mirror (disk 3) and the 2nd 2500 PEs onto the 2nd mirror (disk 4)?

[Brian M Rawlings]

Hi, Dave. I can't think of any way to structure a command line that will actualy specify which PEs go on which disk, but, it occurs to me that LVM may accidentally do what you are looking for automatically.

Unless you specify extent-based striping (distributed allocation policy), I believe that lvextend allocates available extents from the first PV on the list, until they are gone, and then starts allocating extents from the next PV on the list. If you put the PVs in the correct order on the command line, I'm pretty sure you'll end up with things as you're requesting.

This is from the 'lvcreate' man page...
...
When the distributed allocation policy is turned off, all available free extents are allocated from each available physical volume before proceeding to the next available physical volume. This is the default.
...

I hope this is helpful. You've certainly helped me, by reminding me of another reason to never have more than two PVs in 'vg00', the primary and the mirror, and let the extents fall where they may...

Regards, --bmr

[Iain Ashley]

Hi Dave,
When mirroring, LVM will write until it runs out of space before moving on to the next disk with free extents. You will only get it to work properly if disk 1 currently has no free extents. Once you have made disk 3 bootable, mirror all the lvols in order to disk 3 (supply the PV in the command line) then mirror lvol4 giving it the PV's in the order that you want them to be used:

lvextend -m 1 /dev/vg00/lvol4 /dev/dsk/ /dev/dsk/

LVM will mirror up to the end of disk3 and will then move on to disk4.

Regards
Iain Ashley

[Dave Wright]

Unfortunately, when I ran a vgcfgbackup, the first mirror disk was toast and was not put into the config. So when I tried to do a vgcfgrestore, there was no knowledge of the disk and vgsync would not work.

I was able to put all of the lvols back, although not necessarily in the correct order. I put lvols1-3,5-7 on the first mirror disk, create a 53PE lvol to "reserve" the space that is unused on the first primary disk, then create lvol4 with 2500PE on the first mirror disk and 2500 on the second mirror disk.

And I agree, 2 mirrored primary disks - is a pain.

[Iain Ashley · Accepted solution]

Hi Dave,
I just want to clarify that when you replace a mirrored root disk, that once you get the config restored to it, you will need to reduce it from the VG entirely so that you can make it bootable. This is very important because if you just do a vgcfgrestore, you will not be able to boot from the disk if you lose the primary!

Also the layout of the mirrors is more or less irrelevant with some noteable exceptions. root and stand MUST be the first and second lvols on the boot disk! Beyond that the lvols can fall where they may. If you create two physical volume groups (PVG) this will help you keep the mirrors across controllers.

Here is the correct procedure for replacing a mirrored root disk (sorry if you know this already, but I find that it catches a lot of people out):

#Restore the VGRA to the disk

vgcfgrestore -n vg00 /dev/dsk/cXtXdX

#this will probably return a pv error, but if it says that the config has been restored it's ok

#update the list of active volumes

vgchange -a y vg00

#reduce disk from all lvols. When reducing lvol4 you will have to specify both PVs.

lvreduce -m 0 /dev/vg00/lvol(1-4) /dev/dsk/cXtXdX

#Remove the disk from VG00

vgreduce vg00 /dev/dsk/cXtXdX

#Now start again by properly creating the PV with a BDRA

pvcreate -Bf /dev/rdsk/cXtXdX

#Make it bootable

mkboot /dev/rdsk/cXtXdX

#Set the autofile so you can boot without quorum (if you lose a controller). You may want to repeat this on the primary disk also

mkboot -a "hpux -lq" /dev/rdsk/cXtXdX

#Add the disk back into vg00

vgextend vg00 /dev/dsk/cXtXdX

#Create your PVGs
#see the manpage for lvmpvg for the format, but the /etc/lvmpvg file should look similar to this:

VG vg00
PVG primary
/dev/dsk/c1t0d0
/dev/dsk/c1t1d0

PVG alternate
/dev/dsk/c2t0d0
/dev/dsk/c2t0d1

# Now you can start mirroring.
#***IMPORTANT*** You must mirror lvol1 lvol2 lvol3 in order - wait for each to complete before starting the next.

lvextend -m 1 /dev/vg00/lovl1 /dev/dsk/cXtXdX

#when you get to lvol4 specify both PVs or the PVG, doesn't matter

#finally, register the bootdisk with:

lvlnboot -Rv

#you should see your newly mirrored boot disk in the list (excluding dump)


Regards
Iain Ashley