Newbie question on spanning and mirroring drives.

[Daniel Robert]

I am using HP-UX 11.11.

This is my first server that I have which has four internal drives. Since all the data is going on a RAID, I really don't have any need for the internal disks other then for the system. What I was thinking of doing is to span the VGs across two PV then mirror the spanned VGs to the two other PVs. The following is what I have done, but I am not sure if this has accomplished what I wanted:
- I started out with the boot device being c1t2d0
# pvcreate -B /dev/rdsk/c1t0d0
# pvcreate -B /dev/rdsk/c2t2d0
# pvcreate -B /dev/rdsk/c2t0d0
# vgextend /dev/vg00 /dev/dsk/c1t0d0 /dev/dsk/c2t2d0 /dev/dsk/c2t0d0
# mkboot /dev/rdsk/c2t0d0
# mkboot -a "hpux -lq (;0)/stand/vmunix" /dev/rdsk/c2t0d0
# lvextend -m 1 /dev/vg00/lvol1
# lvextend -m 1 /dev/vg00/lvol2
# lvextend -m 1 /dev/vg00/lvol3
# lvextend -m 1 /dev/vg00/lvol4
# lvextend -m 1 /dev/vg00/lvol5
# lvextend -m 1 /dev/vg00/lvol6
# lvextend -m 1 /dev/vg00/lvol7
# lvextend -m 1 /dev/vg00/lvol8
# lvboot -R /dev/vg00

When I look at the drives I see that the lights on c1t2d0 (the original boot drive) and c2t0d0 are on solid and the other two are flashing rapidly.

Is this accomplishing what I described? Is this the best way to setup these drives?

Unfortunately I will be on holidays until Aug. 5th, so please be patient if you want additional information.

[Nido]

Hello Daniel,

Follow the Document for root mirroring the disks.
Also check the points mentioned for Mirroring Critical Information section specially strict allocation policy.

>>>>lvextend -m 1 /dev/vg00/lvol1
You may specify the disks on which you want to extend the mirror.

http://docs.hp.com/en/5991-1236/When_Good_Disks_Go_Bad_WP.pdf



Mirror the Disk First!!
and Enjoy your Holidays!!

Cheers!!

[SoorajCleris · Accepted solution]

Hi,

You Have 4 drives /dev/rdsk/c1t2d0, /dev/rdsk/c1t0d0, /dev/rdsk/c2t2d0, /dev/rdsk/c2t0d0

Befiore mirroring give some considerations to implement more redundancy

Select different controller for mirror disk.Selecting different controllers will help you to have one more level of redendancy that, if your one controller fails also you can boot from other disk.


your current boot disk is /dev/rdsk/c1t2d0. So select /dev/rdsk/c2t2d0 or /dev/rdsk/c2t0d0 for mirror. ( C2)

1. Create a physical volume with a boot reserved area
# pvcreate -B /dev/rdsk/c2t2d0

2. Add the physical volume to the root VG
#vgextend /dev/vg00 /dev/dsk/c2t2d0

3. Use mkboot to place the boot utilities in the boot area and add the AUTO file and verify

#mkboot -l /dev/rdsk/c2t2d0
#mkboot -a "hpux -lq" /dev/rdsk/c2t2d0

Verify using the LIF utilities are copied and autofile is edited.

#lifls /dev/rdsk/c2t2d0 ( ensure it showls the LIF contents)
## lifcp /dev/rdsk/c2t2d0:AUTO - ( ensure autofile is modified)


4. Use mkboot to update the AUTO file on the primary boot disk

#mkboot -a "hpux -lq" /dev/rdsk/c1t2d0

6. Mirror all the system logical volumes. Dont miss to mention the disk name. Check mirror policy also. That should be strict policy

# lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c2t2d0
# lvextend -m 1 /dev/vg00/lvol2 /dev/dsk/c2t2d0
# lvextend -m 1 /dev/vg00/lvol3 /dev/dsk/c2t2d0
*****
******
******
# lvextend -m 1 /dev/vg00/lvol8 /dev/dsk/c2t2d0


7. Modify your alternate boot path
#setboot -a x/y/z.c.t.d ( use the path of your new boot disk)

8. Update your BDRA

# lvboot -R /dev/vg00

9. Ensure BDRA is updated

# lvboot -v /dev/vg00

9. Edit /stand/bootconf and add your new mirrored boot disk. This will help to update the ODE and other utlity in LIF area when ever there is an upadate using SD-UX.

# vi /stand/bootconf
l /dev/dsk/c2t2d0

###################################

For mirroring you may use

# for i in 1 2 3 4 5 6 7 8 ; do
> lvextend -A n -m 1 /dev/vgoo/lvol$i /dev/dsk/c2t2d0
>done
########################################

[Daniel Robert]

Hi Sooraj, thanks for the detailed instructions, it was handy to have to compare to what I have been using in the past. One thing that is that I cannot find is the "lvboot" command. Is that available in 11i Version 1?

Another problem is is that I am not sure how to extend the boot drives (the primary one and the mirrored one). Do I extend c1t2d0 to c2t2d0 and c1t0d0 to c2t0d0?

[Mel Burslan]

he meant "lvlnboot" not lvboot....

[Daniel Robert]

Was it lvboot before (in 11.0) as my documentation has this and I was not able to find that command when I tried to do it again this time (although I have done setup new systems in the past and I don't believe a ran into this, huh)? If not, it may have been a typo in my documentation and Sooraj may have copied it from my question.

[Daniel Robert]

Ooops, I rather meant add c1t0d0 and c2t0d0 to vg00. I that what I have to do?

[Ganesan R]

Hi,

>>What I was thinking of doing is to span the VGs across two PV then mirror the spanned VGs to the two other PVs<<

To achieve this you should understand the allocation policies. You have to configure Physical Volume Group(PVG) to accomplish your task. See man lvcreate to know more about allocation policies.

PVG is like grouping the disks. One set of disks will be part of one PVG and another set of PVG will be part of another PVG. When you do mirror, it will mirror the date copy on second PVG. This way you can have primary copy on first PVG and mirror copy on second PVG. PVG spannes across two disks.

[Ganesan R]

Hi Again,

Configuring PVG is simple. Just create a /etc/lvmpvg file like below.

# vi /etc/lvmpvg
VG /dev/vg00
PVG pvg1
/dev/dsk/c1t0d0
/dev/dsk/c1t2d0
PVG pvg2
/dev/dsk/c2t0d0
/dev/dsk/c2t2d0


You have to create this file before mirroring. Once you created this file you can follow the normal steps to mirror the root disks.

Before mirroring a LV you need to set itâ s allocation policy to PVG-strict, if it is not set already. By default lv will be created with strict allocation policy only. e.g:
# lvchange -s g /dev/vg01/lvol1
# lvdisplay /dev/vg01/lvol1 | grep Allocation
Allocation PVG-strict

For details look at the lvmpvg man page.

[Daniel Robert]

Hi Ganesan, thanks for the additional information, but I think that there is still a piece of the puzzle still missing. To prepare for the implementation of your suggestion, I removed the mirror copies that I had already created. Then I created the lvmpvg file and changed all the LV's allocation policy to be PVG-strict.

On another note, I reinstalled the server and this time made c2t0d0 as the primary boot device and c1t0d0 as the secondary boot device (mirror copy) and have c2t2d0 belong to c2t0d0's PVG and c1t2d0 to c1t0d0's PVG (I did this because it made more sense visually as the top drives are using controller 2 and the bottom are controller 1 (why?), and to me the top drives should be primary ones, sorry, just my quirky preference).

Now back to the problem at hand... when I tried to mirror the LVs I got the following messages:
Information in "etc/lvmtab" differs from that in "etc/lvmpvg".
Physical volume "dev/dsk/c2d2t0" does not belong to volume group "/dev/vg00".
lvextend: Error detected when reading from file "/etc/lvmpvg".

#strings /etc/lvmtab
/dev/vg00
/dev/dsk/c2d0t0
/dev/dsk/c1t0d0

#cat /etc/lvmpvg
VG /dev/vg00
PVG upperdrives
/dev/dsk/c2t0d0
/dev/dsk/c2d2t0
PVG lowerdrives
/dev/dsk/c1t0d0
/dev/dsk/c1t2d0

I used the vgscan -av but it did not add the other two drives. I think that I need to use the vgimport, but I am not sure how to proceed. Here is what I _think_ needs to be done:
#vgimport -v /dev/vg00 /dev/dsk/c2t0d0 /dev/dsk/c2t2d0
#vgimport -v /dev/vg00 /dev/dsk/c1t0d0 /dev/dsk/c1t2d0
vgchange -a y vg00

Sorry, this is probably viewed as handholding, but I was not able find documentation on how to mirror spanned VGs, and the fact that they are spanned is what's tripping me up.

[Ganesan R]

Hi Daniel,

Your steps looks good except that you have not added the disks /dev/dsk/c2t2d0 and /dev/dsk/c1t2d0 into the VG.

I assume that you installed the os on c2t0d0 and added c1t0d0(using pvcreate -B and vgextend) into vg00. And also you made the /etc/lvmpvg file.

Now you need to add the c1t2d0 and c2t2d0 into vg before you start the mirror.

#pvcreate -f /dev/rdsk/c2t2d0
#pvcreate -f /dev/rdsk/c1t2d0
#vgextend /dev/vg00 /dev/dsk/c2t2d0 /dev/dsk/c1t2d0

Now check the content of /etc/lvmtab. it should list all four disks.

If you start the mirror now, the data available on c2t0d0+c2t2d0 will be mirrored with c1t0d0+c1t2d0

[Daniel Robert]

Hi Ganesan,

Thanks for the additional information. One thing I want to make clear before I finalize my documentation; do these three new instructions (the two pvcreate and the vgextend for the two other drives) need to come right after creating/updating the lvmpvg file?

[Ganesan R]

Hi Daniel,

Sorry for the late reply. Since we both are working in two different timezones which caused the delay.

>>One thing I want to make clear before I finalize my documentation; do these three new instructions (the two pvcreate and the vgextend for the two other drives) need to come right after creating/updating the lvmpvg file?<<

It doesn't matter before lvmpvg file creation or after. But you have to add those disks and make the lvmpvg file before mirroring. Important thing is disks should be present in /etc/lvmtab file. You create the lvmpvg file then add the disks.

Note lvmpvg file can be created in two ways. One is manual creation and another way is with the help of vgcreate/vgextend. Both vgcreate and vgextend commands have options to add the pvg groups in the command itself.

[Daniel Robert]

Before mirroring the drives I increased the size of home (since I did not have enough room on the sigle drive for it's intended size) and created another. When I tried to mirror, it works fine until I get to the home volume where is indicates that "The LMV device driver could not extend the logical volume "/dev/vg00/lvol4".

[Daniel Robert]

I think it was because I had created the mirrors before I extended it, then I removed the mirror, extended the volume then tried to mirror it back again. At any rate I guess I could just have cleared the contents of the drive by doing another pvcreate -f, but I decided to restart the HP-UX install over again, but this time I did it a bit differently. I spanned the primary drives in the OS install and defined my volumes the size I wanted and also created my additional volume.

I case someone wants to follow the steps I took to get mirrored spanned disks, here they are:
In the install I selected c2t0d0 as the primary drive
I added the c2t2d0 disk to span the volumes across the two drives then re-sized and added the volumes that needed adjustment. In total there were nine volumes.
After the install was complete I did the following:
#pvcreate -f -B /dev/rdsk/c2t0d0 (to create a physical volume with a boot reserved area)
#vgextend /dev/vg00 /dev/dsk/c1t0d0 (to add the physical volume to the root VG)
#mkboot -l /dev/rdsk/c1t0d0 (define as volume layout disk)
#mkboot -a "hpux -lq" /dev/rdsk/c1t0d0 (Adds the AUTO file to the disk)
#lifls /dev/rdsk/c1t0d0 (ensure it shows LIF contents)
#lifcp /dev/rdsk/c1t0d0:AUTO (ensure AUTO file is modified)
#mkboot -a "hpux -lq" /dev/rdsk/c2t0d0 (update AUTO file on the primary disk)
#vi /etc/lvmpvg (this will define the Physical Volume Groups within the VG)
VG /dev/vg00
PVG primarydrvs
/dev/dsk/c2t0d0
/dev/dsk/c2t2d0
PVG mirrordrvs
/dev/dsk/c1t0d0
/dev/dsk/c1t2d0
ZZ
#for VOL in 1 2 3 4 5 6 7 8 9; do (9 volumes because of an extra one created at install-time)
>lvchange -s g /dev/vg00/lvol$VOL (to change allocation to PVG-strict)
>lvdisplay -v /dev/vg00/lvol$VOL |grep Allocation (to ensure the change was made)
>done
#pvcreate -f /dev/rdsk/c1t2d0 (to create the PV of the second drive for the mirrored set)
#vgextend /dev/vg00 /dev/dsk/c1t2d0 (To add it to vg00)
#for VOL in 1 2 3 4 5 6 7 8 9; do (9 volumes because of an extra one created at install-time)
>lvextend -m 1 /dev/vg00/lvol$VOL mirrordrvs (I used the PVG name instead of the set of disks)
>done
#setboot -a /0/0/1/1.0.0 (to set the alternate boot path)
#lvlnboot -R /dev/vg00 (update the BDRA)
#lvlnboot -v /dev/vg00 (Ensure the BDRA is updated)
#vi /stand/bootconf (This will help to update the ODE and other utilities in LIF area whenever there is an update using SD-UX)
l /dev/dsk/c1t0d0
ZZ

To test that the alternate (mirrored) boot drive was good, I rebooted and interrupted it and specified that I wanted to BO ALT and it worked.


Thanks to Sooraj and Ganesan for their collaborative efforts.