topic Re: PVG-Strict and mirroring in Operating System - HP-UX

PVG-Strict and mirroring

Patrice Le Guyader — Fri, 10 Mar 2006 03:01:26 GMT

Hello,

We have several volume groups spread on two disks array. The server has 4 FC cards, 2 on each array, so each disk has an alternate path on both array. vg01 has 3 disks; 3 on array A and on 6 array B. Each logical volume is created like this :

lvcreate -s g -t 90 -r n -l PE_Number /dev/vg01

/etc/lvmpvg is defined as following :

VG /dev/vg01
PVG arrayA
/dev/dsk/c23t0d3
/dev/dsk/c22t0d3 (alternate)
/dev/dsk/c23t2d3
/dev/dsk/c22t2d3 (alternate)
/dev/dsk/c23t3d0
/dev/dsk/c22t3d0 (alternate)
PVG arrayB
/dev/dsk/c26t0d3
/dev/dsk/c27t0d3 (alternate)
/dev/dsk/c26t2d3
/dev/dsk/c27t2d3 (alternate)
/dev/dsk/c26t3d0
/dev/dsk/c27t3d0 (alternate)

We are in a two nodes Veritas Cluster.
When we make a vgexport on node Alpha :

vgexport -svp -m vg01.map vg01

and make a vgimport on node Beta :

vgimport -sv -m vg01.map

After of course having copied the map and supplied additionnals disks in /etc/lvmpvg on node Beta, we have bad definition of vg01.
We expect to have primary on an array and mirror on the other but we have as primary some on array A and some on array B and so on for mirror disks.

If someone as an explanation ?

Rgeards
Patrice

Re: PVG-Strict and mirroring

RAC_1 — Fri, 10 Mar 2006 03:08:41 GMT

What does lvdisplay on lvols under vg01 on both systems say??

Re: PVG-Strict and mirroring

Senthil Kumar .A_1 — Fri, 10 Mar 2006 03:30:23 GMT

Hi,

can you attach the following output in this thread..We need the outputs on both the servers to explain effectively...

vgdisplay -v
ioscan -fnC disk
cat /etc/lvmpvg

Regards,
Senthil Kumar .A

Re: PVG-Strict and mirroring

Carsten Krege — Fri, 10 Mar 2006 04:43:43 GMT

When you vgimport a VG with the -s option, LVM searches for disks with the VG ID listed in the map file. It does not care and does not know about primary and alternate links as listed in the lvmpvg file.

During import you should make sure to import first the primary links and only then add the alternate links. THis works best without using the -s option but using the -f option.

Carsten

Re: PVG-Strict and mirroring

Carsten Krege — Fri, 10 Mar 2006 04:48:29 GMT

I should have mentioned that there is a vgexport -p -f option that allows to create the infile that you need for import on the other node.
However, you still need to make sure that the disk device files of the VG on both nodes are the same. If they are not you would need to modify the device files in the infile before vgimport on the other node.

Carsten

Re: PVG-Strict and mirroring

Tiziano Contorno _ — Fri, 10 Mar 2006 04:57:29 GMT

Patrice, your (LUN on array <-> special file on the server) couple may be different on the 2 servers.
You should use a tool to identify the real LUN and its special file on the 2 servers. I use "xpinfo" with XP Storage.

Remember you can always play with vgreduce / vgextend on the VG on the alternate node to change primary / alternate pathing.

Regards.

Re: PVG-Strict and mirroring

Senthil Kumar .A_1 — Fri, 10 Mar 2006 04:59:59 GMT

Hi krege,

If I'm can remember right , there is no "-f " option for vgimport, probably you are trying to refer to "ioinit" command which uses the "-f" option.

----------------------------------------
Anyway, Patrice, we can help you rectify the situation if you can give the command outputs we asked for.

Regards,
Senthil Kumar .A

Re: PVG-Strict and mirroring

Carsten Krege — Fri, 10 Mar 2006 07:37:29 GMT

from the vgimport manual page:

-f infile Import the set of pv_paths held in the infile to the volume group. This option is used as an alternative to specifying the pv_paths on the command line. Each pv_path must appear on a new
line in the infile. This option may not be used together with the -s option.

Carsten

Re: PVG-Strict and mirroring

Patrice Le Guyader — Fri, 10 Mar 2006 07:39:07 GMT

Hello,

Here they are.

and /etc/lvmpvg :

VG /dev/vg01
PVG pv1
/dev/dsk/c23t0d3
/dev/dsk/c22t0d3
/dev/dsk/c23t2d3
/dev/dsk/c22t2d3
/dev/dsk/c23t3d0
/dev/dsk/c22t3d0
PVG pv2
/dev/dsk/c26t0d3
/dev/dsk/c27t0d3
/dev/dsk/c26t2d3
/dev/dsk/c27t2d3
/dev/dsk/c26t3d0
/dev/dsk/c27t3d0

Thanks
Patrice

Re: PVG-Strict and mirroring

Senthil Kumar .A_1 — Fri, 10 Mar 2006 07:42:26 GMT

Hi krege,

This is the man page I'm referring for vgimport.

http://docs.hp.com/en/B2355-90129/vgimport.1M.html

This does not show any "-f" option. Just curious, am'I refering an outdated manpage. If you don't mind can you share the link which shows "-f" option.

Thanks,
Senthil Kumar .A

Re: PVG-Strict and mirroring

James R. Ferguson — Fri, 10 Mar 2006 07:48:50 GMT

Hi:

Current 11i manpages for 'vgimport' and 'vgexport' describe the '-f' option:

http://docs.hp.com/en/B2355-60127/vgimport.1M.html

http://docs.hp.com/en/B2355-60127/vgexport.1M.html

Regards!

...JRF...

Re: PVG-Strict and mirroring

Tiziano Contorno _ — Fri, 10 Mar 2006 08:15:10 GMT

Importing VG, from a map created with the -s option, scan PV in no particular order. The first path scanned for a given PV become primary. You have no control on this. You have to manualy vgreduce the primary pathed PV that should be alternate: in this way you promote the alternate to primary. Then vgextend with the just removed PV to set it as alternate path.

Regards.

Re: PVG-Strict and mirroring

Patrice Le Guyader — Fri, 10 Mar 2006 11:33:12 GMT

The previous information are in a situation I would call "clear". After reducing the mirror, pvmove PE from disk to disk, recreating the mirrors.

What I want is to export from one server to the other server without breaking my head.

If I understand what you're saying, when you import with "-s" option no matter of what's in lvmpvg and "PVG-strict" allocation policy on logical volumes ? Strange.

For me when import starts it was like this :

Due to the VGID in the map two possible disks to start, one on each array.
Doesn't matter which one you take first but I thought that after the first disk was choosen with lvmpvg and PVG-strict policy all the other would have been on the same array and the mirror should have been the others disks.

According to me the -f option is quite boring for going from one server to another if you have different controler id.

Any advise ?

Thanks
Patrice

Re: PVG-Strict and mirroring

Tiziano Contorno _ — Fri, 10 Mar 2006 11:52:13 GMT

Sorry, but AFAIK ther's no "painless" import of shared VGs if you need exact PRI ALT couples.

1) Import
2) Check that each LUN on the array has the same special file on server A and B, if not, consider the "translation".
2) If needed vgreduce each VG of the primary path you want as alternate vgextending at the next step.

PVG files it's only a logical reference for the LVM commands.

Regards.

Re: PVG-Strict and mirroring

Senthil Kumar .A_1 — Fri, 10 Mar 2006 12:03:36 GMT

Hi Patrice,

Follow the instructions of vanadio. He is spot on with his replies. Do not bother about the PVG groups. That we can sort out later. You can even create /etc/lvmpvg file later. As vanadio pointed out it is just a Logical group, used to differentiate disks b/w controllers.

First lets sort out the Primary and alternate link after vgimport and then fix PVG part.

If you are not sure of command sequence let us know, we will assist you with that.

Regards,
Senthil Kumar .A

Re: PVG-Strict and mirroring

Senthil Kumar .A_1 — Fri, 10 Mar 2006 13:56:32 GMT

Thanks, James and Krege for the info. I will remember that.

Re: PVG-Strict and mirroring

Patrice Le Guyader — Fri, 10 Mar 2006 16:22:30 GMT

I don't know if I could explain my problem. My problem is that when I import my vg definition on second node there is what I call "cross extension". What I mean is without holding account of mirror only for primary; I have first disk of volume group on first array,second on same ,third on second array and last on first array.

It's not a simple problem of primary/alternate path definition of a disk.
I've already scripts that load balance that across controlers on one disk array.

Thanks for all gents.
Patrice.

Re: PVG-Strict and mirroring

Tiziano Contorno _ — Sat, 11 Mar 2006 07:29:40 GMT

Ok, I got it. You are getting some mirror copies as your primaries on the other node.

I bet you have only different device names.

I don't know how, with your array, but you surely can check the real _LUN_on_Array_ distribution of your imported VG.

Regards.

Re: PVG-Strict and mirroring

Patrice Le Guyader — Wed, 05 Apr 2006 01:26:50 GMT

Hi all,

I've a possible solution to my problem but I would like to know if this scan order is in the H/W path order or in the controler one?

Regards
Kenavo
Pat

Re: PVG-Strict and mirroring

jon2 — Wed, 05 Apr 2006 04:42:20 GMT

Patrice,
You could try the following:
vgimport -m mapfile -s vg01

#Import the vg with unexpected paths.

vgexport -m mapfile -f /tmp/disks vg01

vi /tmp/disks
re-arrange disks to be in expected order
primaries followed by mirrors.
vgimport -m mapfile -f /tmp/disks vg01