PVG-Strict and mirroring

[Patrice Le Guyader]

Hello,

We have several volume groups spread on two disks array. The server has 4 FC cards, 2 on each array, so each disk has an alternate path on both array. vg01 has 3 disks; 3 on array A and on 6 array B. Each logical volume is created like this :

lvcreate -s g -t 90 -r n -l PE_Number /dev/vg01

/etc/lvmpvg is defined as following :

VG /dev/vg01
PVG arrayA
/dev/dsk/c23t0d3
/dev/dsk/c22t0d3 (alternate)
/dev/dsk/c23t2d3
/dev/dsk/c22t2d3 (alternate)
/dev/dsk/c23t3d0
/dev/dsk/c22t3d0 (alternate)
PVG arrayB
/dev/dsk/c26t0d3
/dev/dsk/c27t0d3 (alternate)
/dev/dsk/c26t2d3
/dev/dsk/c27t2d3 (alternate)
/dev/dsk/c26t3d0
/dev/dsk/c27t3d0 (alternate)

We are in a two nodes Veritas Cluster.
When we make a vgexport on node Alpha :

vgexport -svp -m vg01.map vg01

and make a vgimport on node Beta :

vgimport -sv -m vg01.map

After of course having copied the map and supplied additionnals disks in /etc/lvmpvg on node Beta, we have bad definition of vg01.
We expect to have primary on an array and mirror on the other but we have as primary some on array A and some on array B and so on for mirror disks.

If someone as an explanation ?

Rgeards
Patrice

[RAC_1]

What does lvdisplay on lvols under vg01 on both systems say??

[Senthil Kumar .A_1]

Hi,

can you attach the following output in this thread..We need the outputs on both the servers to explain effectively...

vgdisplay -v
ioscan -fnC disk
cat /etc/lvmpvg

Regards,
Senthil Kumar .A

[Carsten Krege]

When you vgimport a VG with the -s option, LVM searches for disks with the VG ID listed in the map file. It does not care and does not know about primary and alternate links as listed in the lvmpvg file.

During import you should make sure to import first the primary links and only then add the alternate links. THis works best without using the -s option but using the -f option.

Carsten

[Carsten Krege]

I should have mentioned that there is a vgexport -p -f option that allows to create the infile that you need for import on the other node.
However, you still need to make sure that the disk device files of the VG on both nodes are the same. If they are not you would need to modify the device files in the infile before vgimport on the other node.

Carsten

[Tiziano Contorno _]

Patrice, your (LUN on array <-> special file on the server) couple may be different on the 2 servers.
You should use a tool to identify the real LUN and its special file on the 2 servers. I use "xpinfo" with XP Storage.

Remember you can always play with vgreduce / vgextend on the VG on the alternate node to change primary / alternate pathing.

Regards.

[Senthil Kumar .A_1]

Hi krege,

If I'm can remember right , there is no "-f " option for vgimport, probably you are trying to refer to "ioinit" command which uses the "-f" option.

----------------------------------------
Anyway, Patrice, we can help you rectify the situation if you can give the command outputs we asked for.

Regards,
Senthil Kumar .A

[Carsten Krege]

from the vgimport manual page:



-f infile Import the set of pv_paths held in the infile to the volume group. This option is used as an alternative to specifying the pv_paths on the command line. Each pv_path must appear on a new
line in the infile. This option may not be used together with the -s option.

Carsten

[Patrice Le Guyader]

Hello,

Here they are.

and /etc/lvmpvg :

VG /dev/vg01
PVG pv1
/dev/dsk/c23t0d3
/dev/dsk/c22t0d3
/dev/dsk/c23t2d3
/dev/dsk/c22t2d3
/dev/dsk/c23t3d0
/dev/dsk/c22t3d0
PVG pv2
/dev/dsk/c26t0d3
/dev/dsk/c27t0d3
/dev/dsk/c26t2d3
/dev/dsk/c27t2d3
/dev/dsk/c26t3d0
/dev/dsk/c27t3d0

Thanks
Patrice

[Senthil Kumar .A_1]

Hi krege,

This is the man page I'm referring for vgimport.

http://docs.hp.com/en/B2355-90129/vgimport.1M.html

This does not show any "-f" option. Just curious, am'I refering an outdated manpage. If you don't mind can you share the link which shows "-f" option.

Thanks,
Senthil Kumar .A

[James R. Ferguson]

Hi:

Current 11i manpages for 'vgimport' and 'vgexport' describe the '-f' option:

http://docs.hp.com/en/B2355-60127/vgimport.1M.html

http://docs.hp.com/en/B2355-60127/vgexport.1M.html

Regards!

...JRF...

[Tiziano Contorno _]

Importing VG, from a map created with the -s option, scan PV in no particular order. The first path scanned for a given PV become primary. You have no control on this. You have to manualy vgreduce the primary pathed PV that should be alternate: in this way you promote the alternate to primary. Then vgextend with the just removed PV to set it as alternate path.

Regards.

[Patrice Le Guyader]

The previous information are in a situation I would call "clear". After reducing the mirror, pvmove PE from disk to disk, recreating the mirrors.

What I want is to export from one server to the other server without breaking my head.

If I understand what you're saying, when you import with "-s" option no matter of what's in lvmpvg and "PVG-strict" allocation policy on logical volumes ? Strange.

For me when import starts it was like this :

Due to the VGID in the map two possible disks to start, one on each array.
Doesn't matter which one you take first but I thought that after the first disk was choosen with lvmpvg and PVG-strict policy all the other would have been on the same array and the mirror should have been the others disks.

According to me the -f option is quite boring for going from one server to another if you have different controler id.

Any advise ?

Thanks
Patrice

[Tiziano Contorno _]

Sorry, but AFAIK ther's no "painless" import of shared VGs if you need exact PRI ALT couples.

1) Import
2) Check that each LUN on the array has the same special file on server A and B, if not, consider the "translation".
2) If needed vgreduce each VG of the primary path you want as alternate vgextending at the next step.

PVG files it's only a logical reference for the LVM commands.

Regards.

[Senthil Kumar .A_1]

Hi Patrice,

Follow the instructions of vanadio. He is spot on with his replies. Do not bother about the PVG groups. That we can sort out later. You can even create /etc/lvmpvg file later. As vanadio pointed out it is just a Logical group, used to differentiate disks b/w controllers.

First lets sort out the Primary and alternate link after vgimport and then fix PVG part.

If you are not sure of command sequence let us know, we will assist you with that.

Regards,
Senthil Kumar .A

[Senthil Kumar .A_1]

Thanks, James and Krege for the info. I will remember that.

[Patrice Le Guyader]

I don't know if I could explain my problem. My problem is that when I import my vg definition on second node there is what I call "cross extension". What I mean is without holding account of mirror only for primary; I have first disk of volume group on first array,second on same ,third on second array and last on first array.

It's not a simple problem of primary/alternate path definition of a disk.
I've already scripts that load balance that across controlers on one disk array.

Thanks for all gents.
Patrice.

[Tiziano Contorno _]

Ok, I got it. You are getting some mirror copies as your primaries on the other node.

I bet you have only different device names.

I don't know how, with your array, but you surely can check the real _LUN_on_Array_ distribution of your imported VG.

Regards.

[Patrice Le Guyader]

Hi all,

I've a possible solution to my problem but I would like to know if this scan order is in the H/W path order or in the controler one?

Regards
Kenavo
Pat

[jon2]

Patrice,
You could try the following:
vgimport -m mapfile -s vg01

#Import the vg with unexpected paths.

vgexport -m mapfile -f /tmp/disks vg01

vi /tmp/disks
re-arrange disks to be in expected order
primaries followed by mirrors.
vgimport -m mapfile -f /tmp/disks vg01

[Tiziano Contorno _]

Don't forget the "-s" option in the vgexport command! :)

[jon2]

Vanio, If you are referring to the vgexport (on the import host after importing with the -s) in my suggestion , do not use the -s. When we re-import them we do not want to use the vgid to select the disks, we want to select the disks by disk device name. If you put -s in the export it will put the vgid in the top of the mapfile and if the import does not use -s (which it should not) then vgimport will error on finding the VGID at the top of the file.

[Tiziano Contorno _]

He has to use -s option not to lose all the vg in the exporting node. No "-s" option if he doesn't care.

Or exporting without "-s" and then reimporting, in this case he has to unmount all lvs and deactivate the vg.

[jon2]

Perhaps I should try to clarify all the steps I was suggesting.

(I ommited to say this step as I assummed this mapfile was already available).

On first correct host to get mapfile:
vgexport -svp -m mapfile vg01
(Preview export - create mapfile with VGID in)


On second (incorrect) host:
After creating /dev/vg01 and group file.
vgimport -m mapfile -s vg01
#Import the vg with unexpected paths using VGID.

(We really mean to export them here - no preview and no VGID in mapfile - create list of disks instead)
vgexport -m mapfile -f /tmp/disks vg01


vi /tmp/disks
re-arrange disks to be in expected order
primaries followed by mirrors.
After re-creating /dev/vg01 and group file.
vgimport -m mapfile -f /tmp/disks vg01


To import the VG with or without -s does not require the first system to deactivate the VGS or lose them. (To activate the VG on the second node is a different matter).