vgexport vgimport -s option alternative for long device list

[Tom Geudens]

Greetings and Salutations,
(sorry, Highlander is my favorite film ;-)

I've got the following issue. I recently went to the MC/ServiceGuard course and they taught me an option of the vgexport/vgimport command that I didn't know yet :
vgexport -s
vgimport -s
In a nutshell, when you use the -s option with a vgimport, LVM searches the physical disks itself. This only works with a mapfile created with a vgexport -s.
Also note that the -p option is mandatory in the vgexport when using -s. This is logical in a ServiceGuard environment where ALL nodes need the same volumegroup definitions.

My question. Would this work in a NOT ServiceGuard environment to actually move volumegroups from one system to another (or to the same system where the devices changed) ? The reason I ask this is that I'm implementing a SAN at the moment. The EMC guys guarantee that all devicefiles are going to change. I'm talking about a lot of devicefiles. Which means a lot of searching and a lot of devicefiles in the vgimport. Unless, unless the following would work :
# Creating the special mapfile
vgexport -m -s -p -v
# Removing the old volumegroup
vgexport
*** Implementing the SAN ***
# Creating the new volumegroup
mkdir /dev/[volumegroup]
mknod ...
vgimport -m -s -v

I don't have any systems outside of production to test this ... So if someone can verify this I would be very gratefull (and I'll award points too :-).

Regards,
Tom

[Santosh Nair_1]

Tom,

Actually the -p option is not manditory when doing the vgexport. The reason that its included is that otherwise, the VG gets exported and the system no longers knows about the VG. The purpose of doing the vgexport is to create the map file and NOT export the VG and hence the pretend (-p) option.

As for whether this can be done in a non-MC/SG environment...absolutely. I've done it many times to move VGs from one system to another...or to re-import a VG after a new OS install. Works like a charm.

Hope this helps.

-Santosh

[James R. Ferguson]

Hi Tom:

The '-s' option of 'vgexport' actually results in the Volume Group ID (VGID) being written into the mapfile. This allows direct identification of disks that belong togeher (in the same Volume Group) when a 'vgimport' is performed, perhaps on a MC/ServiceGuard node or perhaps following a cold-install of an operating system on the same server. As Santosh noted, the additon of the 'p' option is "mandatory" in ServiceGuard node construction simplly because you don't want to remove volume groups from a running system! (nor could you, in reality, without having done a 'vgchange -a n').

Remember, too, that the basic 'vgexport'/'vgimport' mapfile is nothing more than a simple list of logical volume numbers and logical volume names. You do *not* need it, as some people presume, to do a 'vgimport'. In its absence, logical volumes are simply named according to the default nomencalature ("lovl1", "lovol2", etc.) as documented under 'lvcreate'. The VGID is the entity that identifies the physical volumes that belong together and can be scanned by 'vgimport' in the absence of mapfiles. Of course, its a volume group creation ('vgcreate') that a unique VGID is first written to a physical disk. In fact, the VGID is made up of the server's 'uname' idenentification ('uname -i') and the epoch timestamp of the 'vgcreate' action.

Another very useful trick when manipulating larger numbers of physial disks during a 'vgimport'/'vgexport' process is the use of the "outfile"/"infile" of pv_paths. Have a look at the man pages for both 'vgimport' and 'vgexport'. As noted there, the '-f' option writes "the current set of pv_paths for the volume group to the outfile. The outfile may then be used as the infile for the vgimport -f option. If used together with the -p option the volume group is not exported but the list of pv_paths is still written to the outfile. This may be useful to derive a list of pv_paths for the volume group or to use on another system which is sharing the volume group and which has an identical configuration."

Regards!

...JRF...

[A. Clay Stephenson]

You can certainly use the -s option. There is really only one downside and that it if you have optimized your LUN's to use multliple paths when you use the -s method typically the path optimization is lost. You can use the -s import to get the new primary/alterate path for each PV. If you want to optimize the PV paths for each host in the cluster you can then do a vgexport/vgimport using the now known PV paths.

[Tom Geudens]

Hi again,
Just to be clear ...
1. From the manpages of vgexport
"... When the -s option is specified, then the -p, -v, and -m options must also be specified ... "

So the -p is mandatory in this case, right ? Which means that I do need the 2 vgexport commands to get the job done ?

2. Losing the "multiple paths"
Clay, I don't quite follow what you mean "exactly" with this. Will I only get the primary link in the volumegroup ? I assumed that - since "-s" actually scans the disks - he would find both (we've got 2) "links" to each disk to contain the VGID and thus add them both to the volumegroup. I will definitely lose my powerpath definitions (and I am prepared for that) but I didn't think I was going to lose one of the links ...

Regards,
Tom

P.S. Thanks for the quick responses. Points will follow (just want to keep the rabbit away a little longer).

[Michael Tully]

Hi Tom,

I am fairly certain I know what Clay means.

If you have both primary and secondary paths
utilised when the export is done, when you go
to import these same disks, you will get the
path that was used for the primary only.


Example:- Primary path is /dev/dsk/c13t1d5
When I go to import this disk again, the import
will only have /dev/dsk/c13t1d5 and not the alternate link.

PV Name /dev/dsk/c13t1d5
PV Name /dev/dsk/c16t1d5 Alternate Link
PV Status available
Total PE 2157
Free PE 749
Autoswitch On

The -p is a snapshot or preview which does not wipe the lvmtab file, /dev/mygroup etc but creates the mapfile.

HTH
Michael

[Tom Geudens]

Hi Michael,
Maybe a little more explanation ...
The point of this "exercise" :-) is checking how "vgexport -s / vgimport -s" works exactly ... since my first experiment (!) will be in production with a system that has access to approx. 70 disks (times 2 for the links).
To make things even more funny, these 70 disks are part of a range of 150 disks. The system sees ALL these disks ... which makes for a lot of devicefiles.

I know this is not an ideal situation, but I've been given no choice (and no environment to test it either).

If you guys can convince me of the fact / way that this works, I will gain a lot of time. If not, I'll have to search for those 70 disks and add them to the vgimport command.

So there's the situation,
Regards
Tom

[Tom Geudens]

Hi Michael,
Maybe a little more explanation ...
The point of this "exercise" :-) is checking how "vgexport -s / vgimport -s" works exactly ... since my first experiment (!) will be in production with a system that has access to approx. 70 disks (times 2 for the links).
To make things even more funny, these 70 disks are part of a range of 150 disks. The system sees ALL these disks ... which makes for a lot of devicefiles.

I know this is not an ideal situation, but I've been given no choice (and no environment to test it either).

If you guys can convince me of the fact / way that this works, I will gain a lot of time. If not, I'll have to search for those 70 disks and add them to the vgimport command.

[James R. Ferguson]

Hi Tom:

The requirement to use the '-p', '-m' and '-s' options together is that you don't want to actually export the volume group information from the running system ('-p'); rather, you *want* to create a map file '-m' because you *want* to be able to include the VGID ('-s').

You are *not* going to loose any pv_paths furing 'vgexport' and 'vgimport'. Rather, you have no assurance that the *order* of paths will be optimial following the import process suing the '-s option. That is, if you have situation where exactly one-half of your paths are are one controller (as primary links), and exactly one-half are on the other controller (as alternate links), then it is quite likely that you will *not* end up with this again. All this means is that afterwards you will have to 'vgreduce' and 'vgextend' various pathes to swap their order as primary and secondary. Remember, a primary pv_path is simply the one declared first in /etc/lvmtab. A secondary pv_path is simply the path declared second(arily).

One of the values to using the '-f' option I cited, is that if you know the device file nomenclature on the "sending" and on the "receiving" systems, you can 'vgexport" with the '-f outfile' option; *edit* the resulting file of pv_links; and 'vgimport' the editted file. In this way, in one step, you can specify and maintain the primary and alternate (secondary) links such that a perfect 1:1 balance of alternate links occurs accross two disk controllers -- optimal performance. Since LVM does not load-balance between primary and seondary paths, it is important to establish the balance of IO yourself.

Regards!

...JRF...

[Pete Randall · Accepted solution]

Tom,

I just went through this exercise this weekend when I wanted to re-connect my directly attached FC60 array through a Brocade switch. Despite what the man page says about the -p option being required, it works just fine without it. I ran "vgexport -s -m /tmp/vg01map /dev/vg01", switched the cables around, ran "vgimport -s -m /tmp/vg01map /dev/vg01" (after recreating the vg01 directory and running the mknod on the group file, of course). It worked like a champ.

Pete

[Tom Geudens]

Hi,
<<< be patient with me here, the penny / dollarcent / eurocent is about to drop :-) >>>
So, I use "vgexport -s" to create the special mapfile, drop the volumegroup with an ordinary "vgexport", connect both the EMC-array and the server to the SAN, do a little zoning magic ... and then I use a "vgimport -s" to recreate the volumegroup. I will have ALL devices although alternate / primary might be mixed up. I do not (immediately, I may / will rectify this situation at a later time) worry about this since PowerPath does the loadbalancing anyway.

Is this a correct summary ?
Regards,
Tom

[Tom Geudens]

Hi Again,
YES ... it dropped.
Pete (and the others), I only saw your answer after I posted my latest reply ... but your answer was the "assurance" I needed.

If anyone still wants to add thoughts/comments/do's/dont's ... it's all welcome (and will be rewarded), I'll do the followup of the thread until I did the conversion myself (and I'll post the results :-).

Thanks for your help (learned a lot about LVM these last two weeks).

Regards,
Tom

[James R. Ferguson]

Hi (again) Tom:

If you look at the man pages for 'vgimport' (at least on 11.i) you will note:

"Since the -s option causes a search on the system for each disks with the same vg_id. When vgimport reconstruct the newly imported volume group entry in /etc/lvmtab file, the order of disks could be different than it was before. And the following will happen:

The designated primary and alternate link might not be the same as it was configured before.

Alternate links will be added to the importing volume group even if they might not be configured in the volume group initially."

I agree, however, that if you are using EMC's PowerPath, this is probably moot.

On another point, the man pages are very clear: the use of the 's' option means that the volume group is not removed from the exporting system. The use of the '-p' option (on import) means that /etc/lvmtab is not updated.

Regards!

...JRF...

[Pete Randall]

Tom,

Another interesting side-note on the alternate link discussion. The number of alternate links, in my case, at least, doubled. It makes sense to me because I've got two paths to the switch, and then two paths to the array - so there's four ways to get to each of the physical volumes. You'll see this in an ioscan but you'll notice it most when you issue an lvm command. For example, when I boot the system and the VG's are activated, it takes roughly twice as long to do the activation. I paniced at first because I thought all my I/O was going to be slow but it definitely seems to be just the lvm commands scanning the extra paths.

Good Luck,
Pete

[Tom Geudens]

Hi Pete,
My EMC consultant told me as much (although he didn't say anything about the longer boottime :-). EMC's PowerPath (our loadbalancer) - which already makes the boot take longer - is probably going to have a "field day" too with all those posibilities ... :-).

Regards,
Tom

[Nick Wickens]

Hi Tom

Having spent the last 18 months mucking about with our SAN implementation I am not sure from what you have said that you will be able to use a mapfile for this if, as you mention you are expecting your device ID's to change.

In my case everytime we changed something the LUN's stayed the same but the instance number for the FC card changed. This meant that any mapfile that held details of the physical volumes would surely refer to the wrong devices if you tried an import ? (ie device c16t0d5 would become c23t0d5)

I would suggest that in addition to that in the answers above you also write a script to output your current volume group and device ids in such a way that you can then use that information to peform a vgimport without the mapfile, firstly amending the device ids (hopefully just the instance number of the card) to the new instance numbers.

[James R. Ferguson]

Hi Nick:

It's not the mapfile, but the 'outfile' of 'vgexport' which becomes the 'infile' of 'vgimport' that holds the pv_paths. I've re-assigned devices using this technique. You simply edit the file of 'pv_paths' as required. You can also establish the order of devices such that you balance primary and secondary pv_links among/between disk controllers during the 'vgimport' in one pass.

Regards!


...JRF...

[Tom Geudens]

Hi Nick,
... wow, a graduate already, congrats ...
When using "vgexport -m -s", extra information is written to the mapfile, the VGID to be exact.
When using "vgimport -m -s", this VGID is matched against the LVM-information that is physically present on the disks (the vgimport actually scans all the disks). No physical volumes need to be added in the vgimport list !

Normally this command is only used in cluster environments, where volumegroups are shared. But as you can see above, this will also work for "ordinary" systems.

Regards,
Tom

[Tom Geudens]

Hi again,
It worked ... It worked like a - how do you say that again - ah yes ... It worked like a BOID (sorry if this happens to be offending language, I'm "Stephen King"-taught :-) !

My flashy new SAN is finally starting to roll.

And just in time for my "anual review too" (see Michaels thread "What makes us tick") ...

Thanks (again) to everyone that helped !
Tom

P.S. I do not know if anyone is keeping track, but this time the beer is on me :-)

[Michael Tully]

Well Done Tom!!!!

Well where is it? I'm waiting (2:38PM and the sun is around the yardarm!)

Cheers
Michael

[Tom Geudens]

Hi Michael,
Would you believe that I actually expected that reaction from you ... :-) ? Well, 06u40AM is a little early for me to start on the heavy stuff (coffee is needed more at this point), but especially for you :
http://beer.trash.net/beerpage.php?beernum=909

It's about the best we've got over here (no, I don't want to start a discussion about beer, Dan would flame us for not being professional :-) ...

Enjoy it,
Tom

P.S. If ever we meet "life", make sure to remind me I owe you a couple of real ones !

[Michael Tully]

Tom,

Sometimes a little fun is in order, especially after solving and implementing something this big. Here's one of mine at number 8.

http://beer.trash.net/beerpage.php?beernum=65

Cheers
Michael

[Rainer Kranner]

Hi All

thank you very much for your professional comments.

Do I right, if I do switch Devicefiles from Quickloop to Fabric in a SAN with a running
MC/SG Cluster, then i can switch all Pakages to one Node and make :

1.) switch all pakages to 1th Node
2.) vgexport -p -s -m /tmp/vgxx.map /dev/vgxx
3.) switch SAN Device files on 2nd Node
4.) vgimport -s -m /tmp/vgxx.map /dev/vgxx to
the 2nd Node with new Devicefiles.
5.) switch Pakages back to the 2nd Node
6.) switch SAN Device files on 1th Node
7.) Change Clusterconfig with new
Clusterlockdisk Devicefile

let me know if i wrong or right or correct me.

thanks in advance
Rainer