clustered volume groups mirrored and imported to another system

[Eric Malzahn]

Hello,
We have a set of clustered volume groups that we split off a copy of (EMC Timefinder) and mounted on another system. When this system rebooted, it didn't want to activate the volume groups because they were set with the vgchange -c y on the previous system. I have a few questions regarding this:
1) Do I need to do the vgchange -c n in order to make these volume groups activate and mount normally upon reboot?
2) Is the only effect of leaving the VG's set with -c y that they won't activate and mount upon reboot? (I noticed that I can still reimport them again without changing to -c n).
3) How can I see if a VG has the -c y set?
4) What's the whole point of the -c y anyway? Does this just prevent mounting VG's on both nodes at the same time? How do the originals mount automatically on my SG cluster? Does the MCSG software control that?

Thanks,
Eric

[Steve Lewis]

1. Yes, alternatively if you want to remove MCSG, activate them manually one at a time in /etc/lvmrc using 'vgchange -a e' and set AUTO_VG_ACTIVATE=0.

2. No, its the value of AUTO_VG_ACTIVATE in /etc/lvmrc that determines whether VGs are activated. Vgimport doesn't activate, you have to manually activate afterwards.

3. vgchange -a y . If it fails then it will say "Activation conflicts with configured mode" or something similar which means that the VG is currently -c y'd. vgcisplay may also give a clue.

4. Yes it is the mechanism that prevents activation on more than one node at once and protects your data. The originals mount on your cluster without issue because the package compiler handles it all. These days you don't even have to vgchange -c y before you build the cluster as MCSG does that for you as well. You just have to ensure that the VGs have been imported to each node.

[Sanjay_6]

Hi Eric,

vgchange -c y means the vg is cluster aware. To make the vg cluster unaware you have to do vgchange -c n vg_name. When the cluster configuarion is applied, it makes all the vg involved in the cluster configuration as cluster aware, to be used by cluster packages.

On a cluster node, a cluster aware vg is activated using vgchange -a e /dev/vg_name. The node activates the vg in exclusive mode for use on that node.

Hope this helps.

Regds

[MANOJ SRIVASTAVA]

Hi Eric


Most of you answers lie in man vgchange ,

-c is a must for the cluster env.
y marks the volume group as a memeber of high availability cluster

if the vg is deactivated using -a n option then -c y n is to executed to activate , even if the volune group is activated on one node you can still mountit on the other but with warnings , it is jsut a flag set to mark it as availbe to the cluster.

The opther option is to change it in /etc/lvmrc and then specify what to mount at the time of booting .

The other option works is bcoz you are exactly not using the BCV's for the MC Service gaurd ,

Manoj Srivastava

[Eric Malzahn]

Thanks for the information. I went back and looked at my import script and it does, in fact, do a vgchange -c n. I just have one final question:

Do I have to do a vgchange -c n (or vgchange -a e) every time? I would have thought my first vgchange -c n would have permanently turned off the "cluster aware" attribute for all future reboots. This is not what I'm seeing.

Thanks,
Eric

[Duncan Edmonstone]

Yes you do need to change it every time. Remember that timefinder is mirroring *physical disks*, and the information on the configuration of the volume group is stored on the physical disk in a reserved area (called the Volume Group Reserved Area, or VGRA). So every time the BCV is re-synced with your clustered volume group, the old VGRA is written back over the disks.

HTH

Duncan

[Sanjay_6]

Hi Eric,

Once you do vgchange -c n for a volume group, the cluster awareness for the vg is turned off permanently. To make the vg cluster aware once again, you have to do vgchange -c y /dev/vg_name.

You should not have to do vgchange -c n /dev/vg_name for the cluster aware vg every time you reboot the box if the vg has been made cluster unaware once.

Hope this helps.

Regds

[Duncan Edmonstone]

Hmmm, I've just re-read what I have written there, and of course Timefinder isn't mirroring *physical disks*, its mirroring EMC Hypervolumes (LUNs) - but the point is that they appear as physical disks to the HP servers.

Cheers

Duncan

[MANOJ SRIVASTAVA]

Hi Eric

There are two issues:
1.If you are just working on the BCV's then you donot even have to use the -c option , jsut a vgchgid and a vgimport shoud work .

2.If you are talking about the physical disk for the Service GAurd then you will have to use it every time you boot or set the control script to run it.


Manoj Srivastava

[Eric Malzahn]

These replies all make sense, but here's what happened and what I saw:

1) sync and split the BCV's
2) vgimported the volumes onto another system with no MCSG.
3) did the vgchange -c n, activated the vg's and mounted the volumes.
4) everything worked fine for a few days.
5) rebooted the box
6) on reboot the volumes didn't come up.
7) once the vgchange -c n and vgchange -a y were issued again, everything could be mounted.

Maybe I should reboot again and see if this happens again...

Thanks,
Eric

[MANOJ SRIVASTAVA]

Eric


When u rebooted the server the BCV's got umounted and also the volume group got unacitvated if the /etc/lvmrc the option is set for AUTO_REBOOT . You can do the following , in /etc/lvmrc set the auto_vg_activate flag to 0 and then maually specify which volumes has to eb deactivated and reactivated at the time of reboot . In your case since this option was set to 1 is deactiveted the volume group so you have to reactivate them again .

We have a similar structure ansd in case the node on which we mount the BCV's has to be rebooted after the BCV's are mounted we just run a mount for the VG's ofcourse the /etc/lvmrc auto_vg_activate is set to 0.


Manoj Srivastava

[Eric Malzahn]

Now that the VG's are set to -c n, I don't understand why I have to do anything special with lvmrc to bring these volumes back each time. The volumes are specified in fstab. I would think this would be enough to bring them back after reboot.

[MANOJ SRIVASTAVA]

Eric

This is because when you reboot and the autio_vg_activate option is set to 1 then it will only activate all the volume groups defined in the file .The volume groups get deactivated . /etc/fstab only contain the mount information for the activated volume grouos , when you are rebooting the machine if you see the messages coming on the console that it will try to activate the volume groups ( either all of them if the option is 0) or the ones specified if the option is set to 1.

To know the status of the VG just do a vgdisplay -v /dev/vgXX


Manoj Srivastava

[Sanjay_6]

Hi Eric,

Check the file /etc/lvmrc on this system and make sure the variable "auto_vg_activate" is set to "1". If this is set to "0", the vg's will not be activated automatically during system boot except the ones mentioned in this file to be activated.

Hope this helps.

Regds

[Eric Malzahn]

Thanks for your help, and I hate to drag this out further, but I think the last 2 responses state the exact opposite of each other. Am I wrong?

Anyway,
My auto_vg_activate is 1 and I specify no vg's in the file. Yet on reboot, while it should activate all my vg's, it doesn't activate these ones that were previously set to "-c y". And this I don't completely understand still...

[Steve Lewis]

Let me explain a bit more.

The file /etc/lcmrc determines which volume groups are activated at boot-up, not the ones activated by MCSG. The package control file determines the VGs activated when the Serviceguard package comes up. You should set AUTO_VG_ACTIVATE=0, to stop HP-UX trying to activate volume groups controlled by MCSG, when the system boots up.

Normally the filesystems controlled by MCSG are also held in the package control file, not /etc/fstab. This is to stop it trying to mount filesystems it shouldn't, or stop it trying to mount filesystems that are on logical volumes controlled by MCSG.

If you have split some mirrored disks and want to activate this VG as a separate system, not synced with the other side, then it is easiest to keep AUTO_VG_ACTIVATE=0, then add that specific VG to the manual activation procedure on your separate node, using vgchange -a e.

This will ensure that your data is protected, in case you mix up some disks. If you undo the clustered option and activate VGs using vgchange -a y, then it will activate the VG even if it is already activated on another node, giving you the potential to corrupt data.

Its best to keep the VG clustered ( vgchange -c y / -a e ) if the disks are attached to more than 1 node at a time. Thats my opinion.

[Steve Lewis]

Sorry thats /etc/lvmrc, not /etc/lcmrc.
Steve