Adding disk in SG Cluster

[Rita C Workman]

I have a 3 way cluster. I know how to add disk and so forth...but generally I take the packages down, make my changes, do the vgexport and vgimport for everything and bring my packages up. I like to be on the safe side.
Today, I need to add disk to 1 node. So I can take down 2 of the nodes in the cluster with no problems. But one of the other nodes I can't take down; production running till midnight.
So my question is....
Since on this other node I only need to do the vgimport to keep it consistent with everything else...can I do the vgimport while it's up and running...safely.
Thanks,

[Anthony Goonetilleke]

Yes I am pretty sure that you can do a vgimport as long as the volume group is not mounted on that node safley. Although if its a production machine you might want to ring HP support to make 101% sure.

[Stefan Farrelly · Accepted solution]

To do the vgimport each node in the cluster needs to see and own the disks, so you cant do it on the non-active nodes because the disks are in use on the active node. You will need the package down so you can do the vgimport on each node in turn im afraid.

[Anthony Goonetilleke]

I thought she said all three nodes were in the cluster already (which would mean a vgimport wouldnt be a problem as long as the machine has access to the disks).... hmm maybe I misread it.

[Alan Riggs]

She did say the disks are all in the cluster, which means that all systems have the ability to import/activate the VG. However, if the package running on those didks is active, then the VG will be activated in exclusive mode on whichever server runs that packege. This means the VG cannot be imported to the other servers.

You do not have ot shut down everything on the "production" server, but you do need to halt the package in question.

[Rita C Workman]

OK..the VG I am adding disk to on the 3rd node, is not actively mounted on the other node.
So if I'm understanding....I could do the vgimport on the other node, while it is up and running, because the affected VG is not mounted on this other node at this time.

Right?

[John Palmer]

I agree with Anthony.

vgchange -c n vg??
vgexport vg??
vgimport
vgchange -c y vg??

should be possible with the cluster running and the VG in use on another node.

[Rita C Workman]

Gotcha !!

Thank you all !!

[Alan Riggs]

If the VG in question is not activated in exclusive mode, then you should be able to import it on the production server without interfering with any other packages.

Note: on teh vgchange -c options, the VG must be deactivated before these markers can be executed. (per man page)

[John Palmer]

I don't believe that the VG is actually 'activated' when it is imported - you have to vgchange it subsequently.
All you are doing is adding the VG to /etc/lvmtab and creating the device files.

If it was then could you not temporarily vgchange -c n to remove exclusivity while you did the import and then put it back?

[Rita C Workman]

That's exactly it....on this other node I really am only updating the /etc/lvmtab. This VG is not active on this node and neither is the package from the (3rd) node. So even though the box is up and running - I think I'll be okay doing this.

Like I said...I tend to be 'very safe' - so I never make changes on MC/SG unless I take it all down...it's just this time, I just don't have that option..

Thank you all again for all your wonderful help ! It helped.

[Alan Riggs]

John,
You are correct that the vgimport does not activate the VG. However, for the disks to be imported the system must take "ownership" of the disks (in order to write/examine LVM headers). This is not possible while the disks are activated in exclusive mode by another system within the cluster. You can use the vgchange -c n command to remove the flag indicating that the VG is part of a clustered environment, but as I noted above this requires the VG to first be deactivated. (ie -- the package must be down.)

That's just my opinion, though, I could be wrong.

[John Palmer]

Alan,

This is theory to me as well! I have a couple of L servers that I'm setting up at the moment. I'll give this a try at the earliest opportunity!

However I thought that the 'exclusive' use of a VG only prevented another node from activating the VG. I don't believe that you can prevent any other node from reading or writing the individual disks.

Regards,
John

[Alan Riggs]

Exclusive mode does not prevent read access. It does prevent write access from another node. The vgchange -c n command, though, requires a flag to be set/unset which indicates a write access. (since the status need be set only on one system in teh cluster, I am assuming this flag is set in the LVM headers *not* maintained in a system table on a specific server. I would be happy for someone with explicit knowledge of the mechanism to correct my assumption.)

Do test it, though, I don't have a cluster that I can play with at the moment and I am curious. If you don't feel like posting the results generally, please email me at riggsa@sensormatic.com.

Thanks.

[Anthony Goonetilleke]

OK guys no more replies for the extra 10 points she has been very generous already and I think she got the point :-) {right?}

pack of vultures!! (heheheh just kidding mates! bit of aussie humour! ((thats humour with a u)))

[Alan Riggs]

Boy -- let some people host an Olympics and they think they're something special!!

Next thing you know they'll think they have better swimmers than the USA. 87)

Good on ya, mate. Knock back a VB for me.

[Anthony Goonetilleke]

Oh mate I wish I could but I am in the US at the moment.. would you believe it!!
They sent me away exactly when the olympics started hmmm (not very happy at all!!)

[melvyn burnard]

Just for interest sake, the cluster id, written when a vgchange -c y is issued, is stored in the VGDA on EVERY PV in the VG.
This is why you should do a manual vgcfgbackup once the vg is made cluster aware!
The idea behind Exclusive mode for MC/SG is to prevent any other node activating the VG in writeable mode, although you are allowed to activate it in read-only mode with vgchange -a r.
This also allows a vgimport using the map file while the vg is already active on a node.

Hope that clears up a few points

[Anthony Goonetilleke]

Sorry Mate doesnt help with Alans VB question..Can you get VB in the US?

[Alan Riggs]

Thanks Melvin.

Anthony -- I have yet to find one. Heck, I'd even settle for a Twoey's Dark.

[John Palmer]

Testing has confirmed that you can indeed import a cluster aware VG on another node while it is activated exclusively.

I was also able to modify the VG configuration (added a disk) on the 'exclusive' node, reimport the VG on the standby node and fail the package over. The additional disk was recognised on the standby node.

So it looks as though 'exclusive' mode doesn't prevent anything regarding vgexport/vgimport. All it does is prevent a second node from activating the VG for writing.

If anybody has any further suggestions for testing, let me know.

Regards to all,
John

[Rita C Workman]

G'Day Mates...
Just 'cause I'm a woman and like the last word - I'd thought I'd let you know how it went.
Well, it does work.
3 node cluster...stop the package on the node that the package runs on. Do what you usually do there and vgexport the mapfile and rcp it to the 2 other nodes (that are up and running on their own packages).
Now before you bring that package on node 3 back up....go over to each of the other nodes and do the following:
vgchange -c n /dev/node3_volgrp
vgchange -a y /dev/node3_volgrp
**** here's the tricky part - - normally you would rm the group and recreate with mknod and do the vgimport...but it doesn't like that !. -- So at this point do a vgexport /dev/node3_volgrp and get it out of the /etc/lvmtab that way. Then go back and do the recreate of the group with mknod....then do the vgimport -vs /dev/node3_vg/vg.map etc. ****
Now just vgchange activity back to the cluster.
When you finish this on all your nodes, restart the package on node3 and voila ! All done.
I'd like to say this was a piece of cake...but it wasn't, because this node 3 runs FileNet and if anyone out there has ever worked with it, you know adding disk or making any modifications isn't *fun*. FileNet has a mind of it's own....so it was a long, long, long night....
**************************************
Anyway, your right I was generous with points...(but only cause someone told me once that I was 'kissing but' when I said something about points once before...so I decided to be generous, where they felt I was rude and insulting on this issue....).
Enjoy the points....they were earned and worth every bit !!
Thank you All !
Rita