LVM, Oracle, MC/SG - "Shared" vs "Exclusive" Cluster VG?

[Greg OBarr]

I'm running a MC/SG Cluster with Oracle. I know I need to upgrade the versions (HP-UX 11.00 and MC/SG A.11.09).

I "inherited" this cluster configuration, and the Cluster VG is configured such that it is mounted in "Exclusive" mode by the node running the database package.

Now, I wanted to add a third PV (array) to the VG and split it off to do backups. But I wanted to be able to shut down the database, split off the third mirror copy, restart the database on the other two copies, then mount the split copy on another node and run backups on the other CPU.

This operation is supported with LVM, but with the "Exclusive" VG activation, I can't mount the split copy on another node because I can't activate the VG there.

Since we're using filesytems, not raw partitions, there is risk of data corruption if I change the VG to "Shared" and one of the active LVs is mounted from both nodes simultaneously.

Is anyone else doing anything like this in an MC/SG cluster?


-greg

[Sanjay_6]

We used to do this with EMC SRDF.

Sync the two copies of R1 and R2 and after split, activate the R2 copy on another node and backup from there.

Hope this helps.

Regds

[Steven E. Protter]

Also consider Online JFS filesytem snapshot.

You down the database, create a temporary lvol to store changed blocks, bring up the database and get a cold backup while the database is runing.

Quite slick.

SEP

[Geoff Wild]

We use bcv's on EMC - like a third mirror - but not seen by the host - only the backup server.

In MC/SG - we do the default for vg's:

vgchange -a e

The backup server syncs the bcv's, then put's the DB in hotbackup mode, splits the bcvs, then mounts the file systems on the backup server (after doing a vgchange -c n, then a vgchange -a y).

I wonder if you can split the mirror on the production, then on alternate node, do a vgchange -c n, then vgchange -c y? You would have to do a vgexport first of the vg, then a vgcreate - specifying only the disks that have been split.

Reason for export, that node knows all the disks and would do a vgchange on all of them - which wouldn't work cause it's active on the prod node.

Rgds...Geoff

[Rita C Workman]

Like the others before me...we too use EMC's SRDF and BCV technology.

But since your obviously not. You might play with this. Try creating a dummy pkg & small dummy vg. Do what you said about mirroring to another array and then split it off.

Now to mount those other disks (now split off) try changing the vgid on them by doing vgchgid (see man vgchgid). You would probably have to change all the disks at the same time in one command. Then see if you can activate the vg after and then try to mount these disk/copies to a different server that is absolutely not within your cluster.

I don't know that this will work...but hey, it's a thought.

Rgrds,
Rita

[Rita C Workman]

You know if your mounting them on another server, that is NOT in the cluster, you may not even have to vgchgid.....

hmmm...just a thought,
Rita

[Sundar_7 · Accepted solution]

Hi Greg,

Even though your VG is activated in exclusive mode in one node, you could still activate the VG in "READ-ONLY" mode in the other node.

That is supported and should not cause any data corruption.

1) Add the third PV and establish mirror
2) Shutdown the database and split the mirror
3) Start the database using the first other 2 PVs
4) Now activate the VG in read-only mode in Node2 - vgchange -a r /dev/vg01

I assume it is just for backing up to the tape.

[Greg OBarr]

Sundar: That sounds like the perfect solution. I will have to try it in my test environment (when I finish setting it all up :)

Rita: That's also worth trying to see if a node with access to the array, but not in the cluster would be able to activate the VG.
I don't know if I'd use it on the production systems, but I'd like to see if it works. Call me paranoid if you like, but I've had systems reboot on me before doing things like mounting fs in an LV r/w on one system, then r/o on another, then trying to umount on the system where it was mounted r/w from while still mounted r/o on the other system. May have needed a patch or version upgrade (I'm far from the latest version at MC/SG 11.09, vxfs 3, and HP-UX 11.00)

Steven: The Online/JFS snapshot is a good idea too. I could have another "shared" VG with a filesystem there that could be mounted from both systems.

Thanks for the replies everyone. Looks like I could solve a lot of problems by using EMC.