Oracle on each node in Serviceguard cluster

[Keith Wesson]

I am managing a two node K-class cluster running as an Oracle database server. Right now the Oracle software and databases are configured to fail over during a Serviceguard TOC. I was considering reconfiguring the servers so that each had an oracle software installation. Serviceguard would only shutdown the instance and move the data volumes (and, not the Oracle software) to the failover node. I feel that with this configuration we could gain utility from the second server which, now sits idle. Perhaps we would move our test DB onto it. We could also use the second server for practice recoveries. Our corporate agreement with Oracle would allow us to license the second server at no additional cost. Are their any other drawbacks to this plan? Do most users try to keep their failover servers idle while waiting for a TOC?

[Carsten Krege]

Not knowing exactly what your plans are, here are some things you might want to consider:

0) I believe most customers configure local executables and put only the data on the shared bus.

1) Many customers use the failover server for other applications/packages. If you don't do this, you waste a lot of money for an idle server. If you want to run another instance of Oracle on the failover server, you have to make sure that they do not disturb each other (different config files, log files etc.)

2) You should also make sure that the failover node has either the ability to run both Oracle instances at a time, stops the test instance once the other productive package fails over or guarantees the resources needed (CPU, memory, disk bandwidth etc) for the productive instance (e.g. by using the Process Resource Manager software).


Carsten

[Carsten Krege]

One thing to add:

3) You might want to check whether SG OPS Edition (integrating Oracle Parallel Server (with Oracle 9.x aka known as Real Application Cluster (RAC)) is an option for you.
This would require a complete redesign of your software environment though and might not be appropriate for your requirements but could also boost your performance and make use of both servers, accessing the same database at the same time.

See the SG OPS manual
http://docs.hp.com/hpux/pdf/B5158-90031.pdf

Carsten

[Duncan Edmonstone]

Technically there is absolutely nothing wrong with what you are proposing, and it is certainly a more effective use of the kit you have paid for!

The challenge, particularly when the 'standby' node is being used for testing is one of control. Test systems are often administered in a more 'relaxed' way than production systems, but you have to remember that although you have a test database on your standby server, it is still effectively a 'Production System', and should be managed as such.

If you can live with this, then you don't really have a problem.

[A. Clay Stephenson]

Hi Keith:

I have a 2-node K-box cluster and do use it to support 2 Oracle instances. In my case, one of the servers is the primary for instance A and the other is the primary for instance B. After using it in both ways, I find if safer to actually move the executables (including the listener) with the package. It makes it much easier and safer to do an Oracle upgrade.
There are some gotchas: 1) Make sure that you have enough resources and big enough kernel tunables to run both packages on one node. 2)
Make sure that you test after patch installations that the packages will run on either node. You can run into linkage problems when libraries are changed on one node and not on the other. Make sure that you apply such patches to both machines in a timely fashion.

My 2 cents, Clay

[Christopher Caldwell]

One added benefit of your suggested topology, is that you can patch the stand-by instance of the database server. You can fail over to the patched instance. If there are problems with the patched instance, you can fail back to the un-patched instance.

[John Poff · Accepted solution]

Hello Keith,

We are doing just what you described. We are running 3 V-class boxes in a cluster. Two of them are production and the third box is for failover. The third box also runs a couple of test and development Oracle instances, which are in MC/SG packages. If one of the production boxes goes down, the control scripts for the product package will halt the test/dev package on the failover box and bring up the production package.

It works great for us. As Clay mentioned you'll need to stay on top of your kernel parameters and patches. I agree that the boxes are too expensive to be sitting around waiting for a failure.

The other thing to add is that if you setup your cluster this way, you should test it often. We set aside one Saturday a month for maintenance (patching, hardware upgrades, etc.). I test a failover of all the production packages to the failover box each month.

JP