Wild idea to implement SG

[Brian Street_1]

Hello everyone,

I have a wild idea to create a service guard cluster and I'm wondering if it will work.

I have already built a system which was slated to become a cluster. Due to hardware problems it was decided to abandon the idea and get one system up and running. That has been done with the following configuration:

mirrored root disk
SureStore SC10 disk array mirrored containing 2 filesystems (home and opt)

I'm almost ready to tackle the second system and I thought, hey! why not break the mirror on the first system and swap disks with the second system, change hostname and IP address and bring second system up.

The reasoning behind this is that I have installed a lot of software on the first system (both local and on opt) and I want to make sure the software is available on the second system as well. It seems like its doable.

The only difference between the systems (both N-class systems) is that 1 has 8 CPUs with 8 Gb of RAM while the other has half as much of both. The second system will only be used if the first one fails so this shouldn't be a problem right?

Thanks for your help,
Brian Street.

[Ron Cornwell · Accepted solution]

Rather than do this, I would look into making a golden image with Ignite-UX and installing the second system using that image.

[Sridhar Bhaskarla]

Hi Brian,

make_tape_recovery tape is the safest solution of all. It will take care of many things.

But if you insist to be *wild*, lemme put down the following

1. When you break the mirror (you have to vgreduce the disk which means you have to take out the lvols) theoritically you lost that disk. It may boot but not guaranteed.

2. However, you can use my "steal the disk" approach which may sound radical but works on hot-swappable systems. Replace the mirror disk with another disk and restore the configuration (mkboot, vgcfgrestore, vgsync process)

Now put the stolen disk in the same slot in the other N-class server boot it. Act as if the primary disk is lost and again follow the vgcfgrestore process.

Now this will work as long as you do not put it in the same cluster as of the other system. But once you decide to do it, you will need to 'vgchgid' the disks on the second server. Since you stole the mirror disk, it will carry the same VGID as of the primary server and when you do 'cmapplyconf', it will confuse the serviceguard.

Boot the system into LVM maintenance mode, export vg00, run 'vgchgid' on both the disks together and import them back as vg00. Run lvlnboot commands. That should work.

But if you can spare couple of hours, make_tape_recovery can save you a lot of headaches and fat fingers.

-Sri

[John Poff]

Brian,

Are your /opt and /home filesystems in vg00 or are they in another volume group? At first glance it doesn't look real good. Even if they are in a separate volume group, you'll still need to mount them up on the failover system and they will mount over any existing /home and /opt filesystems you have there. The /home wouldn't be too bad but /opt could make things interesting for all the software that lives out there.

As for cloning the systems, I'd go with Ignite. If you just have lots of time on your hands and you want to play around, you could try switching the disks, but if you really need the thing to work right I'd suggest using Ignite. The Ignite software will get every last thing you have without a doubt. No guess work.

JP [who likes crazy MC/SG ideas!] :)

[Brian Street_1]

thanks for all of your ideas. I just installed Ignite-UX on the first server and used it to install the second server since I only had 1 CDROM drive.

It sounds like using Ignite-UX to create the image and then installing the second server from that might be my best bet.

The opt and home filesystems would supersede the same filesystems on the second server (I thought this idea would work since the second system is only going to be used if the first fails) and I thought (only because I haven't RTFM yet) that as long as I had the systems cabled up correctly the second system is just sitting there waiting for the failover and not need the opt and home filesystems locally.

Am I reading this failover stuff right?

Brian.

[Sridhar Bhaskarla]

Hi Brian,

I would not suggest /opt to be a floating filesystem. There are services like rpcd that run from /opt and you might need them.

-Sri

[John Poff]

Brian,

The real trick is, what will happen to your running failover server when the primary server crashes, and the package starts up on your failover server and mounts up /opt over top of the existing /opt?

In a MC/SG cluster, typically you'll want to setup volume groups and filesystems that can be mounted on any node without clobbering anything. Since /opt is one of the operating system directories, it will make things interesting to come along and mount up a new one in the middle of things.

JP

P.S. Of course, I noticed the posting date for your thread so I'm keeping an open mind. ;)

[Brian Street_1]

John,

I must be spending too much time behind the keyboard...never even realized what day it was.

You guys have helped me from making a big mistake, so I really appreciate your input.

It sounds like if I'm going to install this SG Cluster I will have to do a lot of redesign...separate 3rd party apps/tools into another lv. The home filesystem is basically a development area and not really a home as we think of it so I don't think that filesystem would be much trouble...but opt is definitely a different story.

Thanks again.
Brian.

[Sridhar Bhaskarla]

Hi Brian,

It's not a mistake at all. What if you don't have a tape drive and a seperate server for make_net_recoveries and there is a lot of customization?.

But you have to be very careful while attempting to do it particularly because HP does not support it.

-Sri

[Duncan Edmonstone]

Brian,

The clustered apps data doesn't just have to be in a seperate lv, they need to be in a seperate volume group altogther, and hence on completely seperate disks.

I hope you have some disks available to do this!

If this is going to be (or is already) a production system, then I'd suggest at the very least reading the ServiceGuard manual cover to cover, and also consider attending a HP training course, and getting an experienced SG consultant to help build the cluster.

Poorly implemented ServiceGuard clusters generally have worse availability than non-clustered applications.

HTH

Duncan

[Brian Street_1]

The first system is almost ready to go into production mode. There were hardware problems that prevented me from creating the initial cluster configuration and it was decided in the essence of time to move forward with one system and explore the available options at a later date.

so, to make my life as easy as possible in the limited amount of time available to complete this project, it sounds like my best bet might be to:

1. create an image of root disk
2. create an image of /opt filesystem
3. break mirror of /opt and create a new vg/lv combo for the second system
4. restore images to second system

Does the restore of the Ignite-UX image allow me to specify the hostname before the restore starts? Does it allow me to specify filesystem sizes as during an installation?

Perhaps the time has passed to efficiently create a SG cluster and I should start looking at other uses for the second system?

As always, thanks for your input,
Brian.

[Pete Randall]

Brian,

Yes, the Ignite restore process will allow you to interact with it much like the initial install. You can adjust file system sizes, set host name, ip address, etc.


Pete