reboot vs. uname -S / hostname <hostname> and more

[Emiel van Grinsven_1]

Hi,

I'm having an issue here that I'll try to explain briefly.
We have 2 nodes, 1 is production, 1 is test.
The systems are exactly similar (HW and SW).
OS is exactly similar (patches etc)
the customer didn't want to use serviceguard but wants a disaster recovery option. So they thought of the following:
use scripts to gather specific system info, user info (home dirs) etc. and ftp these every evening to the test system. Build scripts to install these files and attach the PROD storage to the test system.
When disaster strikes, prod is down, test will be altered to function as the prod system. hostname and ip address will be changed so that it looks like the prod system is up and running.
OK, you see what I'm facing here? Terrible solution but I'm stuck with it..
This all is working already (in test phase) but what i would like to know is.
Should I expect any problems when I don't reboot the test system and let it boot with a new hostname/ip-adres but use # uname -S and # hostname to alter the ... hostname? All apps will be down at this point but the system will be in runlevel 3.
Any idea's if this will create a problem?
Thanks in advance

Emiel

[Techtricks]

Dear Van,

You can't change the hostname just
by typing #hostname. The #uname -s hostname
will always give you the HPUX the ver of OS you are running.

If your system i,e production and
test servers are working fine as the fault
redundance without any problem using your
customised script, you can check the same
at the test phase by removing network connection to the production server.

In this case your test server should get
alert to get activate with the new ip and
hostname. If this is not working everything
you have struggled or enjoyed to prepare
script will be waste and it will not be the
redundance server.

Thanks and regards

Techtricks

[Emiel van Grinsven_1]

Hi,

Actually, I tested with a system and after i changed the hostname and the uname the system replied in every case the new given name..
Sure that the unique ID will not be the same but I don't think this will be a problem.
What i'm wondering about mostly if something will stay in memory that refers to the old nodename and will give me a problem.

[Thierry Poels_1]

"... attach the PROD storage to the test system ..."

If disaster striks on your PROD storage, you'll be left with two nice servers but without operational data.

Well, on the other hand this solution gives an extra benefit to the test server.

regards,
Thierry.

[Techtricks]

Dear Emiel,

Check for the /etc/rc.config.d/netconf file
after changing the HOSTNAME, check for the
HOSTNAME entry.

If it is reflecting the hostname in the conf
file, there is no such things to keep it in the memory for the hostname change.

Regards

Techtricks

[Emiel van Grinsven_1]

About storage.
This is all on EMC symmetrics devided over 2 sites.
the 2 systems are about 10KM apart and on the site where the test system is, prod storage is available with a srdf mirror (read only)
When disaster strikes, the mirror will be altered so that the test site storage will become leading. I'm not that worried about the storage actually..

netconf is not altered with uname or hostname, this is a manual action that we will perform when the hostname will be changed.

Please understand that if it was my decision we would not use such a procedure that is waiting for something to go wrong but would use serviceguard.. but.. it's not my decision

[Thierry Poels_1]

"... EMC symmetrics devided over 2 sites ..." Now you're talking!! ;-)

The nodename is generally not so important. Users (applications) generally connect via the IP address.
If the application does not check on the nodename, I would only change the IP address.

regards,
Thierry.

[Emiel van Grinsven_1]

the apps on this system get data from incomming ftp sessions that use the hostname...
we were just wondering if a reboot is needed or not to 'clean the system'.
If not, it will save us some time.

[Gordon Morrison]

May I ask *why* they don't want Serviceguard? The only reason I can think of is that they're too cheap to pay for the S/W & licences, in which case I'll say:
a) you forked out for a Symmetrix, dust off your wallet!
b) compare the cost of Serviceguard to the cost of man-hours & downtime that this so-called "solution" will involve when (not if!) you have a disaster.

Stepping down from my soapbox, I'll now say that although changing the hostname & IP address of a system can certainly be done, there is often some niggly little thing that is overlooked and bites you in the bum. So I suggest one of the following 2 alternatives, in order of preference:
1) keep your hostname & primary IP address for each server , but have a spare LAN card, and a name & IP address for the service that you're running. In case of disaster, configure the spare lan card with the address of the service, mount the EMC disks and away you go. This is basically what serviceguard does, without the bells & whistles.
2) In the event of disaster, take an Ignite tape from your production server and boot your test server from it.

[Emiel van Grinsven_1]

guess you don't even want to hear that these systems are superdome partitions, fully loaded? (and almost unused)
Yeah, you're completely right that they're to cheap but there's nothing I can do about it more..
In the end it will cost more and it's much less fault tolerance.
we need the hostname for the ftp users who connect on hostname.
Ignite is an option but because of the HW it takes a long time to finish..
My biggest problem is that I was never consulted, it was simply put on my desk with the notification: "make it happen".

Yep, they want it all. But enough about politics :)

Are there any problems that cannot be overcome without a reboot?
Now I can still claim that a reboot is needed, if later after i claimed otherwise it shows that i was incorrect I'll have a problem.
In such situations it's important to make sure you can't be the one to blame... sad but true

[Gordon Morrison]

In Unixland, most things can be overcome without a reboot, but sometimes a reboot is quicker.
:o/
(Too bad you can't reboot your managers:o)

Using the name of the service will work for telnet, as long as the service is associated with an IP address, even if that address "floats" between various servers (as long as your network is set up to allow it). Just make sure that the service name is defined in /etc/hosts on all relevant servers and in DNS, and make *VERY* sure that only one server is actively hosting that address at any one time.

[Emiel van Grinsven_1]

yeah, that's another issue we have to overcome.
What if the original node will come up again?
We thought about it and since it's a manual action it can be halted or booted without networking but still.. you know that this always happens when some new guy has standby etcetera..

the network is not an issue.

in 2 weeks we have to test this whole procedure. I think we already overcome most of the potentional problems.
Thanks for the help!

Emiel

[Gordon Morrison]

Hi Emiel,
Again, we're re-inventing the wheel here. This is more or less how Serviceguard does it:

The "spare" lan card (i.e. the one that is used for the failover service) is NOT configured automatically, so if a machine reboots, that card will have no address. The card should be configured & activated in the startup script for the service, which also does NOT start automatically. It would also be a good idea to put a simple test in the startup script to see if the service is already running, say, try to ping it & if you get a response, abort the startup.
Of course, this means that every time the service fails, it will need to be restarted manually, but that's so much better than suddenly having 2 machines with the same address on the same network!

Of course, Serviceguard would do all this automatically & seamlessly in a couple of minutes, but then you already know that.

p.s. Having re-read this thread, I see you have 2 EMC's in dual sites. So I presume you also have 2 Superdomes in dual sites? (and probably 2 managers...)
And still no budget for software. I sympathise. I really do.

[Travis Harp_1]

A couple things here...
First, man this is a scary option but you know that already. 8)
Second, you could turn off autoboot on the systems so that it will sit at the BCH (the boot prompt) waiting to be told what to do.
That way if the primary drops and tries to reboot, it will only make it to the BCH and sit there. If you have the LAN/Web console configured, it would be a simple and remote process to boot it up the rest of the way.