Operating System - HP-UX
1836623 Members
1929 Online
110102 Solutions
New Discussion

Re: Package Address IP Change

 
Chad Brindley
Regular Advisor

Package Address IP Change

Hi,

We run HPUX 11:23, Oracle 9.0.2.4 with SAP R/3 4.7.

We have MC Serviceguard cluster, we have a two node cluster.

Our cluster name is SAPPRD, our two nodes on the cluster are COVSAPP (our SAP Production server) and COVSAPQ (our SAP QA server).

On node COVSAPP there are 3 packages;

paDP (Data Protector Package)
paNFS (NFS Package)
dbciPRD (Control startup and shutdown of SAP, and oracle database, listener etc...)

On node COVSAPQ there is 1 package;

dbciQAS (Control startup and shutdown of SAP, and oracle database, listener etc...)

The packages I want to change the IP address for is dbciQAS and dbciPRD.

They currenlty run on our backbone network 192.168.103.x and we need to move them onto our internal network 10.212.16.x?

Any idea how this is achieved?

Regards,

Chad
11 REPLIES 11
Lee Harris_5
Valued Contributor

Re: Package Address IP Change

Hi Chad,

Go to /etc/cmcluster/. You should have both a dbciQAS.conf and a dbciQAS.cntl. One of these files (I can't remember which off the top of my head) has an IP_ADDRESS variable in it. This is the IP it will bring up on the public interface when the package is started. Change the details here. I'd test it first though before you change anything like this on a production system.

HTH - Lee
RAC_1
Honored Contributor

Re: Package Address IP Change

Basically you will have to do following.
1 stop packages.-cmhaltpkg
No need to halt the cluster.
2 backup package configuration files-just in case required.
3 modify control files to reflect ip changes and subnet changes.
4 modify conf files-if required.
5 distribute the changes to al nodes in cluster-cmapllyconf -p "pkg" -n node1 -n node6 start pcakges and check failover
7 Final start on desired node.
There is no substitute to HARDWORK
Chad Brindley
Regular Advisor

Re: Package Address IP Change

Hi,

In SAM under network interface cards there are entries in here which relate to the addresses of the packages, If I make the change here does it update the cntl files do you know?

If not do I just vi the cntl files and make the changes?

everything I have should point to the package hostname rather than the ip address so I should be ok to just change the address i am guessing.

You are right though I will be doing it on the test system first.

Rgds,

Chad
Lee Harris_5
Valued Contributor

Re: Package Address IP Change

Personally, I'd do it by hand editing the files and restarting the package. I don't trust SAM with such important things.

You're right though, if your apps which connect to this package are referencing a hostname, then all should be well provided your DNS is updated at the sametime as you change the package IP.

You might have a problem if you have Windows clients connecting to the package IP though. I've seen it before where they resolve the package hostname to the new IP when doing an nslookup, but they store a cached query result so keep trying to reach the old address. If you get this kind of behaviour you have to run the ipconfig /flushdns from the command line on your Windows boxes.

Regards - Lee
Chad Brindley
Regular Advisor

Re: Package Address IP Change

Hi guys,

i am back in the office today. BTW - thanks for all your help so far.

If I change the .cntl files on the test server I am guessing for consistency as we have the CCMON tool that I need to copy the file onto the Production server and visa versa when I do the Production server IP change and change the dbciPRD.cntl file? Is this correct?

Both servers need to be consisitent?

Regards,

Chad
Stephen Doud
Honored Contributor

Re: Package Address IP Change

The standard treatment for adoptive nodes is to use the same package control script that the primary node uses. Therefore, changes in a package control script should be propogated to the other nodes.
Chad Brindley
Regular Advisor

Re: Package Address IP Change

Hi,

I am a little confused with the way we want to go with this and the limitations with Serviceguard.

I did a cmviewcl -v (as attached) and as you can see there is a lan0, lan1 and a lan2.

Lan0 is the server address also has the packages virtual ip addresses assigned. Serviceguard sees this as a primary interface.

Lan1 is the heartbeat IP between both nodes and physically there is a crossover between both servers. Serviceguard sees this as a primary interface.

Lan2 is not configured but serviceguard sees this as a standby interface but physically is plugged into the same network as lan0 currently.

To recap we want to configure the package ip addresses onto a different subnet but still using lan0 card on both systems for the actual and virtual ip's.

Lan1 will stay the same so there is no change to the heartbeat configuration.

Lan2 we want to configure on a different subnet to lan0 which will give us a backbone network to take allot of the system network load.

Is this going to be possible with lan2 being seen via serviceguard as a standby card I suppose for network failover and us wanting to have both cards configured to different subnets?

I seem to be unveling things as I go along whilst preparing for the change. I was not part of the serviceguard install here so the above may not be relevant at all but this is how I see it working.

Any help really appreciated.

Regards,

Chad


Stephen Doud
Honored Contributor

Re: Package Address IP Change

In order for a LAN to be a standby LAN, it cannot be configured with an IP/netmask. You indicated that you wanted to distribute the network load from lan0 to lan2 (operating lan2 on a different subnet, if I'm not mistaken). If you assign an IP to lan2, Serviceguard will not be able to use it as a standby LAN.
Furthermore, cmapplyconf will fail on a cluster ASCII file representing lan2 as a standby LAN, though it is not (after assigning an IP to lan2).
If you need to reduce traffic, consider a 1000baseT NIC or get another NIC per server.
Chad Brindley
Regular Advisor

Re: Package Address IP Change

Thanks Stephen, you have confirmed my suspicions.

If I want to do it my way I am guessing we have additional lan cards installed and then configure them on the different subnet.

Then I dont have to mess with serviceguard too much and it will make my serviceguard changes a little easier, my serviceguard changes would then be a simple ip change but to a different subnet ensuring the primary lan and standby lan are on the same network.

Could you confirm I am on the right lines

Thanks Chad

ps - once I know I am on the right track I will assign points and close thread.

Thanks to all so far.
Doug O'Leary
Honored Contributor

Re: Package Address IP Change

Hey;

Let me try to summarize to see if I got what you're asking for:

1. You have a couple of packages that are currently using virtual IPs on lan0. You want to change those virtual IPs to a different subnet and still use lan0.

2. You want to reconfigure lan2 to your backbone network.

From what I've read, you seem to be on the right track; however, it's important to step back occasionally and review:

To satisfy #1, you want to verify the infrastructure is setup *BEFORE* making any cluster changes. Try plumbing the new virtual IP address manually before making any cluster/DNS changes and see if you can get out:

ifconfig lan0:${num} inet ${new_ip} netmask ${netmask}

ping ${any_other_host_on_that_network}
ping ${router_on_that_network}

The reason you want to do that is there's some router magic that has to happen behind the scenes in order to support multiple networks on the same wire. I honestly don't know what that magic is - however, talking to router people, it's not difficult, but it's also not default.

Once you have that working it's time to start making some choices.

1. You don't actually have to bring the package down in order to make this change. It is by far preferable; however, you can leave the IP manually plumbed and update DNS. After the DNS cache timeout, all clients should be using the new IP address. Make sure you update the /etc/hosts as well.

If you decide to go this route, update a copy of the package control script with the new IP addresses. During your maintenance/testing window, the old package control scripts will take down the old IP address. Once it's down, copy the new package control script over the old one (RCS is a wonderful thing) then bring up the pacakge. Troubleshoot as necessary. Once everything's working on the primary node, copy the control script over to the adoptive node and bring the package up over there. Troubleshoot as necessary.

If you decide to shut everything down first, manually deconfigure the IP address you plumbed earlier, shutdown the package, and update the control script. From there test on primary/alternate nodes and troubleshoot as necessary.

2. Decide whether or not you want a standby lan. They're not mandatory. They're pretty cool in that they don't fail over the whole package if a cable or switch port goes out; however, they're not mandatory. The advantage to not having the standby lan is cost - you won't have to buy new cards. I would suggest keeping them; however, that's more of a business decision.

If you decide against the standby lan, you can either update the cluster ascii file manually or regenerate one. Regenerating one is probably safer. Stop all packages, drop the cluster, then update /etc/rc.config.d/netconf as appropriate then either reboot or manually configure lan2.

cmquerycl -C ${config} -v -n ${node1} -n ${node2}

update, check and apply:

cmcheckconf -C ${config}
cmapplyconf -C ${config}

If you want to edit the config, generate the current running config and edit that file:

cmgetconf -c ${cluster} > ${config}
update, check, and apply.

If you decide to add new lan cards, shut everything down, have them installed, then generate a new cluster config file as described above.

Regardless which option you choose, either reconfiguring lan2 or adding new NICs, you'll have to update your cluster config file as MCSG is very sensitive to network configuration.

Last suggestion: Do your updates in two discrete steps - better yet on two separate maintenance windows. You can upate the package IPs first; verify over the course of the week that everything's working, then update the network/cluster configuration.

HTH.

Doug O'Leary

------
Senior UNIX Admin
O'Leary Computers Inc
linkedin: http://www.linkedin.com/dkoleary
Resume: http://www.olearycomputers.com/resume.html
Stephen Doud
Honored Contributor

Re: Package Address IP Change

Serviceguard features a standby LAN function as you know. When a cluster is configured, and when more than one IP-bearing NIC is on the same physical network as the standby LAN, the STBY NIC can swap in for only one of these live NICs at any given time. When the non-functioning IP-bearing NIC returns to working order, the STBY NIC becomes available for swap-in for the other IP-bearing NIC on the same physical network.
If you add another NIC on the same physical network (but different subnet), add it to the cluster ASCII file, halt the cluster and cmapplyconf the file, to take advantage of this feature of Serviceguard.