topic Re: MAC Address changes during SG Migration in Operating System - HP-UX

MAC Address changes during SG Migration

Al McKay — Mon, 16 May 2011 18:04:12 GMT

I have an unusual issue. Whenever I migrate a HPVM using SG (Online Migration) The MAC address for the network interface changes. Any ideas/

Re: MAC Address changes during SG Migration

Steven E. Protter — Mon, 16 May 2011 18:54:00 GMT

Shalom,

This is normal.

The floating IP address is a second virtual IP address that runs on the the same hardware (network card) as another IP on the system.

When the package is running on node A it gets a MAC address from node A. If its running on node B it gets the MAC address from node B.

SEP

Re: MAC Address changes during SG Migration

Al McKay — Mon, 16 May 2011 19:05:07 GMT

OK Now here is the real issue, I am forced to manually add the MAC Address to the arp table to communicate with other HPVM's on Blades servers (860i2) on the same enclosure. I can communicate with HPVM's on the same blade as well as other servers (physical and virtual) outside the C7000 enclosure without any problems. Just having issues talking between HPVM's on different blades on the same enclosure. This does not seem correct.

Re: MAC Address changes during SG Migration

Bill Hassell — Mon, 16 May 2011 19:13:06 GMT

It doesn't really matter where the machine is (real, virtual, same enclosure, etc). The MAC address is the lowest level of the communication protocol and the arp table's job is to remember where an IP address belongs. It is a cache and will eventually get refreshed and pickup the new MAC, but for immediate usage, you'll need to use arp -d to clear the cache or arp -s to change the MAC for this host on machines that need to talk to this new MAC address immediately.

Re: MAC Address changes during SG Migration

Al McKay — Mon, 16 May 2011 19:16:39 GMT

I understand that but the underlying issue of no communication between HPVM's on different blades in the same enclosure is an issue. No ping, telnet, ssh, ftp, sftp. No communications whatwoever. That is why we are forced to manually add the MAC to the Arp Cache. I know this is a band-aid but what else can we do?

Re: MAC Address changes during SG Migration

Mel Burslan — Mon, 16 May 2011 19:57:40 GMT

Without knowing your exact network topology and how your routers are configured, it is very hard to make a suggestion, but if you are having disconnects when the MAC address changes, this usually points to something on the arp table refresh not happening as fas as you expect it to be. Your network gear may be configured to issue a pre-emptive arp refresh command, every so many seconds or minutes, which will eventually take care of this problem, but at the same time, issuing a lot of unnecessary arp refreshes will take its toll on the network load. But till you figure out the reason why this is not taking place, you might consider the pre-emptive refresh option.

Re: MAC Address changes during SG Migration

Bill Hassell — Mon, 16 May 2011 21:58:14 GMT

arp -s to all the blades and VMs that need an immediate update should be a trivial network event. The rest of the participants will see the new MAC after arp cache timeouts. I guess I would just add this to the packages.

Re: MAC Address changes during SG Migration

Emil Velez — Tue, 17 May 2011 02:45:06 GMT

Did you specify a mac address with the virtual machine when you created the NIc. If you did not it will not have the same mac address when it starts on another node. You need to have specified a MAC address when you did the hpvmmodify to create the mach card.

Re: MAC Address changes during SG Migration

Bob_Vance — Tue, 17 May 2011 12:31:56 GMT

Something doesn't seem right, here.

What is your network interconnect module?

When the MAC address changes, the network stack should send out a Gratuitous ARP, which lets the other devices on the network see the new MAC/IP combo and update or clear their cache entries.

In normal SG environment, when you move a Service Guard package, the VIP arp cache entry is immediately cleared on other servers on the network (not just other nodes in the cluster).

You say that other VMs in the other local blades have:

>No ping, telnet, ssh, ftp, sftp. No communications whatwoever.

When a 'ping', for example fails, is it the case that the arp cache has NO entry for the IP, or that it still has the OLD entry with OLD MAC address?

When you add the arp entry manually, are you using the "temp" argument?

If not, then the entry will be permanent.
So, perhaps you've built up a bunch of permanent entries that are now not changing normally?

bv

Re: MAC Address changes during SG Migration

Bob_Vance — Tue, 17 May 2011 13:30:13 GMT

The HPVMs cannot communicate, but what about the main O/S hosting the VMs?

bv

Re: MAC Address changes during SG Migration

Bob_Vance — Tue, 17 May 2011 15:57:29 GMT

BTW,
The arp behavior can be seen easily by following steps,
if you happen to have 2 servers to play with :

1) add a TEMP BOGUS arp cache entry for temp, unused IP

Lilly1 ## arp -s 172.16.99.99 0:1:2:3:4:5 temp

2) ping it on another server and dump arp cache

Lilly1 ## ping 172.16.99.99 -n 1 -m 3 | grep loss \
; arp -a | grep 99.99
1 packets transmitted, 0 packets received, 100% packet loss
172.16.99.99 (172.16.99.99) at 0:1:2:3:4:5 ether

Still BAD arp entry exists;
ping failed because arp entry wrong

3) add the temp IP as an alias on a second server

Pine4 ## ifconfig lan0:9 172.16.99.99 netmask 255.255.0.0

4) dump arp cache on first server again (NO ping)

Lilly1 ## arp -a | grep 99.99
<>

arp cache cleared because of GARP issued.

5) ping and check arp cache again

Lilly1 ## ping 172.16.99.99 -n 1 -m 3 | grep loss \
; arp -a | grep 99.99
1 packets transmitted, 1 packets received, 0% packet loss
172.16.99.99 (172.16.99.99) at 0:30:6e:f4:87:f8 ether

GOOD MAC now in cache because of normal arp,
and ping works

6) clear IP; check arp cache

Pine4 ## ifconfig lan0:9 0
Lilly1 ## arp -a | grep 99.99
172.16.99.99 (172.16.99.99) at 0:30:6e:f4:87:f8 ether

Still in cache, manually delete

Lilly1 ## arp -d 172.16.99.99
172.16.99.99 (172.16.99.99) deleted

NOTE: the default arp cache cleanup interval is 5 minutes:

Pine4 ## ndd -get /dev/arp arp_cleanup_interval
300000

bv