Multiple IP on two NIC

[raver_1]

Hi all,

I had two L1000 servers with 2 NIC each.
I'm trying to configure 2 heartbeat links on the cards on top of the physical and virtual IP.

i.e.

Host A
------
lan0 (host IP)
137.130.10.11/255.255.255.244
lan0:1 (virtual IP)
137.130.11.10/255.255.255.244
lan0:2 (heartbeat 2)
10.11.1.11/255.255.0.0
lan2 (heartbeat 1)
10.12.1.11/255.255.0.0

Host B
------
lan0 (host IP)
137.130.10.12/255.255.255.244
lan0:2 (heartbeat 2)
10.11.1.12/255.255.0.0
lan2 (heartbeat 1)
10.12.1.12/255.255.0.0

From Host A, I'm able to ping lan0 and lan0:2 IP of Host B EXCEPT lan2.

Same from Host B, cannot ping lan2 IP of Host A.

Pls advice.

Jayce

[Duncan Edmonstone]

Jayce,

Not sure why this sin't working - everything here looks good as far as I can tell, can you post the output of:

netstat -in
netstat -rn

for both machines

This might help us diagnose your problem.

As an aside what are you trying to acheive here? Why add an extra hearbeat IP address on an existing LAN interface, why not just use heartbeart over the existing IP subnet? If this is for MC/ServiceGuard, I'm not convinced what you are trying to do would be supported.

Also may I respctfully suggest to get more responses to this post, you should consider assigning some points to your older posts - if you thought the posts were not helpful, you can assign a point score of zero:

http://forums.itrc.hp.com/cm/TopSolutions/1,,CA748472!1!questions,00.html

HTH

Duncan

[Santosh Nair_1]

Actually there is a problem with the above configuration in that you have two IP addresses on DIFFERENT subnets bound to the same lan card. As far as I know, that's not good.

I understand you're trying to set up two heartbeats, but you should be able to use the primary lan connection, i.e. lan0 as a secondary heartbeat without assigning another IP address to it.

Keep in mind that recommendations are not requirements, so while they recommend that you have two dedicated heartbeats, they don't require it. Check their minimum requirements to see if they require two dedicated heartbeats.

Hope this helps.

-Santosh

[Duncan Edmonstone]

Santosh,

What Jayce is attempting should work - this is from the man page for ifconfig:

Multiple IP addresses assigned to the same interface may be in
different subnets. An example of an interface name without an IP
index number is lan0. An example of an interface name with a IP index
number is lan0:1. Note: specifying lan0:0 is equivalent to lan0.

Cheers

Duncan

[U.SivaKumar_2]

Hi,
You have to specify broadcast addresses for 10.11.0.0 network as 10.11.255.255 and 10.12.0.0 network as 10.12.255.255 and try again.

regards,
U.SivaKumar

[raver_1]

Hi Santosh,

I understand that they is no need to specify an IP for heartbeat.
I just wanted to make sure the two cards can "see" each other between the two servers.

U.SivaKumar,
I tried that before, but to no avail.

Anymore suggestion???

[Sanjay_6]

Hi Jayce,

Try ifalias command to set multiple ip's on the same card.

http://support1.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000053199190

Hope this helps.

Regds

[raver_1]

Hi Sanjay,

Tried your suggestion. But still the same.

Pls note that there's no problem pinging the IPs on lan0 between the two servers.

Only lan2 is giving problems.

Thanks

[Duncan Edmonstone]

Can you confirm that the lan2 NICs can talk to each other at link level:

On node A use 'lanscan' to find out the MAC address of lan2

On Node B, attempt a link level ping to lan2 on node A, out of lan2:

linkloop -i 2 0x

The repeat in the opposite direction

HTH

Duncan

[Tim D Fulford]

You are missing the [1] set of stuff in your netconf file

e.g.

INTERFACE_NAME[1]="lan0"
:
:
INTERFACE_NAME[2]="lan0:2"

You MUST use continously incring numbers to do this...


I have a few other comments
o why are you using lan0 then lan0:2 & missing lano:1? - I suspect you reserved it for the package but I'm pritty sure it will come out on lan0:3...
o Why do you want to put the heart beat over lan0 interface but on another subnet? You have two heart beats

Tim

[Tim D Fulford]

oh I forgot, ifalias is a HP-UX 10.20 thing for HP-UX 11.00 use

# ifconfig lan[:] netmask up

to check use ifconfig lan[:]

The networks & subnets you have chosen are OK BUT there is a problem when subnetting addresses with zeros in it, like 10.0.0.12 with netmask like 255.255.255.0 or 255.255.0.0 you get an error. I think this has been fixed but I do not know the patch that does it. All Ive done in the past is use 10.1.1.12 instead.

Tim

[raver_1]

Duncan,
I'll try your suggestion tommorrow when I get back to work.

Tim,
You're right, I'm reserving lan0:1 for the virtual IP after this heartbeat issue is resolve.

As for why two heartbeats? It is a minimum requirement for Veritas Cluster Server (VCS). Basically it works in a round-robin fashion.

BTW, the two servers were previously configured with MCSG. Heard that MCSG can configure to bring-up a "sleeping" NIC if the default is to fail.

May I know how to check if it is being configured and hpw to disable it?

Thanks.

[Santosh Nair_1]

Duncan,

I know that its possible to assign multiple IP addresses to the same physical LAN card, but I was under the impression that the IP addresses had to be on the same subnet. So I guess I stand corrected.

-Santosh

[Duncan Edmonstone]

Just ensure MC/ServiceGuard is stopped, and that it won't restart on reboot:

on both nodes:

cmhaltcl -f

then

edit /etc/rc.config.d/cmcluster, and change env. variable to 0.

HTH

Duncan

[Aashish Raj]

Try with
ROUTE_COUNT[3]=1 in netconf

AR

[Tim D Fulford]

Jayce

My fingers are made of sausage. I mistyped part of my response

you are missing INTERFACE_NAME[1]=lan... (..[1] set of environment variables) in the netconf file on host A. You have ..[0] ..[1] & ..[2] variables. This will mean when booting it will stop at assigning lan0; as defined by ..[0]; and will not do lan0:2 (..[2]) & lan2 (..[3]). The variable (array) numbers must not have any gaps & start at [0]

That is what I was trying to say which migfht be why you cant ping lan0:2 etc

Tim

[Duncan Edmonstone]

Tim,

I don't think missing out an array index in the way Jayce has will matter - if you take a look at the actual code that reads this file in /sbin/init.d/net you will find that the script actually figures out the highest index value in the netconf file, and then loops thro each index until it reaches the maximum value - any empty indexes just get ignored. This is born out by the fact that Jayce's routing tables on node A do contain entries for lan0:2 and lan2.

That said, it won't hurt to change them as you suggest...

HTH

Duncan

[raver_1]

Hi all,

Hit the following errors issuing linkloop command.

Note that I had try using a cross-cable but are getting the same results.

Pls advice.

----

Host A
========
# linkloop 0x001083FD767E (lan0 of Host B)
Link connectivity to LAN station: 0x001083FD767E
-- OK

# linkloop 0x001083FC2360 (lan2 of Host B)
Link connectivity to LAN station: 0x001083FC2360
error: get_msg2 getmsg failed, errno = 4
-- FAILED
frames sent : 1
frames received correctly : 0
reads that timed out : 1

# linkloop -i 2 0x001083FC2360 (lan2 of Host B)
Link connectivity to LAN station: 0x001083FC2360
error: expected primitive 0x30, got DL_ERROR_ACK
dl_error_primitive = 0x2d
dl_errno = 0x04
dl_unix_errno = 57
error - did not receive data part of message


Host B
========
# linkloop 0x001083FDB6C6 (lan0 of Host A)
Link connectivity to LAN station: 0x001083FDB6C6
-- OK

# linkloop 0x001083FC235B (lan2 of Host A)
Link connectivity to LAN station: 0x001083FC235B
error: get_msg2 getmsg failed, errno = 4
-- FAILED
frames sent : 1
frames received correctly : 0
reads that timed out : 1

# linkloop -i 2 0x001083FC235B (lan2 of Host A)
Link connectivity to LAN station: 0x001083FC235B
error: expected primitive 0x30, got DL_ERROR_ACK
dl_error_primitive = 0x2d
dl_errno = 0x04
dl_unix_errno = 57
error - did not receive data part of message

[U.SivaKumar_2]

Hi,
Please Try this-
#ifconfig lan0:1 up
#ifconfig lan0:2 up
and test the connection again

regards,
U.SivaKumar

[raver_1]

Hi U.SivaKumar,

I'm afriad all interfaces are up.

In fact, lan0 is perfectly alright as from the linkloop result.

Only lan2 is giving problem from both hosts.

Thanks

[T G Manikandan]

Are you sure you have it plugged into the correct card i.e.your lan2

2.check replacing the cable.

3.Also find out whether your card is running at a greater speed than your switch port.

#lanscan -x lan2
to find the speed
Might be the switch port is unable to take that speed.Might be running at 10Mbps.


THanks

[Tim D Fulford]

Duncan & Jayce

The reason I say missing out the ..[1] variables matters is that I had a LAN interface the would not be configured after a boot (but could be done manually). If may well have been a 10.20 system, but some bright spark pointed out that we had commented out an old definition & so the array index was not continuous... I've remembered it ever since (nice and subtle, just the sort of thing that takes weeks to figure out).

In truth I stopped looking for more evedence of a problem after I saw the missing indicee. (how un-scientific)

just my 0.02??? worth

Tim

[Tim D Fulford]

Jayce

So in short LAN2 on BOTH machines cannot see each other (even though they should)

How are they connected? x-over cable, hub, switched hub, switch.....

1 - If the machines are close I would start out with using an x-over cable, (or swaping out the x-over cable for a new one) & seeing if that works. If so you know the interfaces are OK.
2 - Then if you are using a hub I would try another hub, or changing the ports the wires are plugged into. or utilising the LAN0 hub (fidle...)
3 - If you have a complex netrwork arangement (bridge groups etc) I would start looking into that. Be careful if you are using spanning tree algorhythems (did I spell that correctly!!) as you can cause network loops and this could easily bring down your routers/switches
4 - If you are connected to a switched hub or something cleaver, check that there is not a Full/Half duplex mis-match on the interfaces. You can do this is "sam" or "lanadmin"

Basically simplify the problem to find out exactly which component is not doing its job & build from there. I've had to do this kind of thing may times and it can be a bit of a bore but usually sticking to it does the trick

Good luck

Tim

[Duncan Edmonstone]

Tim's right Jayce, the problem is the connection between the lan2 interfaces on both nodes. The first error you get from linkloop (when your not using the -i option) can be discounted as you wouldn't expect lan0 to be able to communicate with lan2 on the other node (when you don't specify an interface with the -i option on linkloop it just uses the first lan interface it finds).

We've narrowed the problem down to two areas now:

1. A link level error, this is either due to a port speed negotiation problem as mentioned by T G Manikandan, or a Spanning tree problem (if there is more than one hub/switch between the two hosts).

2. A physical interconnect problem - whether this is due to a loose connection, dead port, or using a normal cable instead of a cross-over cable I don't know.

The quick way to determine which issue you are having: look at lan2 on the back of both systems - do you have a link light on both? If you do then the problem is a link level error - if you don't the problem is physical.

HTH

Duncan