Resolving asymmetric routing with ip_strong_es_model?

[Olivier Masse]

Hi

I'm no networking guru so I have a question for those of you who are. :)

My system is multi-homed on two different routable LANs, for reasons I won't discuss here to spare you of a long post. However, this created an asymmetric routing problem which I can't easily get rid of.

Here is an example of my setup:
- lan0 is on 1.2.3.4 and lan0 is my default route
- lan1 is on 5.6.7.8

Now I try to ping or do a simple ssh to 5.6.7.8, from a totally different source subnet, let's say 9.10.11.12. Any IP traffic that comes in on either lan0 or lan1 always gets back from lan0. This is logical since the default route to 9.10.11.12 will be on lan0.

This problem went unnoticed until I saw that doing asymmetric routing mixes up IP Filter (with reason), which will hang after a while. Doing this kind of routing is not advisable anyway.

I looked in the forums and saw a few posts (mostly from Rick Jones) that mention that setting ip_strong_es_model to 1 might resolve this, but it doesn't work as expected. When it is set to 1, I can no longer reach 5.6.7.8 at all, either with an ICMP or TCP.

According to the documentation on this parameter, setting it to 1 results in the following behaviour:

(A) A host MUST silently discard an incoming datagram whose
destination address does not correspond to the physical
interface through which it is received.

(B) A host MUST restrict itself to sending (non-source-
routed) IP datagrams only through the physical
interface that corresponds to the IP source address of
the datagrams.

It looks as if I'm hitting case #A but it makes no sense; why would packets with a destination of 5.6.7.8 try to come in through lan0?

Any ideas? Of course if there are other ways around this to setting ip_strong_es_model, I'd like to hear from you.

Of course I can get rid of one of the two lans on the server but I'd rather keep this as a last resort.

Thanks

[Olivier Masse]

Just to add some info, I did try setting ip_strong_es_model to 2, and it doesn't have any effect. If I simply ssh to the box to 5.6.7.8, I get a connection, but the packets are still returned through lan0.

[Olivier Masse]

I found some more info in doc #NETUXKBRC00005778. It mentions to set up TWO default routes. As soon as I enable a second default route on lan1, it works as expected.

Now the fact that it does work doesn't make me more comfortable... I have no clue how many HP-UX customers actually use this. Is anyone doing this, in a similar manner as mine?

Thanks

[Laurent Menase · Accepted solution]

If you are up to date with patch it will work.

Just disable the default gateway probing, else if one of the gateway doesn't answer you may loose a gw and come back to the previous behavior.

Just an other thing. if you have a client application, if you want that it use one or the other address you will need to bind to the corresponding local address.

[Olivier Masse]

Anybody has comments on this routing issue, please share with me in this thread.

Not being too much network-savvy I dug deeper into this since yesterday, here are my notes.

1. No matter if I have an application that's bound to the floating IP or not, my understanding is that if I leave the default routing algorithm in place (i.e. ip_strong_es_model to 0), HP-UX will always use the default route to send outbound packets, causing asymmetric routing. If I'm right, I've probably been living with this for years, but I've never noticed up until now.

2. It seems that when you have an indexed interface for a floating IP such as lan1:1, setting ip_strong_es_model to 1 is not a good idea... tcpdump shows packets coming in on the interface with the floating IP as the destination but from what I can see, the packets seem to be discarded at a lower level since there's no answer. I *think* that's because the IP stack feels that lan1 and lan1:1 cards are different from an inbound perspective, so it discards the packets as per RFC1122-3.3.4.2.

3. So... what can be done? Well, setting ip_strong_es_model to 2 makes it more relaxed and it does seem to make routing work as expected with floating IPs.

4. Why care and not let it be? The problem I'm having is that we're running IP Filter, and doing asymmetric routing seems to be causing IP Filter to hang. If I log a call about this, chances are good that support will tell me "don't do asymmetric routing", and they would be right.

Thanks

[Laurent Menase]

hi Olivier

1. No matter if I have an application that's bound to the floating IP or not, my understanding is that if I leave the default routing algorithm in place (i.e. ip_strong_es_model to 0), HP-UX will always use the default route to send outbound packets, causing asymmetric routing. If I'm right, I've probably been living with this for years, but I've never noticed up until now.

yes it will always use the default gw, but
if your multiple local address or vip are in the same subnet;
it may select any of the local address, depending of any inbound connection which could have been received and answered


2. It seems that when you have an indexed interface for a floating IP such as lan1:1, setting ip_strong_es_model to 1 is not a good idea... tcpdump shows packets coming in on the interface with the floating IP as the destination but from what I can see, the packets seem to be discarded at a lower level since there's no answer. I *think* that's because the IP stack feels that lan1 and lan1:1 cards are different from an inbound perspective, so it discards the packets as per RFC1122-3.3.4.2.

i think the behavior there depend on your patch level; but i am not 100% sure of that.

3. So... what can be done? Well, setting ip_strong_es_model to 2 makes it more relaxed and it does seem to make routing work as expected with floating IPs.

yes it is done for that;
you have also the default gw per interface which may help if your multi homes system has addresses in the same subnet.




4. Why care and not let it be? The problem I'm having is that we're running IP Filter, and doing asymmetric routing seems to be causing IP Filter to hang. If I log a call about this, chances are good that support will tell me "don't do asymmetric routing", and they would be right.

i think you can try to contact support; it is here for that too; describe well your network topology; ipfilter config and give a nettl trace at nic , ip and tcp layer or all layer showing what's happen;
Say it doesn't work, and not how to configure it;
the answer may be it is not supported, but it worth trying it;
Best regards,

[Olivier Masse]

I'm closing this thread. Thanks to Laurent for the answers.

For what it's worth, I did a great deal of reading on the subject and wrote a paper describing clearly what ip_strong_es_model does. You can get it here:
http://www.mayoxide.com/presentations/Understanding_hpux_routing.pdf