topic Re: which the source address in case of SG? in Operating System - HP-UX

which the source address in case of SG?

Enrico Venturi — Mon, 14 Nov 2005 05:10:54 GMT

Hello colleagues,
we were using Serviceguard.
Each cluster node can be identified through the physical IP and the virtual IP, the last one only when there's a package running there.
My question is: the applications running on the node which IP do they use to present themselves to the outside?
In other words, if a remote node receives an IP connection request from a local application (telnet, rlogin, rcp, .... ) which is the source IP? Is it always the physical address or it can randomly be the virtual one?

thanks
Enrico

Re: which the source address in case of SG?

Devender Khatana — Mon, 14 Nov 2005 05:18:26 GMT

Hi,

Applications always should bind to the virtual IP of the package which is also called the floating IP, as it is accessible through the node where that particular package is currently running. You should also use the same for telnet, rlogin etc. also but you shall not ignore the status of package ( Which nodes hosts the package) while doing so.

FYI
Physical address allthough can be more than one to each server, floating IP also will be different for each package. i.e. 10 package running will have 10 seperate IPs.

HTH,
Devender

HTH,
Devender

Re: which the source address in case of SG?

Enrico Venturi — Mon, 14 Nov 2005 05:27:14 GMT

I didn't ask which IP to use from external nodes to access cluster nodes / packages, this is well known.

I asked to know which is the source IP when an application inside a cluster node accesses the external servers.
This is needed in case of firewalls:
if a want to allow the applications running in a cluster node (telnet, rlogin, rcp, ...) I need to enable their source IP. Am I sure that the source IP doesn't change when there's a MC/SG package running there?

Re: which the source address in case of SG?

melvyn burnard — Mon, 14 Nov 2005 05:46:07 GMT

A package running in a Serviceguard configuration will show the stationary ip address of the node it is running on as it's source address.
If you need to have the package respond with the relocatable or floating adress, you need to compile the application so that it binds to the floating address.
See page 377 in Appendix C of http://docs.hp.com/en/B3936-90079/B3936-90079.pdf for more details

Re: which the source address in case of SG?

Enrico Venturi — Mon, 14 Nov 2005 06:24:26 GMT

I cannot explain myself,
maybe the attached doc. will help you in understanding what I mean

Re: which the source address in case of SG?

melvyn burnard — Mon, 14 Nov 2005 06:37:14 GMT

So are you asking the following:
If an application is started OUTSIDE of SG (i.e. no relocatable ip address involved) what ip address will the application supply as it's source? It will be the stationary ip address of the primary network on the host it is running on.

Re: which the source address in case of SG?

Enrico Venturi — Mon, 14 Nov 2005 07:21:54 GMT

yes, and moreover:
when a package is running on the node and the package IP is set to the LAN 0, does the IP exposed by the applications change (maybe randomly) or not?

Re: which the source address in case of SG?

Dietmar Konermann — Mon, 14 Nov 2005 08:44:15 GMT

Venturi,

no random behaviour is involved here. It's all about routing.

If an application binds to a specific IP address, then the kernel routes outgoing traffic via the corresponding interface. If it is an relocatable IP, then this will be used as source IP address.

If an applikation binds to INADDR_ANY (non-clusteraware applications usually do that), then the kernel's current routing table determines what interface to use, i.e. what source IP address to use.

Melvyn is right... this is usually the stationary address, since it has usually the topmost (network) entry in the routing table.

Here's a small example:

Machine nero has stationary IP 15.140.11.19 and a relocatable one 15.140.8.63:

15.140.11.19[nero]:/ # netstat -in
IPv4:
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
lan0 1500 15.140.8.0 15.140.11.19 79587867 0 73777210 0 0
lan0:1 1500 15.140.8.0 15.140.8.63 12278061 0 11839431 0 0

Routing to network 15.140.8.0 goes through the stationary IP:

15.140.11.19[nero]:/ # netstat -rn
IPv4 Routing tables:
Destination Gateway Flags Refs Interface Pmtu
15.140.11.19 15.140.11.19 UH 0 lan0 4136
15.140.8.63 15.140.8.63 UH 0 lan0:1 4136
15.140.8.0 15.140.11.19 U 3 lan0 1500 <<<<
15.140.8.0 15.140.8.63 U 3 lan0:1 1500

So when I access machine lola on 15.140.8.65 then lola's netstat output shows my stationary UP as source address:

15.140.11.19[nero]:/ # remsh 15.140.8.65 netstat -an | grep '514 .*ESTABLISHED'
tcp 0 0 15.140.8.65.514 15.140.11.19.1005 ESTABLISHED

Now I add an host route through my relocatable address:

15.140.11.19[nero]:/ # route add host 15.140.8.65 15.140.8.63 0
add host 15.140.8.65: gateway 15.140.8.63

Please note that host routes precede network routes... so routing to lola goes through lan0:1 now, using my relocatable address:

15.140.11.19[nero]:/ # netstat -rn
IPv4 Routing tables:
Destination Gateway Flags Refs Interface Pmtu
15.140.11.19 15.140.11.19 UH 0 lan0 4136
15.140.8.63 15.140.8.63 UH 0 lan0:1 4136
15.140.8.65 15.140.8.63 UH 0 lan0:1 0 <<<<
15.140.8.0 15.140.11.19 U 3 lan0 1500
15.140.8.0 15.140.8.63 U 3 lan0:1 1500

This can also be seen in lola's netstat output:

15.140.11.19[nero]:/ # remsh 15.140.8.65 netstat -an | grep '514 .*ESTABLISHED'
tcp 0 0 15.140.8.65.514 15.140.8.63.1005 ESTABLISHED

qed.

Best regards...
Dietmar.

Re: which the source address in case of SG?

Steven E. Protter — Mon, 14 Nov 2005 09:20:05 GMT

SG floating IP addresses are virtual IP addresses. They are subject to their main IP address and looking carefully at the source IP will let you know which node you are working on.

EX:
Mainip 192.168.0.90 lan0

floater 192.168.0.91 lan0:1

Traffic from the floater will actually come out of the main interface.

In theory that is. That is what I believe you'll see if you use ethereal or tcpdump to look at the traffic across the lan.

SEP

Re: which the source address in case of SG?

Dietmar Konermann — Mon, 14 Nov 2005 09:39:15 GMT

Steven,

only SG talks about stationary and relocatable/virtual addresses.

From the networking stack's point of view you have just a bunch of indexed IP addresses assigned to the interface. You could e.g. just bring lan0:4 and lan0:5 up... and leave lan0 (aka lan0:0) down.

As I already said, the used route determines what source address shows up. No matter, if you look with tcpdump or ethereal or just look at netstat.

Best regards...
Dietmar.

Re: which the source address in case of SG?

rick jones — Mon, 14 Nov 2005 20:18:29 GMT

However, applications probably SHOULD NOT rely on a given order of interfaces/IPs in the routing tables given them a specific IP address.

Any application where you want it to use a specific IP address for outbound traffic should make the calls to bind() to bind to a specific local IP address. Service Guard or no.

Re: which the source address in case of SG?

Enrico Venturi — Tue, 15 Nov 2005 04:11:06 GMT

Not all the applications use the bind(); most of them use connection oriented communication, so the bind() isn't needed.

Re: which the source address in case of SG?

Jim Keeble — Tue, 15 Nov 2005 07:59:15 GMT

True, bind isn't required for applications calling connect(), but it is recommended in order to have control over what IP address is used as the source for outbound connection requests. This is documented in the Managing Serviceguard manual under Designing Applications to Run on Multiple Systems ----> Bind to Relocatable Address / Calling bind() before connect(). See:

http://docs.hp.com/en/B3936-90079/apcs03.html#d0e21794

There are some unsupported tricks in route configuration to cope with this. For example, if the applications connects to hosts on a remote subnet, and the router leading to that subnet supports proxy arp, then a route can be added in the user defined commands section of the package script, as :

route add net 0

This will cause connections about from this node to go out with the package IP as the source. the route should be deleted when the package is brought down, also in the user defined commands section of the package script.

One of the problems with this routing trickery is that now ALL connections outbound to that remote subnet use the package IP, whether part of the package or not. Also, there is no way to use this trick for more than one package, unless the remote users for different pacakages are all on different remote subnets.

Best to bind in the application if possible.