Default Routing not working after Patch on HP-UX 11.31

[Tuerk_1]

After patching HP-UX11.31 for Oracle 11.1.0.7 we encounter problems with default routing.
We cannot ping server using the default gateway, which was working before the patches.

local ip address of server:

ifconfig lan3
lan3: flags=1843
inet 10.161.84.12 netmask ffffff00 broadcast 10.161.84.255

When we now traceroute the former available server we receive the following output:

traceroute to by-sap-e64 (10.190.103.197), 30 hops max, 40 byte packets
1 by43a6 (10.161.84.12) 0.042 ms !N 0.034 ms !N 0.013 ms !N

Former output was:

traceroute to by-sap-e64 (10.190.103.197), 30 hops max, 40 byte packets
1 LVRRSH1 (10.161.84.1) 0.938 ms 0.933 ms 0.491 ms
2 10.161.135.253 (10.161.135.253) 2.010 ms 2.475 ms 2.480 ms
3 10.161.253.125 (10.161.253.125) 2.487 ms 1.502 ms 1.983 ms
4 10.167.1.18 (10.167.1.18) 2.982 ms 2.512 ms 2.503 ms
5 10.167.1.18 (10.167.1.18) 2.959 ms 2.496 ms 3.025 ms
6 172.29.0.5 (172.29.0.5) 2.451 ms 2.555 ms 2.435 ms
7 10.190.254.233 (10.190.254.233) 2.985 ms 2.508 ms 2.484 ms
8 10.190.254.225 (10.190.254.225) 2.991 ms 2.738 ms 2.482 ms
9 10.190.252.253 (10.190.252.253) 2.989 ms 2.479 ms 2.986 ms
10 by-sap-e64 (10.190.103.197) 2.992 ms 5.248 ms 2.963 ms

Routing information: netstat -rnv
Routing tables
Dest/Netmask Gateway Flags Refs Interface Pmtu
127.0.0.1/255.255.255.255 127.0.0.1 UH 0 lo0 32808
10.161.84.12/255.255.255.255 10.161.84.12 UH 0 lan3 32808
10.161.84.0/255.255.255.0 10.161.84.12 U 2 lan3 1500
127.0.0.0/255.0.0.0 127.0.0.1 U 0 lo0 32808
default/0.0.0.0 10.161.84.1 UG 0 lan3 1500


ping to 10.161.84.11 is working fine

Any suggestions?

[Steven E. Protter]

Shalom,

My first instinct would be to back out hte patch with swremove.

IF that won't work, look for issues in /etc/rc.config.d/netconf

Chec:
/etc/hosts,/etc/nsswitch.conf networking

SEP

[rick jones]

the !Ns and their RTT's suggest the stack itself is tossing-back ICMP network unreachable messages, but it might be best to get a packet trace just to be certain

Is the statement "ping to 10.161.84.11" a typo and you really meant to list the IP address of the default gateway - 10.161.84.1?

Does the MAC address associated with the default gateway's IP address in the arp tables on the system match that of the default gateway?

Out of idle curiousity, what is the "metric" for the default route in /etc/rc.config.d/netconf ?

[Tuerk_1]

We can ping servers in the subnet 10.161.84.x/255.255.255.0, but we cannot reach servers over the default gateway.
When we replace this server with a pc using the same network address and routing entry, we are able to reach servers over the default route.

[Tuerk_1]

@Steven
We reduced the /etc/rc.config.d/netconf to one entry for the nic using 10.161.84.12 and no routing.

INTERFACE_NAME[2]="lan3"
IP_ADDRESS[2]="10.161.84.12"
SUBNET_MASK[2]="255.255.255.0"
BROADCAST_ADDRESS[2]="10.161.84.255"
INTERFACE_STATE[2]="up"
DHCP_ENABLE[2]=0
INTERFACE_MODULES[2]=""

After reboot we set the routing manually by 'route add default 10.161.84.1 1'.

[Tuerk_1]

@Rick
Ping to 10.161.84.11 is possible. Should show that subnet 10.161.84.x is reachable.

Server is running in a two node cluster, and other server of the cluster is still working fine.

[Horia Chirculescu]

How about:

ping to 10.161.84.1

Is it working fine also?

[Tuerk_1]

The default gateway 10.161.84.1 does not reply to a ping due to security reasons.

[Horia Chirculescu]

How about

arp 10.161.84.1

What is the output of this command?

[Tuerk_1]

arp LVRRSH1 (10.161.84.1 - default gateway)

on running server - lvrhp1:

LVRRSH1 (10.161.84.1) at 0:d0:c9:ad:48:fd ether

on failed server - lvrhp2:

LVRRSH1 (10.161.84.1) at 0:d0:c9:ad:48:fd ether

The failed server was running before the patches very well - no problems.
No changes made manually after patch!

[Tuerk_1]

From the output of the 'traceroute' it seems to me as if the server is not using its routing information when trying to reach servers not in the local subnet!
The first hob is not the default gateway as it is on the running server lvrhp1.

Hope this might be a hint?!

[Horia Chirculescu]

Can you access any service running on the gateway (http/ftp/ssh/whatever...) from your local server (84.12)?

[Tuerk_1]

No access to the default gateway.
I asked the network admin to grant access to the gateway, but this will take a while!

[Horia Chirculescu]

Can you post the result of:

cat /etc/rc.config.d/netconf

[Horia Chirculescu]

You could also manually add the desired network (10.190.103.0) through the gateway and see what it is happening.

[Tuerk_1]

cat netconf - look above

Trying to remove patch now and test default gateway.

After this I will try to route the desired network directly.

[Tuerk_1]

Removed the patches and everything is fine!!!

No network patches were installed, only PHKL kernel patches required for Oracle 11.1.0.7

[Tuerk_1]

The patches were NOT the problem!!!
A firewall was set up an blocked ICMP!!!
After enabling ICMP an the gateway accessing the routing was possible again.

Thanks a lot to your support!

[Tuerk_1]

ICMP was enabled on the gateway!

[Horia Chirculescu]

Now you are talking!

Great to hear that.