topic Re: connection problems after Red Hat upgrade in Operating System - Linux

connection problems after Red Hat upgrade

Graham Smith_8 — Wed, 31 Dec 2008 05:49:02 GMT

Hello, one and all.

I have just upgraded a production linux server from Redhat ES v4.5 to V4.7. Now I can no longer connect to the network, getting

[root@dcpbl03 default]# ping 172.29.240.1
PING 172.29.240.1 (172.29.240.1) 56(84) bytes of data.
From 172.29.242.195 icmp_seq=0 Destination Host Unreachable
From 172.29.242.195 icmp_seq=1 Destination Host Unreachable
From 172.29.242.195 icmp_seq=3 Destination Host Unreachable
From 172.29.242.195 icmp_seq=4 Destination Host Unreachable

--- 172.29.240.1 ping statistics ---
7 packets transmitted, 0 received, +4 errors, 100% packet loss, time 6014ms
, pipe 3

I haven't touched any config files, just ran up2date.
The server was perfectly OK before the upgrade. It can see the other member servers within it's local VLAN but cannot pass through the gateway to the rest of the world.

Any magic spells? Getting desperate..

Thanks in advance and I do give out 10 points for replies...

Graham

Re: connection problems after Red Hat upgrade

Steven Schweda — Wed, 31 Dec 2008 06:54:28 GMT

> [...] ping 172.29.240.1

> From 172.29.242.195 [...]

These appear to be on different subnets,
depending on the (unrevealed) subnet mask.

> [...] cannot pass through the gateway [...]

We non-psychics can't divine the netmask or
routing on your system, or the identity of
that gateway to which you refer. Perhaps a
few ifconfig and/or netstat (-r) commands
would reveal some useful information.
Clearly ("Destination Host Unreachable"),
there's no route to that destination.

> Any magic spells?

Find out what it's doing, then fix it?

> Getting desperate..

Get organized instead.

> [...] within it's local VLAN [...]

"its".

Re: connection problems after Red Hat upgrade

Vitaly Karasik_1 — Wed, 31 Dec 2008 10:23:05 GMT

Do you use some 3rd-party NIC driver?
Can you send us "ifconfig" and "netstat -rn" output?

Re: connection problems after Red Hat upgrade

Graham Smith_8 — Thu, 01 Jan 2009 03:55:44 GMT

OK,

[root@dcpbl03 ~]# ifconfig
bond0 Link encap:Ethernet HWaddr 00:15:60:09:AA:90
inet addr:172.29.242.195 Bcast:172.29.242.223 Mask:255.255.255.224
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:534 errors:0 dropped:0 overruns:0 frame:0
TX packets:10962 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:35244 (34.4 KiB) TX bytes:714291 (697.5 KiB)

bond1 Link encap:Ethernet HWaddr 00:15:60:0A:EC:7E
inet addr:192.168.240.113 Bcast:192.168.240.255 Mask:255.255.255.0
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:19106 errors:0 dropped:0 overruns:0 frame:0
TX packets:308 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1296166 (1.2 MiB) TX bytes:19722 (19.2 KiB)

eth0 Link encap:Ethernet HWaddr 00:15:60:0A:EC:7E
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:10721 errors:0 dropped:0 overruns:0 frame:0
TX packets:308 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:759526 (741.7 KiB) TX bytes:19722 (19.2 KiB)
Interrupt:50

eth1 Link encap:Ethernet HWaddr 00:15:60:0A:EC:7E
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:8385 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:536640 (524.0 KiB) TX bytes:0 (0.0 b)
Interrupt:217

eth2 Link encap:Ethernet HWaddr 00:15:60:09:AA:90
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:284 errors:0 dropped:0 overruns:0 frame:0
TX packets:10962 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:18176 (17.7 KiB) TX bytes:714291 (697.5 KiB)
Interrupt:225

eth3 Link encap:Ethernet HWaddr 00:15:60:09:AA:90
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:250 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:17068 (16.6 KiB) TX bytes:0 (0.0 b)
Interrupt:233

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:20222 errors:0 dropped:0 overruns:0 frame:0
TX packets:20222 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:11629098 (11.0 MiB) TX bytes:11629098 (11.0 MiB)

[root@dcp

The gateway IP is virtual IP address within the VLAN.
root@dcpbl03 ~]# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
172.29.242.192 0.0.0.0 255.255.255.224 U 0 0 0 bond0
192.168.240.0 0.0.0.0 255.255.255.0 U 0 0 0 bond1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 bond1
0.0.0.0 172.29.242.222 0.0.0.0 UG 0 0 0 bond0
[root@dcpbl03 ~]#

Thanks for you input,

Graham

Re: connection problems after Red Hat upgrade

Steven Schweda — Thu, 01 Jan 2009 06:55:20 GMT

Ignoring the fact that I've never dealt with
anything named "bondX", ...

> bond0 Link encap:Ethernet HWaddr 00:15:60:09:AA:90
> inet addr:172.29.242.195 Bcast:172.29.242.223 Mask:255.255.255.224

A netmask of 255.255.255.224 takes all but
five bits for the network. This seems odd to
me, but not impossible.

If I read it correctly, this is the default
route:

0.0.0.0 172.29.242.222 0.0.0.0 UG 0 0 0 bond0

and the gateway, 172.29.242.222, is in the
same subnet as 172.29.242.195 (netmask
255.255.255.224), so, if I'm doing my
arithmetic correctly, you ought to be able to
get to the gateway, so I don't see a problem
there.

172.29.240.1 is on a different subnet, so you
would need a route and gateway, but you seem
to have them, so it all looks good to me.

Does the gateway system know how to get back
to this subnet? (Same commands on it?) I'd
expect a different complaint if the problem
were at that end, but either I'm missing
something (too), or else desperation is
starting to look more appropriate.

Re: connection problems after Red Hat upgrade

Vitaly Karasik_1 — Thu, 01 Jan 2009 09:08:48 GMT

2Steven - bondX is a device name used by linux bonding driver.

2Graham - it is possible that RHEL upgrade changed NIC ordering in linux.
I suggest you to re-check that every NIC is connected to the right VLAN and included in the right bond devce.

Re: connection problems after Red Hat upgrade

Vitaly Karasik_1 — Thu, 01 Jan 2009 09:11:36 GMT

in fact, I'll suggest to boot with the previous (RHEL4.5) kernel in order to verify that everything works - in order to exclude all other factors like router's config change and so on.

Re: connection problems after Red Hat upgrade

skt_skt — Fri, 02 Jan 2009 01:22:43 GMT

r u able to ping the GW? check "ethtool ethx" and see if anyting abnormal" .. there are possibilities the switch ports gets error disabled after a reboot. So check at switch level too

Re: connection problems after Red Hat upgrade

Graham Smith_8 — Thu, 08 Jan 2009 03:21:07 GMT

Folks,

Turns out to be a dead NIC in a bond. It seems that it will failover when online but will not restart after reboot.

Thanks for all your advice.

Graham

Re: connection problems after Red Hat upgrade

Graham Smith_8 — Thu, 08 Jan 2009 03:23:19 GMT

See above.