Proliant DL380 Strange NIC Problems

[jcayer]

I've got a DL380 G3 exhibiting some very stage network behavior.

First the specs:
Dual proc
4 gig ram
Dual GB nics NC7781
Windows 2000 fully patched
Oracle 10Gr2

Obviously this is a DB server, one of a pair in our production environment. One is a backup of the other. This problem has happened on both servers as we switched at one point to try to solve this problem. We are not able to replicate/cause this problem, so troubleshooting it has been difficult. What we have been able to determine, since it happens on both boxes, it is very unlikely be the physical hardware of the server unless there was a bad run, or we got 2 bad ones. It cannot be the network cable since they both connect and work fine for weeks. We think it is very unlikely to be the switch, but have not ruled that out entirely. It could be an OS problem, Oracle problem, application problem, or HP software problem.

Of the 2 nics, only one is used for Oracle, the other doesn't do much. Basically, the nic stops responding. Pings to it result in no response and the arp table on the box pinging it shows a Mac address of all zeros.

Since the server has dual nics, I am able to RDP in on the unused one. This is were it gets weird. Attempting to disable/enable the broken nic hangs the server. Going to the command prompt and taking a look at the Arp table only shows the NIC I'm connected in on. The second nic isn't listed. One other odd thing is both servers are set to Auto negotiate link connections to the switch. Every port on the switch is set to 100/full, yet the servers connect at half duplex.

Packet captures of the down nic show it trying to broadcast out to the DC and the sister DB server, but no other traffic.

Thanks in advance.
Josh

[Rob Leadbeater]

Hi Josh,

Not sure why this is happening but I would make one comment.

What you are seeing with the full duplex/half duplex is normal behaviour if one end is fixed and the other auto.

If the switch is fixed to 100/Full then you must also set the NICs on the server to be 100/Full.

Hope this helps,

Regards,

Rob