DL360 G2 Teaming acting funny

[Christoffer Järnåker]

Hi,
We have about 20 servers in the DL360/380 range but only one with the below problem.
The two NC7780's are installed together with the teaming software.
When NIC #1 is connected alone the team realizes this and makes use of the working link. If I move the connected cable to NIC #2 then the team realizes this and starts to use NIC #2. So far so good.

When I connect both NIC's at the same time that's where things starts to go wrong.
The team sees NIC #2 come online and detects a 'wire fault' on NIC #1 and switch over to NIC #2. Three seconds later it detects a 'wire fault' on NIC #2 and switch back to NIC #1
It will continue to swap between the two NIC's every three seconds until one link is disconnected.

Initially we were running NIC driver 7.103.0.0 and team driver 8.0.0.0 but last night I updated the drivers to NIC driver 8.52.0.0 B and team driver 8.35.0.0 hoping that this would solve the problem. It didn't.
Then came something that made me realize that something very strange was going on with the team. On the first attempt to update the driver for the team the installation hung after deactivating the old team. The system became unresponsive.
After a cold reset and another try to install the driver it installed properly. But after changing the configuration of the team the teaming software hung again while applying the configuration. After 20 minutes of waiting I had to resort to another cold reset as the system once more became unresponsive.

Now we're back at the starting point where only one NIC is connected. What is causing this? The procedure to upgrade the driver is the same as on the other 20 servers and there all the teams work fine.
Could this be a hidden hardware problem? Seems odd as both NIC's work fine as long as they're not connected at the same time.
The system is running W2k SP4. The problem has 'always been there' according to previous technicans.

Best regards
Chris

[Connery]

Hi Chris,
We haven't seen this problem before. Can you export a portion of your event log showing the event entries for this wire fault failovers? I will help diagnose the problem.

Also, reconnect both NICs and then run the following command from a command line:
HPNETSVY >teamoutput.txt

Attach the "teamoutput.txt" file to this post.

Regards,
-sean

[Christoffer Järnåker]

Hi Sean,
I've attached a zip file with two diagnosics dumps, one with one nic connected and one with two connected, a screen dump from the team utility with both NIC's connected showing one of NIC's down, and an extract from the system event log.

I hope you get more out of it then I've managed to :)

Kind regards,
Chris

[Connery]

Thanks for information, Chris.

A couple of more questions:
1. Are the other "working" servers also running W2K or are they running W2k3?

2. Have you swapped NIC cables between this non-working system and a working system to see if the problem follows the cables?

3. You may need to troubleshoot the switch that this server is connected to. I recommend looking at a working server's switch ports and compare them to the non-working server's NIC ports. I would look for configuration differences, cabling differences, switch module hardware differences, etc. Some of these questions will be eliminated by action #2 above.

4. Have you tried different Team types? Does "Network Fault Tolerance Only (NFT)" behave any differently that "Automatic" or "Transmit Load Balancing with Fault Tolerance"?

5. Are you installing the Teaming driver and making configuration changes with the administrator account? Even if you are using your personal account with admin priviledges, try the plain ol administrator account to see if it is able to install the driver more cleanly. Honestly, we don't have a problem as long as the user has admin privs, but it's a good troubleshooting technique. Also, some servers have security settings that "protect" the registry. That can affect the installation/configuration of teaming since some/all registry entries can be "denied".

Sorry for the laudry list. Im trying to be exhaustive to eliminate the back-and-forth due to the nature of this forum.

Post the results and we can go from there. If we don't get is solved here, I'll help you get a case created with the right support group.

Regards,
-sean