NCU 10.20 and 10GbE 6.0.60

[Eric Datwyler]

It's good to see I'm not the only one seeing this issue. I contacted HP support and they stated, "when you update the network drivers, its always advisable to remove the teaming and then update the drivers. Its usual procedure". This can't be correct as those of us who have many servers with multiple VLANs on each server would NEVER update drivers and NCU. I've been working with Proliant servers for quite a few years and never had to do this "usual procedure". Something is wrong in the install and it's advisable not to upgrade until this is corrected. Is HP listening?

[Aaron Blosser]

For what it's worth, I have seen a similar issue before, but it was a ways back. Same deal where updating the NIC drivers was failing, and it turned out to be something like the miniport driver needed updating, and it was just confusing. I'm fuzzy on the details...it may even be here in the forums from 4-5 years ago.

I have 2 identical servers right now, one with the old drivers and one that I udpated to the new stuff the hard way, so maybe I can compare the reg keys and see if anything looks different.

I even tried extracting the files out of the cpXXX.exe files and manually updating the drivers. That actually worked great, but as far as the HP stuff was concerned, it still had the old drivers installed, and even though the HP teaming driver was udpated, the HP teaming utility was still the old one. So I thought if I just ran the update utilities as normal, it might work then, but sure enough, the drivers were no longer able to start after that.

Clearly it's some bug in the driver update program, it's just a hassle.

[Ali]

Well... looks like thats the correct procedure. I had the same answer when I called HP a while ago.
Surprisingly... release date for NCU and NIC driver also matters.

I guess some NIC driver files do not get updated while it is teamed and NCU is running

So, give it a try on one server for testing..
disolve teaming
remove the teaming driver from TCP/IP properties
download the latest NIC driver
Remove the NIC driver completely including .sys file.
update the NIC driver manually and restart the server.
Now check the NIC properties to make sure NCU is completely removed.
Now you may install the latest NCU and check the options (TOE and side scaling) in NIC properties and disable them.
Also : try different combination of NIC driver with latest NCU... may be one release down to latest revision of NIC driver.

About TOE: Toe takes off the load from system processor while doing network transaction or calculation. however, this does not work in some of the windows flavors and microsoft has article for that...I will findout the article for you...

thanks
aftab

[Eric Datwyler]

That is not a very scalable procedure. Anyways, I tried it and have the same results but I did find something interesting. The RSS/TOE options are available and enabled in NCU but are not available after a team is created. Also, these options re-appear after dissolving the team but TOE is still enabled......however, RSS is left disabled. Another interesting point, even though RSS is disabled in NCU it is still enabled in Device Manager. What a mess......so how is this all supposed to be configured in regards to TOE and RSS?

Also, it seems HP released 10.20(B) of the NCU today......I hit refresh and there it was......cp014381. Wonder if a new driver is coming soon.

[Eric Datwyler]

OK, I figured it out.....or at least I got the update to successfully complete without having to recreate the team every time.

I had noticed that TOE was enabled in both device manager and NCU after a fresh install but before a team was ever created. Then, TOE was disabled in device manager and not even available in NCU after a team was created. So, this sounds like NCU disables TOE on the NIC driver when a team is created. However, on a server I haven't touched yet, with a team already working correctly, I noticed TOE enabled in device manager. I installed NCU and it installed correctly, but then I installed the latest NIC driver and that's when I lost connection.

The latest NIC driver release notes state "Some issues that occur during high network utilization with TOE enabled have been corrected." as the listed fix. On another server with NCU 10.10 and the older NIC driver, I went into device manager and disabled TOE on each adapter before upgrading. I then ran through the upgrade and it all worked correctly! Sure beats recreating the NIC team, VLANs, and IP information.

In doing some research during all of this, I read somewhere that TOE is not supported in Windows 2008 with NIC teaming. I have a hunch this "fix" allows the driver in Windows to have TOE correctly disabled by NCU when a team is created.

It would be nice to see some information from HP in regards to this.

[Aaron Blosser]

Well that's cool. I was encouraged by your report of a work around so I tried it and it worked.

I went into dev manager, disabled ToE (ipv4 and ipv6) on both of the network cards. Then I installed the udpated NCU and nic drivers, and it all worked great.

Further, I went back into dev manager and re-enabled ToE on both cards and it's still good. I saw that on a system I updated the hard way, ToE was still showing "enabled" on both cards (the 1Gb Broadcom, not the 10Gb, so maybe that's a difference there).

It could just be that the new driver stores the ToE in a different reg key or something funky. Either way, it's good to have this workaround, and maybe HP can fix this in the next driver release, or do a "B" release that handles that better.

[Eric Datwyler]

Very cool. Well this particular update is for the 10GbE NC532i cards. I haven't tried this on our DL series servers yet with the NC382i cards.

I've been reading quite a bit about disabling TOE and RSS since it seems to create more problems for people than actually having it enabled.

[Ali]

thanks Eric... this would be helpful for others got into same situation.

Good that you read the release notes.. infact I was also reading it since the last post...

thanks again :)

Cheers!
Aftab

[Aaron Blosser]

FYI, I just tried this on another server that wasn't using the teaming (but a similar server had the same issue of the NIC's failing after the driver update).

It works on that configuration as well...disable ToE options in dev manager, update the driver, reboot if it asks, then re-enable ToE options as before.