ProLiant Servers (ML,DL,SL)
cancel
Showing results for 
Search instead for 
Did you mean: 

NCU 10.20 and 10GbE 6.0.60

Eric Datwyler
Frequent Advisor

NCU 10.20 and 10GbE 6.0.60

All,

I just recently installed the HP Network Configuration Utility for Windows Server 2008 R2 10.20 and HP NC-Series Broadcom 10GbE Multifunction Driver for Windows Server 2008 x64 Editions 6.0.60 on a Proliant BL460c server and found that both NICs had a yellow indicator in Device Manager....neither was working. Reboot, remove and re-add, re-install, nothing worked except downgrading to the previous versions (NCU 10.10 and NIC driver 5.2.22). This happened on every BL460c that we have so we stopped our upgrade process. Now, I was able to get the new versions installed if I dissolved the NIC team. Once dissolved, the NICs immediately showed up normal in device manager. Re-created the NIC team and it is working fine.

Has anyone else seen this? Dissolving and recreating the NIC team is not an option when I have 45 of these BL460c servers to upgrade. I upgraded HP NIC drivers and NCU software many a time but never have I seen this. Am I missing something or is there something I need to do before the upgrade? The release notes don't show anything of concern regarding new functionality.
18 REPLIES
Ali
HPE Pro

Re: NCU 10.20 and 10GbE 6.0.60

Hi Eric,

this behavior is observed when you upgrade NCU to NCU 10.20 on windows 2008 and windows 2008 with hyper V server.

you may try this registry settings
Simply navigate to ¦

HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{4D36E972-E325-1
1CE-BFC1-08002BE10318}

Find the key (folder) that represents your NIC Team (for example 0012), which you can find by looking for the â  TeamNameâ  value under each key until you find your team.

Create a new String value (REG_SZ) called:
VLANPromiscuous

Set it to 0 (disabled/default) or 1 (enabled).

check the reference thread:
http://h30499.www3.hp.com/t5/ProLiant-Servers-ML-DL-SL/HP-Network-Config-Util-v10-10-0-0-Crash-after-upgrade-380-G6-on/m-p/4701659#M106544


hope this helps,

Cheers!
Aftab

I work for HP
Looking for a quick resolution to a technical issue for your HP Enterprise products? HP Support Center Knowledge-base – Just a Click Away!
See Self Help Post for more details
Eric Datwyler
Frequent Advisor

Re: NCU 10.20 and 10GbE 6.0.60

I do not have HyperV installed on the servers having this issue. I did add the VLANPromiscuous registry value and it did not have any effect.
Eric Datwyler
Frequent Advisor

Re: NCU 10.20 and 10GbE 6.0.60

Anyone? Anyone? HP? This is concerning since recreating the Team on every one of my servers is not a desirable solution.
Ali
HPE Pro

Re: NCU 10.20 and 10GbE 6.0.60

Hi Eric,

I was away all this while... unfortunetly I have not updated the latest version on my systemm so can't check your issue.

however, I would share few observations

side scaling and TOE feature in multifucation adapter create lot of problem under windows environment. disable these features under NIC property and check..

I will see if I can test this in my lab... I will let you know.

thanks,
Aftab
I work for HP
Looking for a quick resolution to a technical issue for your HP Enterprise products? HP Support Center Knowledge-base – Just a Click Away!
See Self Help Post for more details
Eric Datwyler
Frequent Advisor

Re: NCU 10.20 and 10GbE 6.0.60

Thanks, another note......this problem did not happen on the servers that do not have their NICs teamed. This is only isolated to servers with their NICs teamed.
Ali
HPE Pro

Re: NCU 10.20 and 10GbE 6.0.60

Hi Eric,

Side scaling and TOE features should be disabled from NIC property with in NCU. so try this.

Most probably I will do some testing this weekend and would let you know.

anyone else has experienced the same issue or know this behaviour is welcome to share his/her observations.

thanks,
Aftab
I work for HP
Looking for a quick resolution to a technical issue for your HP Enterprise products? HP Support Center Knowledge-base – Just a Click Away!
See Self Help Post for more details
Eric Datwyler
Frequent Advisor

Re: NCU 10.20 and 10GbE 6.0.60

Is there any documentation that states side scaling and TOE should be disabled any why? If it should be disabled, why isn't it disabled by default?
Eric Datwyler
Frequent Advisor

Re: NCU 10.20 and 10GbE 6.0.60

OK, funny thing. One server does not have the NICs teamed and I can see the TOE and receive-side scaling setting in NCU. However, I do not see these settings when looking at NIC properties in NCU on a server with NICs teamed......only Receive Buffers and Transmit Buffers. So it doesn't even appear that I could disable these if I wanted to.

Re: NCU 10.20 and 10GbE 6.0.60

I've been running into this issue too, and driving me nuts.

I've already applied the registry patch (the VLANPromiscuous key) so that I can edit the properties of a team with 10.10, but that doesn't help with the issue of the drivers failing to start properly when I install the 6.0.22 drivers (from my current 5.2.22.0)

It does this whether my NICs are teamed or not. It doesn't even let me roll back the driver, so I've had to remove the network cards from device manager, reboot, reinstall the NIC drivers, and then they show up. By this point I've lost my existing NIC team settings so I have to re-create that and reconfigure the IP settings.

It's a pain, especially if you have a lot of servers. For now I'm just skipping this on the rest of my systems and hoping the installer will do a better job. Obviously something is getting missed when doing an update, whereas removing the drivers and doing a fresh install seems to work.
Eric Datwyler
Frequent Advisor

Re: NCU 10.20 and 10GbE 6.0.60

It's good to see I'm not the only one seeing this issue. I contacted HP support and they stated, "when you update the network drivers, its always advisable to remove the teaming and then update the drivers. Its usual procedure". This can't be correct as those of us who have many servers with multiple VLANs on each server would NEVER update drivers and NCU. I've been working with Proliant servers for quite a few years and never had to do this "usual procedure". Something is wrong in the install and it's advisable not to upgrade until this is corrected. Is HP listening?

Re: NCU 10.20 and 10GbE 6.0.60

For what it's worth, I have seen a similar issue before, but it was a ways back. Same deal where updating the NIC drivers was failing, and it turned out to be something like the miniport driver needed updating, and it was just confusing. I'm fuzzy on the details...it may even be here in the forums from 4-5 years ago.

I have 2 identical servers right now, one with the old drivers and one that I udpated to the new stuff the hard way, so maybe I can compare the reg keys and see if anything looks different.

I even tried extracting the files out of the cpXXX.exe files and manually updating the drivers. That actually worked great, but as far as the HP stuff was concerned, it still had the old drivers installed, and even though the HP teaming driver was udpated, the HP teaming utility was still the old one. So I thought if I just ran the update utilities as normal, it might work then, but sure enough, the drivers were no longer able to start after that.

Clearly it's some bug in the driver update program, it's just a hassle.
Ali
HPE Pro

Re: NCU 10.20 and 10GbE 6.0.60

Well... looks like thats the correct procedure. I had the same answer when I called HP a while ago.
Surprisingly... release date for NCU and NIC driver also matters.

I guess some NIC driver files do not get updated while it is teamed and NCU is running

So, give it a try on one server for testing..
disolve teaming
remove the teaming driver from TCP/IP properties
download the latest NIC driver
Remove the NIC driver completely including .sys file.
update the NIC driver manually and restart the server.
Now check the NIC properties to make sure NCU is completely removed.
Now you may install the latest NCU and check the options (TOE and side scaling) in NIC properties and disable them.
Also : try different combination of NIC driver with latest NCU... may be one release down to latest revision of NIC driver.

About TOE: Toe takes off the load from system processor while doing network transaction or calculation. however, this does not work in some of the windows flavors and microsoft has article for that...I will findout the article for you...

thanks
aftab

I work for HP
Looking for a quick resolution to a technical issue for your HP Enterprise products? HP Support Center Knowledge-base – Just a Click Away!
See Self Help Post for more details
Eric Datwyler
Frequent Advisor

Re: NCU 10.20 and 10GbE 6.0.60

That is not a very scalable procedure. Anyways, I tried it and have the same results but I did find something interesting. The RSS/TOE options are available and enabled in NCU but are not available after a team is created. Also, these options re-appear after dissolving the team but TOE is still enabled......however, RSS is left disabled. Another interesting point, even though RSS is disabled in NCU it is still enabled in Device Manager. What a mess......so how is this all supposed to be configured in regards to TOE and RSS?

Also, it seems HP released 10.20(B) of the NCU today......I hit refresh and there it was......cp014381. Wonder if a new driver is coming soon.
Eric Datwyler
Frequent Advisor

Re: NCU 10.20 and 10GbE 6.0.60

OK, I figured it out.....or at least I got the update to successfully complete without having to recreate the team every time.

I had noticed that TOE was enabled in both device manager and NCU after a fresh install but before a team was ever created. Then, TOE was disabled in device manager and not even available in NCU after a team was created. So, this sounds like NCU disables TOE on the NIC driver when a team is created. However, on a server I haven't touched yet, with a team already working correctly, I noticed TOE enabled in device manager. I installed NCU and it installed correctly, but then I installed the latest NIC driver and that's when I lost connection.

The latest NIC driver release notes state "Some issues that occur during high network utilization with TOE enabled have been corrected." as the listed fix. On another server with NCU 10.10 and the older NIC driver, I went into device manager and disabled TOE on each adapter before upgrading. I then ran through the upgrade and it all worked correctly! Sure beats recreating the NIC team, VLANs, and IP information.

In doing some research during all of this, I read somewhere that TOE is not supported in Windows 2008 with NIC teaming. I have a hunch this "fix" allows the driver in Windows to have TOE correctly disabled by NCU when a team is created.

It would be nice to see some information from HP in regards to this.

Re: NCU 10.20 and 10GbE 6.0.60

Well that's cool. I was encouraged by your report of a work around so I tried it and it worked.

I went into dev manager, disabled ToE (ipv4 and ipv6) on both of the network cards. Then I installed the udpated NCU and nic drivers, and it all worked great.

Further, I went back into dev manager and re-enabled ToE on both cards and it's still good. I saw that on a system I updated the hard way, ToE was still showing "enabled" on both cards (the 1Gb Broadcom, not the 10Gb, so maybe that's a difference there).

It could just be that the new driver stores the ToE in a different reg key or something funky. Either way, it's good to have this workaround, and maybe HP can fix this in the next driver release, or do a "B" release that handles that better.
Eric Datwyler
Frequent Advisor

Re: NCU 10.20 and 10GbE 6.0.60

Very cool. Well this particular update is for the 10GbE NC532i cards. I haven't tried this on our DL series servers yet with the NC382i cards.

I've been reading quite a bit about disabling TOE and RSS since it seems to create more problems for people than actually having it enabled.
Ali
HPE Pro

Re: NCU 10.20 and 10GbE 6.0.60

thanks Eric... this would be helpful for others got into same situation.

Good that you read the release notes.. infact I was also reading it since the last post...

thanks again :)

Cheers!
Aftab
I work for HP
Looking for a quick resolution to a technical issue for your HP Enterprise products? HP Support Center Knowledge-base – Just a Click Away!
See Self Help Post for more details

Re: NCU 10.20 and 10GbE 6.0.60

FYI, I just tried this on another server that wasn't using the teaming (but a similar server had the same issue of the NIC's failing after the driver update).

It works on that configuration as well...disable ToE options in dev manager, update the driver, reboot if it asks, then re-enable ToE options as before.