DL380 + DL360 G6 not starting Windows after driver update

[FloXXI]

Hi,

last week i did an update of HP related dirver/firmware via Version Control Agent. You can see the version installed before and after in the picture. I did it on a DL360 G6 and a DL380 G6. After the updates both servers started windows but serveral services were not starting and network was not working. I did a second reboot of the DL360 and that server did no longer start windows, not even in safe mode. I then did a fresh installation of a german windows server 2008 R2 on that server, installed windows updates and finally started hp sum from spp dvd 2014.02. Installed all hp drivers, did a reboot and found myself were I started. The server is again not starting windows!!

Does anyone know the issue? Any idea what to do (except of usind older drivers)?

Thank You!

[waaronb]

You should check the other thread in this same section of the HP forum that is probably related to your issue.

NIC firmware update

In essence, there was a firmware update for Broadcom 1GB multifunction NICs (373, 382, etc) that causes some pretty bad problems, like the NIC not working at all.

The thread does have some clues on how to restore functionality by reflashing it, and HP has an official statement:

HP Advisory

[Phil2014]

Hi,

We have exactly the same issues as FloXXI, and I have double checked and we have not applied the NIC firmware updates to any of the affected servers. We have a number of G5, G6 and G7 affected by this, the only thing in commons seems to be they are running Windows Server 2008, as none of our 2012 boxes have been affected so.

The only way to fix these servers seems to fix these servers appears to be remove all the HP NIC drivers from the driver store, reboot and install the latest drivers.. Which seems to work in approx 80% of cases (For us anyway) but then we are getting random behavoirs (black screen of death & Non-P&P Drivers not starting) on the other 20%, so I;m not convinced that uninstall the NIC drivers is enough...

Phil

[justinalot12]

We are having exactly the same issues. Black screen of death and Randon Non Plug and play drivers comming up as disabled.

So far we have identified these as WIndows 2008 R2 only. We have a mixture of OS and strangley enough a 2008 Server was unaffected. 2012 is also unaffected.

[FloXXI]

NIC firmware does not seem to be installed by VCA. It is not in the picture I posted and when I check firmware wit SPP 2014.02 it tells me, that I have these active version on my NC382i DP NICs:

Bootcode 4.6.4

iSCSI 3.0.6

NCSI 1.0.3

CCM 0.0.0

I was wondering if version 0.0.0 is OK?

I tried to remove the latest driver from driver store as described here and use an older one,  but that also did not work.

[FloXXI]

The two affected servers are also only 2008 R2

[Phil2014]

Hi

I have done some more testing, and have a clean built DL380 G7, with Windows 2008 R2 with all microsoft security updates, running:

P410i Firmware 6.40
Bios Version 2013.07.02
HP Homepage 7.3.2.1
Hp Version Control Agent: 7.2.0.0

HP NC382i NICs with the following Firmware:
Bootcode - 5.2.3
iSCSI - 742
NCSI - 2.0.12

If i install the 7.8.50.0 NIC drivers, then they install OK (no reboot is required), and the network works. When the server is rebooted, it never comes back; all I get is the black screen of death. The way we have recovered from this is to restore the windows registry files from the regbackup folder, and then the server boots, but this is far from ideal!!

I have tried the same scenario with the old 7.8.6.0 NIC drivers, and the server works fine both before and after a reboot..

Whilst the 7.8.50 drivers appear to be unified ( also used on the HP 373 NIC) we are experiencing different issues with these, which I believe are also related to this driver. None of our servers have had the NICs flashed with the bad NIC firmware...

Phil

[waaronb]

That's weird. I have a couple DL360 G6 machines and a few DL360/DL380 G5 machines all running Server 2008 R2 with the latest HP software and drivers, including the NIC driver 7.8.50.0.

If the 7.8.6.0 drivers work (don't give any problems booting up), you should probably consider going even further back to version 7.4.14.0.

Myself and others have seen some bad things happen with version 7.8.6.0 where network connectivity wasn't always working quite right.

EDIT:  Oh, and if you're using older or newer versions of the NIC drivers, you might also want to either uninstall the Network Config Utility entirely while trying things out, or at least install a version of NCU that works okay with that version of the NIC driver.  I've often noticed that a newer NCU won't play nice with older NIC drivers, and vice versa.  If you have any NIC teams setup, break them apart and uninstall the NCU just be sure that's not causing any problems.

[FloXXI]

HP-Support finally gave me this information:

Title: **Network Card HP NC382i Broadcom fail while applying HPSUM with SPP2014.02 in Windows 2008 R2
Object Name: mmr_kc-0116789
Document Type: Support Information
Original owner: KCS - ProLiant Servers
Disclosure level: HP and Customer Viewable
Version state: draft
Environment
FACT:
Model : Proliant DL380 G6
OS : Windows 2008 x64 / 2008 R2
NIC : HP NC382i Integrated Dual Port PCI Express Gigabit Server Adapter
SPP : Version 2014.02.0
Questions/Symptoms
SYMPTOM:
SYMPTOM:Network connectivity lost after online update with  (SPP) Version 2014.02.0 in Windows 2008 R2
SYMPTOM:When issuing a IPCONFIG, no network connections or details are returned.
SYMPTOM:Error 1068: The dependency or group failed
SYMPTOM:Error 1058: The service cannot be started, either because it is disabled or because it has no enabled devices associated with it.
Cause
CAUSE:
The server lost network & some services stopped, this happen only with model DL380G6 (with HP NC382i Integrated Dual Port PCI Express Gigabit Server Adapter)
Tested on server ML350G6 and BL460cG6 and it works, so seems we have an issue with the driver v7.8.50.0 and the model NC382i (BCM5709C)
 
After HPSUM, server lost all network cards and services like “workstation”, “server”… -> Unable to start due to dependency service (error 1068)
HPSUM log = CP021772.exe was installed successfully. (same for all other Drivers & Firmware)
Eventvwr = Events ID 700x, NSI proxy service driver failed to start because it’s disabled or no enabled devices.
System info = NSI proxy service driver is disabled
Registry key = HKLM\SYSTEM\CurrentControlSet\Enum\Root\LEGACY_NSIPROXY\0000\Control\  The ActiveService value is missing.
 
Checked and found on HP Advisory same symptoms and issue:
Title: HP ProLiant DL380 G6/G7 Server - Network Connectivity Lost After Online Update with SPP 2014.02 in Windows 2008 R2
http://saw.cce.hp.com/km/saw/view.do?title=HP+ProLiant+DL380+G6%2FG7+Server+-+Network+Connectivity+Lost+After+Online+Update+with+SPP+2014.02+in+Windows+2008+R2&docId=mmr_kc-0116058-8
Answer/Solution
FIX:
1-Uninstall the  (SPP) Version 2014.02.0
2-Install  (SPP) Version 2013.09.0, NC382i Broadcom v7.8.6.0 (10 Sep 2013) , Test Network connection worked fine.
3-Deployment the latest (SPP) Version 2014.02.0 , NC382i Broadcom v7.8.50.0 (18 Feb 2014) than the issue resolved.
© Copyright 2014 Hewlett-Packard Development Company, L.P.

HP seems to know about this issue, but I have to say the given solution did not work for me.