Proliant DL380 Gen 9 - Starting Drivers Please Wait

[Nightfox_3415]

Hi,
I am having issues with my DL380 Gen 9.

Last night during maintenance we updated the BIOS to version P89 v2.96 with iLO 2.82
We also updated Inteligent provisioning to 2.87.5
Once booted into inteligent provisioning we ran the firmware update to the onboard NIC.
Since the server rebooted, it will not POST.
It sits on "Starting drivers. Please wait, this may take a few moments." With POST code 0711 visible on the bottom of the virtual console window.

I have tried removing the storage controllers, but this hasn't made a difference.
I also tried booting from the backup ROM, v2.90 but this didn't make a difference either.
Their are no other PCI cards or other components installed.

Does anyone have any ideas?

Thank you,

[Suman_1978]

Hi,

There could be issues ranging from HW to FW or even partition.
Here is a document and POST which discuss this issue.

Thank You!
I work with HPE but opinions expressed here are mine.
HPE Tech Tips videos on How To and Troubleshooting topics

[Nightfox_3415]

Hi, Thank you for the reply.

I previously read the linked post but none of the steps worked.

I do not believe the issue is with the storage controller, as even with both storage controllers unplugged I am still having this problem.

Do you have any other suggestions?

[Suman_1978]

Hi,

I can think of only one solution here, Have you tried clearing NVRAM?
S6 switch, please see this document.

Thank You!
I work with HPE but opinions expressed here are mine.
HPE Tech Tips videos on How To and Troubleshooting topics

[Moderator edit: Updated the working link. You nay also refer to https://support.hpe.com/]

[Nightfox_3415]

I have already tried this also (apolgises I forgot to mention this in my first post.)

After resetting the NVRAM, I got the message on screen that it had been cleared at users request.

Anything else worth trying or any further details that would be helpful for me to share?

Thanks,

[Suman_1978]

Hi,

Any USB stick or USB drives connected externally or internally onto the system board?
Can you try with minimum RAM installed?

Link 1 and Link 2 from outside HPE.

If still unable to solve the problem, I would suggest to contact HPE Support to log a ticket.

Thank You!
I work with HPE but opinions expressed here are mine.
HPE Tech Tips videos on How To and Troubleshooting topics

[Nightfox_3415]

Thanks for continuing to provide possible solutions!

Unfortnately, I have already read both of those articles and no luck.

There is no USB drives connected internally or externally or and SD cards.

I have raised a case with HPE support, will reply back here when I get more details.

Thank you!

[PZ4]

As a last try:

It can be one of the harddisks thats hanging up the startup proces. Try one-by-one to take a disk out. When starting up, direct power off server, because otherwise rebuilding will start .....

It can also be the backplane of the diskbay. So, disconnect the power from the backplane (by disconnect the little black cable just beneath the fan assembly which is connecting the backplane to the motherboard. .

Be aware of data lost ....

[martijnp]

I have the exact same problem like Nightfox_3425.
Same firmware and ROM version before problem occured.

I have tried to update firmware using Intelligent Provisioning. I remember noticing an odd message that Intelligent Provisioning wanted to overwrite version 20.xxx with 2.xxx for the onboard 4x1gbe NIC (LOM). I did go ahead (stupid me), afterwards I was unable to download any online content in Intelligent Provisioning.
After a rebooting I got stuck ar the 'starting drivers' message endlessly.

I've tried every possible solution in this post and other, without any success.

Foud this article:
link here 

Seems like it is the firmware of the LOM that is killing the POST process...

Is there any way to upload the right firmware back into the server?
I have tried uploading the bin in iLO, but no success.
Or is there a way to disable the LOM?

Thanks in advance!

[kevimax]

I have exactly the same problem as you!

It seems that we need to revert to firmware 2.27.6-1.1, but since POST does not complete, it is impossible to change the firmware.

Do you have any news from your side?

translated from French with Google

[kevimax]

@nightfox_3425,
Your problem is more than a month old. Was a solution to this found by the HP teams?
FYI: Upgrading the BIOS to 3.30 via iLo did not resolve the problem.
thank you for your feedback

[kevimax]

I opened a folder in support. I am asked to pay for the repair

However, it seems to me that HP has caused damage to our equipment..

martijnp, do you have any possible solutions?

[YKC]

I have the same problem today , even I removed all harddisk and P440AR

[YKC]

Finally , I can pass the starting driver.

when it holding on the POST screen and then I changed the power settings in iLo from Dynamic Power Savings Mode to Static High Performance Mode.

it suddenly passed the starting drive and goto Power and Thermal Calibration , and resumed normal

[kevimax]

Hi !

Thanks for the tip!
I managed to do it once on 1 of my 3 down servers, I was not able to reproduce it again, nor on the others.
Chance ?

Can you say more about the conditions when you succeeded?

[YKC]

ok, no problem

 I tried many method,

1). power off the server , enter iLo to downgrade FW to 2.4

2). power on the server , it will hold on starting driver....

3). press cold boot in iLo , then change One-Time Boot Status to intelligent provisioning

4). power on the server , when performing  early system initialization , press reset in iLo

5). server reboot and POST , when hold on starting driver.... change the power settings to Static High Performance Mode

And then it suddenly pass the "starting driver...." and continue

hope these can help you!

[dom-b]

Had the same issue on my DL380 G9, very frustrating to say the least and poor HP has not taken down the current LOM firmware from online firmware updates.

After MANY hours of messing around and taking inspiration from this forum, here was my approach, which I have now repeated twice:

1). power off the server , I left firmware on 3.08.  Remove the P440AR.  Flip maintenance switch 7 to force a legacy BIOS boot.

2). power on the server , using the ILO reset the server during "Early Processor Init"

3). server reboot and POST , when hold on starting driver.... change the power settings to Static High Performance Mode

And then it suddenly pass the "starting driver...." and continue

4) boot the SPP SPP2021050.2021_0504.129 via the ILO and downgrade the LOM firmware

Should now be all resolved and can you flip back to UEFI mode and install the P440AR.

[kevimax]

Thanks to you dom-b and YKC !

For my part, I had luck with one of my 3 servers, I was able to boot once to downgrade the LOM firmware, but I cannot reproduce the procedure with the other two.

It's strange, they have scrupulously the same configuration, same bios, same ILO, etc.....

I am sad to see that HP has still not removed the defective packet despite my multiple reminders to alert them. They always offer to give me a quote to resolve the problem. It seems to me that they don't realize that this problem is much broader... unless they want to kill hundreds, thousands of "gen9s"...

I'm going to continue testing on my own, your feedback is probably a good basis, I'm still listening to other suggestions too

Bye

translated by google from french

[kevimax]

Hello everyone !

Have any of you been able to discover new avenues for this problem?

THANKS !

[kevimax]

Hello dom-b et YKC,

I realize today that the only difference that remains between the server where I successfully manipulated it and the 2 others is that the 2 others have an ilo license.
Do your servers have ilo licenses?

THANKS

[YKC]

Yes , my machine has iLo license

[kevimax]

Please lokk here : https://community.hpe.com/t5/proliant-servers-ml-dl-sl/starting-drivers-please-wait-this-may-take-hours/m-p/7201562#M184295

[Andrzej_Ang]

I've been struggling with this problem for days - ripped off everything I could from the server (no RAID, no PCI cards, only 1 memory module), tried to do "Reset" and "Cold boot" as it was described earlier - nothing helped.

Changing SW2 switch positions 6 to ON (clear nvram),  7 (legacy boot) to ON - no change.

Came up with the idea that maybe there is a keystroke that can cancel this "Please wait", so I started to press F9 many times even before it appeared on the screen - believe it or not, but the screen passed this horrible message, I loaded ISO image with Service Pack for ProLiant, downgraded NIC firmware from 20.25.42 to 20.22.41, and since then the problem have not happened again! Did many restarts, power offs, cold boots - sever works!

So try the F9 trick, good luck!

Andrzej

[raddie1]

It is just a matter of luck. You need to keep resetting the server until it goes past the hang-up. It may take hours. All the tricks did not work for me. Once part the "Staring drivers" downgrade the Broadcom NIC firmweare. Nothing more. 

[bob20023]

Hi guys . Just experienced similar issue . Had a chat with HP support tried all kinds of fixes but in the end HP support said it seems to be related to system board and it's recommend to swap it. Anyone else had anymore luck with getting passed starting drivers screen . Thanks