DL380 Gen9 hangs on boot. Loading Drivers

[Dugiefresh]

Well, this seems to be a continued haunting of the HP Proliant DL server line.  There are many posts online that attempt to address this issue and I have spent weeks trying every possible remedy that has been suggested.

It's doesn't appear that anyone really knows what is causing this.  Some says RAID controller driver/firmware updates, onboard NICS and the list goes on. 

The one fact that we can all be sure of is that it is happening, it contiues to happen, and there doesn't seem to be an answer.

My machine has been down for weeks and I have made no progress.  I have started with a bare bones system with one processor, minimal memory, nor RAID controllers, no usb, no network.  I have tried the latest BIOS, older BIOS, cleared this, removed that, rebooted so many times I dare not try to count.

So, my last try is reaching out to you - the community!

The last task for my machine was to update the latest supported SPP.  It seemed to install okay but on the reboot the system hangs at "Starting drivers.  Please wait, this may take a few moments....".  I even tried to resort to the backup ROM. 

I am not worried about any data on the machine.  If I could simply remove all drivers and reset it to the beginning of time and I could at least get to the bios it would be a definite celetration.

If you have any suggestions and/or a reasouce that  could steer me in any direction at this point I would be forever grateful.

Thank for your time and cheers!

Doug

[ngnear]

Hi Doug,

You might want to try offline SPP update to get just the firmware versions updated. Once the update is complete, give the online SPP a shot.

https://support.hpe.com/hpesc/public/docDisplay?docId=a00026884en_us&docLocale=en_US

If the issue continues, Feel free to log a support case with ys for further inspection of the logs.

[support_s]

Hello,

Let us know if you were able to resolve the issue.

If you have no further query, and you are satisfied with the answer then kindly mark the topic as Solved so that it is helpful for all community members.

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[Dugiefresh]

@ngnear This issue has not been resolved and I currently do not believe it can be.
This product is useless and will be taken to the dump. Last HP purchase in
my lifetime and for any others that I can influence.

Cheers.

[ngnear]

You can reach HPE support and log a support case for quick log analysis. 

Let me know if you need any help there.

[Dugiefresh]

Thanks for reaching out.  I have continued to try to get around this but I just can't seem to get my hands around it.

Update -  after disconnecting all means of power to the box (unplug from main power supply, remove motherboard battery and RAID battery backup), I waited overnight and then was able to reboot and get past the hang.  Once into setup, I installed VMWare 7.0U3 on the internal MicroSD card - as a test - since based on other reported incidents, I could rule out the P840 RAID controller.

The install went greaty, they system rebooted and I was able to setup an RAID 60 across 8 1.2Tb drives in bay 2.  I then added a datastore and installed Debian12 and Windows Server 2019 successfully.  Keep in mind I have only rebooted once since regaining access to the setup.

I decided to attach 8 more drives installed in bay 3 to the second port on the P840 card.  When I attempted to reboot the system stopped at the Loading Drivers - may take a while display.

I removed everything from the machine - I mean every single attached micro sd card, raid controller, risers, usb  attached media bay in  bay 1.  Now I can't boot again.  

Any thoughts or suggestions would be great.  I just ordered a new DELL server and hopefully I can move on with this soon.  

Thank so much for your time during this most frustrating time!

[Mr_Techie]

@Dugiefresh

It sounds like you're encountering some complex boot issues, potentially related to hardware detection or a conflict with the RAID controller configuration. Since the system initially booted successfully after your efforts and then failed again after adding more drives, it suggests that the system is encountering a problem during the loading of drivers or possibly a hardware conflict related to the RAID controller or connected storage.

Here are few steps you can give a try: 

Verify the Hardware Configuration [ Like RAID Controller Configuration, Drive Configuration and RAID Arrays]

Clear CMOS and Reset BIOS Again and Boot in Minimal Configuration

Check for RAID Configuration Conflicts and review Boot Order and Drive Detection

Use iLO (if available) to check the system logs for any error messages or events that could provide more insight into the issue and look for any specific errors related to the RAID controller, drives, or hardware initialization that might be halting the boot process.

If possible, test with a different boot device (like a USB drive or a different disk). This can help determine if the problem is related to the specific boot media or the RAID configuration. If the system boots fine with an alternative device, it could indicate an issue with the current RAID configuration or a conflict between the RAID and boot devices.

Check Firmware for RAID Controller and Drives

Since VMware installed successfully previously, consider booting from the VMware installation media again to see if the issue is consistent across OS environments. Similarly, try booting from a Linux-based live USB to see if the system can detect the drives and RAID configuration properly.

If none of these steps resolves the issue, it might be worth considering whether the RAID controller itself is malfunctioning or if there’s an issue with the drives in bay 3. If the issue persists despite all attempts, further hardware diagnostics or replacement of the RAID controller could be the next step.

[Dugiefresh]

Hello and thank your for your time!

I have removed everything possible from the machine and started to try and rebuil it bit by bit and trying to boot after each component addition.  I removed a processor, all but the memory required to boot, a power supply, riser cards, RAID controller, optional media bay, microSD card, all disk drives and power removed from the backplane, removed the array battery, and removed all network and usb connections.

When I fired it up, same thing - Starting drivers - please wait.

I switched power supplies, CPU, and memory dimms.  Same thing.

I have attempted to swap ROM and boot from a previous version - same thing.

I have loaded older versions of BIOS and iLO - same thing.

I think I have tried all but a sledge hammer.  I even tried to change different firmware versions for the RAID controller and network card.  I wanted to remove them but it seems that option is only available with ilo5 (using ilorest).

I also tried changing power profiles when I could - not I can't because I have reset and post won't complete.

Here is a pic as I am reinstalling all the hardware I removed.  This is where it sticks every time.  If there was just a way to totallly clear out the firmware, etc from the bios and start anew at least I could have something to work with.  I did try removing the backup battery and also cleared with flipping switch 6.  Still the same.

I have booted over and over, cold booted, reset, to no avail.

Thanks again for your time!

Doug