DL380 G7 Problem

[Tony Barrett_2]

We have a Dl380 G7 in a remote office that's been running ESXi5 for a couple of years. This site recently experienced a power cut, and while this has happened before and the server recovered ok, this time it hasn't.

The symptoms are;

The server will power on, and the fans will go to full speed. No video or POST is observed
After 5 mins of roaring fans, the HP ProLiant logo is displayed on the screen, and POST seems to start
Memory shown is correct. NIC BIOS loads. RAID controller seems to initialize, but then nothing. The server just goes unresponsive.
A message is visible saying that iLO3 is not available or responding. Everying else looks ok.

What I've tried;

Disconnecting RAID controller SAS ports on mobo and pulling the cache card
Pulling the backplane I/O riser completely out of the system
Reseating all memory modules
Setting DIP 6 to on (iLO security), then powering on
Setting DIP 0 to on (clear NVRAM)
Pulling the coin cell battery

None of this seems to help though.

What I've also identified is, with everything disconnected, I can get into the BIOS screen (F9 at boot). When I do this, the fans revert to normal speed, the BIOS screen appears and I can navigate it ok (I restored default settings here to). Once I did this, the server appeared to reboot (fans still at normal speed), but no video and no POST. Left the server overnight - no change. If I pull the power cords completely, then reconnect, server starts, fans go to full speed, and the process starts again.

I have a DVD with the ProLiant Service Pack 2015.10, planning to reflash the iLO. The server doesn't have a DVD drive, so I attached a USB one, and from a cold boot, the server started booting from the DVD. The DVD splash screen appeared, and the message 'copying image to local filesystem' was shown. The progress bar reached about 25% before the screen flashed and the server rebooted again (fans are at normal speed at this stage), then nothing. I left the server for an hour or so, but still nothing. I started to press the power button for 5 seconds to reset, but after around 3-4 seconds, suddenly the monitor showed an image that I can only assume is the Service Pack (it's a blue/black screen with what looks like a wave). Nothing else appeared. I could move the mouse pointer, but nothing further was displayed.

So, I think there must be a way to recover this server. All the hardware looks ok, but I think something has got corrupted following the power cut (iLO probably). I'm way more familiar with Dell servers, so I'm not 100% how best to fix this.

Any advice?

[PDP-Fan]

I had a couple of servers with similar problems. When the iLO firmware is corrupted (as it seems to be in your case) then there is not much you can do.

I change the system board if stuff like that happens. Sorry but that's probably the only solution.
System boards can be bought from hardware brokers for a relatively small price. There is one at Ebay for $370.00 (DL380 G7)

[Tony Barrett_2]

Thanks for the reply. I'm really surprised there's no easy way of recovering from this. There doesn't seem to be any 'deep' recovery option for the DL380's either, which isn't a good sign.

I'll carry on with my investigations, unless someone else has any bright ideas!

[Cederberg]

Hi.

If it is indeed the ILO that is causing this issue you might try reflashing the ilo firmware according to the suggestions in the ilo3 user guide on page 228-229.

http://h20628.www2.hp.com/km-ext/kmcsdirect/emr_na-c02774507-6.pdf

Regards.
Cederberg