DL380p Gen8 - Orange power but and fast flashing red light, doesnt boot

[Everest89]

Hi everyone,
Hope everything is ok with all. First i would like to say sorry for my english, i'm from Portugal and i'm having the a issue wth my server.

My server has beeing working for over 8 months, but last saturday (24/04/2021) i had to moove the server from place so i shutdown the server from VMware and after it shutdown i turned the UPS off and mooved it.

Since then the server dosent turn on.
It does but shuts down emidietly as soon as the lights of the disks and disks start to spin.
I get the a fast flashing red light on the health light next to the power button.
I can enter ILO witch gives me the indication of system health critical but when i press there ot goes to system information where there is a red symbol on BIOS/Hardware Health as Failed.

What could it be?
I leave some photos.
One more thing, i have removed every componente i have on the server e tried putting one by one but it does the same and also tried to use the swap ROM but since it dosent boot, it dosent change the backup ROM.

(ram, cpu, sata controler, disks, nic's, psu, etc..)
I also tried with only one CPU and RAM and also bought new power supplys but does the same.

On the Insight Lights Panel i have no lights on also.
What is more wird is that on my ILO logs i dont have any thing that could indicate what it is.
One more thing i already flashed yesterday with the help of hunter86 to the latest ILO version 2.77.

Thank you,
Carlos

[hunter86_bg]

On iLO overview there is a "Critical" status. When you click on it, it should point you to the IML log line that is causing that "Critical" status.

About the '[Not SET]' , this is due to iLO clock not being synchronized to a NTP source.

[Everest89]

When i click on it, it goes to page  Subsystems and Devices (same one on picture 4 on my post), and wheni press on Bios/Hardware health it goes to the IML Log on picture 2.

[hunter86_bg]

As IML is reusing the log entries (just updates the timestamps for the error message), the error could be somewhere bellow the point in the screenshot.

Either check all messages or wipe the IML log and power on again to force a new IML log entry.

[Everest89]

Ok, so i tried what you ask, wiped the IML Log and tried to turn on the server again and still it didnt register anything on IML log.
ILO Log only registeres when i enter or exit the ILO console or when it restores the power.

What a bummer...

[hunter86_bg]

And I guess the "Critical" status still remains ?

[hunter86_bg]

I guess you can try:

- remove and reinsert the power cables
- remove and replug the power supply units
- Try to power up with minimum hardware setup.

I've never done it before , so I can't guide you with exact details.

[Everest89]

Yes it still remains The Critical status.

I have tried what you sugested and even have dissasemble everything and testet only with one cpu and one ram.

[Everest89]

Does the Active Health System Log help figurer it out what it is?
https://mab.to/fraJycxS7

Thank you once more.
Carlos

[Everest89]

Hello again,

Just spent about some time hover the server again and finally i got a error on IML Log.
Here is the error.

"ID","Severity","Class","Last Update","Initial Update","Count","Description",
"605","Critical","Power","01/01/1970 13:05","01/01/1970 13:05","2","System Power Fault Detected (XR: 24 00 MID: FF 0D FC FE C0 FF FF 2F 2F 0C 0C 00 9E 01 00 01 03 47 00 00 00 00 00 00 00 00 00 00 00 00 00 00)",
"604","Informational","Maintenance","[NOT SET] ","[NOT SET] ","1","IML Cleared (iLO 4 user:administrator)",

Thank you once more for all your help.
Carlos

[hunter86_bg]

In one similar error it was advised to remove the BIOS battery and short the 2 pins.
As per the output the iLO lost it's time, so most probably the battery died.

The following is a cheap fix (if indeed the battery is the problem): fully unplug the server PSUs (drain any leftover power by pressing the Power button) and replace the battery.

I used regular desktop battery (CR2032) for my ML350p Gen8 , so it should work on your server.

I hope someone can decode the error to reveal the details.

[Everest89]

Hi again,
Well, i removed the bios battery as you told and left the server off all night from power supply, just whent and tried it and it does the same.
I also tried switching the button 6 to clear NVRAM but no luck.

Running out os ideas..... 
Lets hope someone can read the Active Health Log.
Thank you.

[Everest89]

Since MyAirBridge the link expires after3 days, here is a new link with my Active Health Log.

https://we.tl/t-dsa5ozE4C4
This one stays online for one week.

[Jan Soska]

Hello Everest89

I am afraid that your motherboard simply died. Symptoms are showing that.

regards

Jan

[Everest89]

Hi Jan,
Thank you for your answer besides beeing a bad one.
Really is the motherboard?
Do you know where i could get a new one or by the difference by a newer generation?

Best regards.

Carlos

[Everest89]

One more thing, if i buy a new motherboard is it safe to use the same components i already have?

[hunter86_bg]

'New' Motherboard -> ebay
It might be cheaper to buy a whole server.

About the components - it should be OK to reuse them, but there is no guarantee that they are still OK. (For example bad voltage control could damage a lot of stuff).

[Everest89]

What would you recommend?
New motherboard or for a difference of money get a new one or new generation?

[crystalyin4]

Please suggest the reson to change hardwaew log, server is not starting

"ID","Severity","Class","Last Update","Initial Update","Count","Description",
"7","Informational","System Revision","[NOT SET] ","[NOT SET] ","1","Firmware flashed (iLO 4 2.78)",
"6","Critical","Power","01/01/1970 00:01","01/01/1970 00:02","15","System Power Fault Detected (XR: 04 00 MID: FF 0D FC CF C0 FF FF 2D 2D 0C 0C 00 05 00 00 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00)",
"5","Informational","System Revision","[NOT SET] ","[NOT SET] ","1","Firmware flashed (ProLiant System BIOS - P70 05/24/2019)",
"4","Informational","System Revision","[NOT SET] ","[NOT SET] ","1","Firmware flashed (iLO 4 1.51)",
"3","Informational","System Revision","[NOT SET] ","[NOT SET] ","1","Firmware flashed (ProLiant System BIOS - P70 05/21/2018)",
"2","Informational","System Revision","[NOT SET] ","[NOT SET] ","1","Firmware flashed (ProLiant System BIOS - P70 05/24/2019)",
"1","Informational","System Revision","[NOT SET] ","[NOT SET] ","1","Firmware flashed (iLO 4 2.78)",

[sudhirsingh]

@crystalyin4 

It appears like sas backplane power fault.

Please remove SAS Backplane from the server and then try to power on (without sas backplane.. or makesure you disconnect sas cable and backplane Power cables completely) the server and see if its power on.

Regards,

Sudhir

[xrobau]

The last post by the HPE employee was the one that made me try it, and THEY WERE CORRECT! The problem IS THE SAS BACKPLANE!
I'm trying to post this response in all the google hits for the problem, but the simple way to check this is to unplug power to the SAS backplane AT THE BACKPLANE  (not at the motherboard, as that causes a similar error - it wants to see that cable connected), and the machine will turn on happily.
On the DL380e LFF that I had this problem on, there were TWO power cables connecting to the SAS backplane. The top one is causing the issue, so I assume that half of my drive bays aren't going to work, but at least the machine is booting now!