Proliant dl 380 g6 - unexpected reboots

[kühlprofi]

Since some weeks a server is rebooting byself unexpected. Before these symptoms started we replaced a disk but this is looking ok in HP insight manager.




First Error Message is
===========================


The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly.

Second Error Message is
===========================


Power-On-Self-Test (POST) errors occurred during the last system startup.

User Action
Check the Power-On-Self-Test (POST) errors, and take corrective action as needed.

WBEM Indication Properties
AlertingElementFormat: 0 0 (Unknown)
AlertType: 5 0x5 (Device Alert)
Description: "Power-On-Self-Test (POST) errors occurred during the last system startup."
EventCategory: 4 0x4 (System Hardware)
EventID: "1"
EventTime: "20110509045100.079000+000"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{6F57FA3F-20C3-47FE-A35D-90B77926DABB}"
IndicationTime: "20110509065057.974000+120"
NetworkAddresses[0]: "fe80::5d76:f502:95e5:955a%15"
NetworkAddresses[1]: "172.22.72.16"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 5 0x5 (Major)
ProbableCause: 8 0x8 (Configuration/Customization Error)
ProbableCauseDescription: "POST Errors Occurred"
ProviderName: "HP POST"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "Check the Power-On-Self-Test (POST) errors, and take corrective action as needed."
Summary: "POST errors occurred"
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2009.07.24"
SystemFirmwareVersion[1]: "2009.07.24"
SystemGUID: "33313934-3531-5A43-4339-3432364B4646"
SystemModel: "ProLiant DL380 G6"
SystemName: "ufasrv5924.main.corp.fenaco.com"
SystemProductID: "491315-421"
SystemSerialNumber: "CZC9426KFF"
TIME_CREATED: 129493902600799336 0x1cc0e04b1198468
VariableNames[0]: "POST Error Code"
VariableNames[1]: "POST Error String"
VariableTypes[0]: 3 0x3 (uint8)
VariableTypes[1]: 1 0x1 (string)
VariableValues[0]: "223"
VariableValues[1]: "POST Error: 1716-Slot X Drive Array - Unregenerable Media Errors Detected on Drives during previous Rebuild or Auto-Reliability Monitoring (ARM) scan. Problem will be fixed automatically when the sector(s) are overwritten."

For more information, please contact HP Support.

Another Error Message after an unexpected reboot: "POST Error: 1792-Drive Array Reports Valid Data Found in Array Accelerator"

=====================================


Power-On-Self-Test (POST) errors occurred during the last system startup.

User Action
Check the Power-On-Self-Test (POST) errors, and take corrective action as needed.

WBEM Indication Properties
AlertingElementFormat: 0 0 (Unknown)
AlertType: 5 0x5 (Device Alert)
Description: "Power-On-Self-Test (POST) errors occurred during the last system startup."
EventCategory: 4 0x4 (System Hardware)
EventID: "1"
EventTime: "20110507021343.433000+000"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{FA7FEB9F-5992-4AFF-B662-A846714DAEEA}"
IndicationTime: "20110507041341.386000+120"
NetworkAddresses[0]: "fe80::5d76:f502:95e5:955a%15"
NetworkAddresses[1]: "172.22.72.16"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 5 0x5 (Major)
ProbableCause: 8 0x8 (Configuration/Customization Error)
ProbableCauseDescription: "POST Errors Occurred"
ProviderName: "HP POST"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "Check the Power-On-Self-Test (POST) errors, and take corrective action as needed."
Summary: "POST errors occurred"
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2009.07.24"
SystemFirmwareVersion[1]: "2009.07.24"
SystemGUID: "33313934-3531-5A43-4339-3432364B4646"
SystemModel: "ProLiant DL380 G6"
SystemName: "ufasrv5924.main.corp.fenaco.com"
SystemProductID: "491315-421"
SystemSerialNumber: "CZC9426KFF"
TIME_CREATED: 129492080234335645 0x1cc0c5c63980d9d
VariableNames[0]: "POST Error Code"
VariableNames[1]: "POST Error String"
VariableTypes[0]: 3 0x3 (uint8)
VariableTypes[1]: 1 0x1 (string)
VariableValues[0]: "92"
VariableValues[1]: "POST Error: 1792-Drive Array Reports Valid Data Found in Array Accelerator"

For more information, please contact HP Support.



Is this a firmware/driver error or could this be also a hardwareproblem?

I fixed the System performance to "high performance"


BR Andreas

[Diego Castelli]

Check Integrated Management Log Viewer to see POST errors without reboot.

let us know of any updated error messages

[kühlprofi]

Hi,

Okay, here it is.
We had to replace a disk some weeks ago, could this be the reason for this?

[kühlprofi]

Post Error - Message: 1716-Slot X Drive Array - Unregenerable Media Errors Detected on Drives during previous Rebuild or Auto-Reliability Monitoring (ARM) scan. Problem will be fixed automatically when the sector(s) are overwritten. - Error: 223

[Diego Castelli]

Well, i lost the whole extra-long post i've done for you and don't know why....

Anyway, here it is:

IMO you should:

1. take an image of the server (verify the software you are using and all the options. Take test run on an expendable machine)

2. Until verified the consistency of the image Boot from the last supp. Smart Start for your system, Destroy the array, optionally recreate the array.

3. take an extensive test on disks subsystem with Insight Diagnostic OFFLINE Edition.

4. optionally replace every faulty component until it passes every test.

5. re-create the array if not done before.

6. deploy the image back

7. everything should be fine.


I'm sorry to tell you must do such hard work, but in my Exp. this is the only (quick and dirty) way to solve the problem NOW.


HTH, BB.

[Diego Castelli]

errata corridge:


here's the correct step 2.

2. Once verified the Server image consistency, Boot with the last supported smart-start and destroy the array. Optionally you can re-create the array after a cold boot (not required, but IMO is better)

[kühlprofi]

Hi Diego Castelli,

Ouh.. Thank you for your assistance and good instructions.
Yes this is looking like hard work (it's an SQL Server with a lot of important/critical databases), but i think i have no other choice. ;|

We have 6 Harddisks. I have configured 3 times a Raid 1(2 disks).
Do i need to backup and restore all 3 RAID's or only the faulty one?

How can i see which RAID of them 3 is the faulty one?

[Ghatkey]

Hi Kuhl,

POST Error "1716 - Slot 1 Drive Array - Unrecoverable Media Errors Detected on Drives during Previous Rebuild or Background Surface Analysis (ARM) scan. Errors will be fixed automatically when the sector(s) are overwritten. Backup & restore recommended 

Device : HP ProLiant DL380 G6

OS : WIn 2008/Win 2003

Contlr: P410i

Try This:

1.http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=15351&prodSeriesId=3884082&swItem=MTX-0d9c28b17e4c4fa5b1dd1694b5&prodNameId=3884083&swEnvOID=4022&swLang=8&taskId=135&mode=5

This might work unless, there is legitimate HW issue with any of the mentioned components:

controller,cache.HDD,cache battery.

Vijay V Ghatkey

[Modris Bremze]

You mentioned that you replaced a hard drive a few weeks ago. If so - ensure that all drives in your server have the same firmware level. The new/replaced drive may well have a newer fimrware than your "old" drives. Also, looks like your server doesn't have the latest system firmware as well. You can use smart update firmware DVD (http://h18004.www1.hp.com/...) to update all components at once.

[Dee Todd]

I realize this is an old post but am experiencing the same issue.  I checked the HP Smart Array P410i controller firmware version - it's 1.66.  my question is can I upgrade from that version - 1.66 straight to 6.00?