DL380G3 SCSI drives blinking amber after successful firware update

[ghooper01]

Here's the situation:

HPDL380G3 with all 6 drive bays populated with 73GB 10K SCSI drives. Server was working perfectly before a rebuild of the OS and a firmware update. I used the 7.91 Firmware update CD to bring everything current. The server was way behind on all firmware, so the update included BIOS, Array controller, all drives, and ILO. All firmware updates reported successful installs and the operating system (2003 Ent SP2) booted without issues. The problem is that drives 0,1,4,& 5 are blinking amber non-stop and the POST indicated imminent failure has been registered, but when going into the Array Controller utility, either in POST or in Windows, nothing is reported as bad.
I searched this forum and found nothing, so my apologies if this is a known issue that is already posted. Thanks for any help.

[ghooper01]

And I should clarify that I built the OS with SS 7.91.

The amber blinking starts prior to the OS load, right after the Array Controller initializes.

[Jae Ellers · Accepted solution]

I'd load the PSP and check the SMH to see what is going on.

Blinking amber means predictive failure.

It does seem unusual that you'd have 4 bad disks in a single server, but that is an older system and we've lost 4-5 disks over the years on a couple of these systems.

[ghooper01]

Thanks for the reminder on the SMH. I had forgotten that it displayed soft and hard errors on the drives.

Interestingly enough, it only showed errors on drives 4 and 5, not on 0 and 1. But all 4 are blinking amber.

I am trying to decide if I want to pull one at a time and let them rebuild? All six are in a single RAID 5 logical drive.

[ghooper01]

I ended up having to replace drives 4 and 5, and then the blinking ambers ceased on 0 and 1. Interesting why 0 and 1 ever blinked in the first place because they show no errors.

Thank you for the reminder on the SMH. Points awarded.

[jayr]

I may have an answer as to why your hard drives show immenent failure just after applying the firmware update. I can only verify this with the MSA20 array controllers, but it appears that HP may have added a new trigger to their array controllers based on excessive Urecoverable Read Errors (URE)...

"On drives that repeatedly report URE on a Logical Block Address (LBA), the controller will mark the drive as imminent failure (blink amber LED on drives that misbehave due to excessive UREs)."

If this feature was added to the firmware of other SmartArray controllers, then it would explain why we are having numerous Ultra320 SCSI hard drives indicating "imminent failure" within 30 minutes of a 7.70 to 7.91 firmware updates.

[ghooper01]

yeah, i read about that on the MSA's and was wondering. So I guess it is a "wait for 7.92" type of thing then. LOL