Rebuilding error on LC2000 server

[WW156629]

Hi,
I have exactly the same issue posted in threadId 1188491 with an LC 2000 NetServer.
Please, can you take a look at attached JPG.
I have a RAID 5 situation composed by six 18GB HDs: 5 disks are composing the unique logical drive, the 6th was as hotspare. Yesterday I realized a red led on front panel.
Now the situation is:
-if I try to rebuild the fail disk (ID 9) I get a "rebuild error" message;
-if I try to change the READY disk (ID 3) in hotspare, it tries automatically to rebuild itself but after few seconds I receve an error and it changes in FAIL status. Contemporarily the prevoius FAIL disk (ID 9) changes in READY status;
-if I try again the "ping-pong" continues!

One of the other ON-LINE disks (ID 1) has some "Media Errors" and this should be the reason of rebuild error.
Looking at other HP articles I found that one of the suggestion is to try to upgrade FW of disks (see article: http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=110&prodSeriesId=50440&prodTypeId=15351&prodSeriesId=50440&objectID=tis13104), but on other posts/articles it is strongly recommended do not upgrade FW wher an array is in CRITICAL status.

Has someone any suggestion? Is the only solution to backup the server, recreate the entire RAID configuration and reinstall NOS and data from backup?

Thanks and Regards
Paolo

[kris rombauts]

Paolo,

the disk f/w upgrade is not going to cure the media errors on ID=1, so it will not help to fix the rebuild issue you have now.

The only solution is to backup/repalced bad disks/re-create array/restore data unfortunately.

Did you had the Netraid monitor service running on this server ? This periodically(once a week i.e.) checks the complete parity in the array and corrects any soft errors, this to avoid running into the issue this array is in now.

The Netraid Assistant log file will explain why the rebuild stops, so you can post the log file and we can look it up.
The log file can be retrieved with the 3th icon (counting from the right) of the icons at the top of the Netraid Assistnt GUI.


HTH

Kris

[WW156629]

Hi Kris,

unfortunately it is just what I imaginated!
I haven't the Netraid monitor service, but now I understand its utility...too late!

Here below the log:
********************************************
Log Started.
- Wed Oct 29 12:58:20 2008
DRIVE STATE changed in CHNL 0 TARG 9 to HOTSPARE - Wed Oct 29 12:58:51 2008
Rebuild Started on CHNL 0 TARG 9. - Wed Oct 29 12:58:56 2008
Notify message : DRIVE STATE changed in Ch 0 ID 3 to READY - Wed Oct 29 12:58:56 2008
Notify message : DRIVE STATE changed in Ch 0 ID 9 to REBUILD - Wed Oct 29 12:58:57 2008
Error on Rebuilding CHNL 0 TARG 9 - Wed Oct 29 12:59:04 2008
Notify message : DRIVE STATE changed in Ch 0 ID 9 to FAILED - Wed Oct 29 12:59:09 2008
NOTIFY:Check Condition on Ch 0 ID 1 with the following sense key - Wed Oct 29 12:59:13 2008

Time Stamp Date = On Oct,28 2008 At 12:58:8

The CDB = 28 00 00 02 89 00 00 00 80 00

Sense Data = f0 00 03 00 02 89 39 28 00 00 00 00 11 01 00 80 00 3f
********************************************

Thank you for your support

Best Regards
Paolo

[kris rombauts]

Paolo,

this decodes into this :

Byte 2: Sense key = 03 = Medium Error
MEDIUM ERROR. Indicates that the command terminated with a non-recovered error condition that was probably caused by a flaw in the medium or an error in the recorded data.

Byte 12: ASC = 11 = \ unrecovered read error
Byte 13: ASCQ= 00 = /


So if you rebuild the array, get disk id=1 out of it also besides the disk that failed first, i guess this must have b een ID=3 (if ID=9 was your original hotspare).

The Windows log file should indicate which disk failed first, go back in history of the system.evt log file and if the monitor s/w was installed, it will show how it all started.

HTH

Kris

[WW156629]

Kris,
I thought that a little portion of log could be enough.
I attached the entire log now.


Thanks and regards
Paolo

[kris rombauts]

Paolo,

the first time the issue got detected was way back in July on ID=1:

"NOTIFY:Check Condition on Ch 0 ID 1 with the following sense key - Tue Oct 28 19:36:11 2008

Time Stamp Date = On Jul,2 2008 At 22:11:32 "

Then on 28 Oct ID=3 kicked in to rebuild, so i assume ID=3 was defined as a hotspare in the original config.
Then ID=9 was defined as a hotspare.

Anyhow, config and history is not 100% clear but there is no way to fix it unfortunately.

Kris

[WW156629]

Hi Kris,

thank you for your support.
I'll take care of your suggestion for the future (NETraid monitor).


I'll proceed with server reinstallation.

Regards
Paolo