Error message on front panel of autoraid 12H "SYSTEM WARNING`.DATA UNAVAILABLE"

[Ahmed ABDOU_1]

I have an autoraid 12H which experienced a failed disk drive(location: A1).
The autoraid automatically rebuild the data from the failed disc on the hot spare disc
and all was ok. When I replaced the deffective drive, the rebuild went successfully but
we had a message on the front panel of the autoraid 12H "SYSTEM WARNING?.DATA UNAVAILABLE".

System rebooted the system --> same problem.
Autoraid rebooted -----------> same problem.
Replaced the disc in A1 -----> same problem.
Cleared NVram of autoraid ---> same problem.

Decided to upgrade the firmware of the autoraid to the latest revision but we are not able
to do it since we have this error message showing on the front panel of the disc array.

Can somebody shed some light on this strange problem? Thanks in advance and cheers.

[Insu Kim]

I'm not sure but there might be another potential problems.
Use logprint command to get the logs in detail.
Take a close look at that sparing was good, rebuilding has completed successfully and other things.

# logprint -a

Best regards,

[Mark van Silfhout]

Hi,

Data Unavailable Troubleshooting Techniques:
The following is a procedure to follow to overwrite the bad block(s) of data to bring the customer?s array back to a READY state. This procedure can eliminate much of the time required for a restore of an entire LUN or LUNs by enabling you to find the file(s) affected and overwrite those locations with ?good? files.
The only way to get the array back to a READY state is to OVERWRITE the bad data with good data. The WARNING message will go away immediately when the bad block(s) are overwritten.
..... However I would recommend to let an HP hardware engineer carry out these procedures.
Please keep in mind that upgrading the firmware also implicates a new ARM utilities patch.

Regards,

Mark

[Vladimir Lemos]

What is the procedure to write over bad blocks ? Has anyone fixed the data unavailability problem this way ? If not, was the problem solved at all ?

[Charbel Bou-eid]

Dear All,

Did anybody has the procedure to overwrite the defective sector without having to delete/re-create the LUN and rstore data.
I have already replaced the HDD having one Bad Sector (found with arraylog -d) but the rebuild is not yet over for me to see if the warning will stay or not.

Also, I had upgraded the firmware to HP62 in order to detect the affected LUN (as per the information found on the HP download page) but the logprint did not change (I can't see any "Additional Code Sense").

Does anybody think that this warning may affect the I/O performance of the AutoRaid on which an Oracle DB is installed.

Please appreciate anybody's help as the situation is becoming critical at the customer site who is "blaming" the poor performance of Oracle on the AutoRaid.

Thanks,
Charbel

[Vladimir Lemos]

You may wanna try the following document Ids:
lpg67023
lpg75005
lpg67074

man page for the following commands

arraydsp
logprint
lvextend

Review related messages.
This is something that caused big headache!
All that I've saved from this nightmare are the above referenced documents.

I don't recall exactly what I've done, but something along the lines of:

Backup data
create partition using lvextend command
move data into good partition
delete bad partition
recreate partition
merge partitions
didn't work
removed a large .pst (outlool mail) file someone had in disk array -> problem solved!
rebuild disk array

Good luck!

[Gary Badger]

we experienced this problem and with hp support solved it by using the unix dd command on each lun until we found the one where records in did not match records out. Having found the lun on which this occured, we wrote a huge (too big) tar file to each directory. This writes data to all unused areas and removed the error.

[Charbel Bou-eid]

Thanks Vladimir for this info.

Hi Gary,
This is what I had started to do 3 days ago, but unfortuantely I can only run 1 dd session a day as this is a production server, and unfortunately I didn't yet detected the affected LUN.
When you say "write a tar file", do you mean that you have a big tar file on tape for example and you untar it on each of the filesystems mounted under this LUN?
Also please I want to know if this warning has affected the performance of your AutoRaid (slow IO, timeout, etc...) then or you continued to work normally on the Autoraid?

Many thanks to all in advance.
Charbel

[Gary Badger]

Charbul
The large tar file was created to write into each directory on the LUN. It was not unzipped, it was designed to just take up space. Also, the error did not seem to affect system response however, our server is not heavily loaded.

[Tonya Underwood]

I do have the procedure to overwrite the data unavailable if still needed... Will write it up now and post shortly.