topic Re: FC60 array problems in Disk Enclosures

FC60 array problems

Juan M Leon — Sat, 08 Apr 2006 13:54:13 GMT

Hello, yesterday we have a disk that was complaining on our FC60 HP Surestore array. Besides the amber flahing light on the panel and drive slot we saw this entry in the amdsp -a output

SCSI Channel:ID = 1:3
Enclosure = 0
Slot (0-based) = 6
Disk State = NO RESPONSE
Disk Group and Type = 07A15C000A4001D9D9 LUN
Capacity = 0.0 GB
Manufacturer and Model = NO_VENDOR NO_MODEL
Serial Number = NO_SERIAL_NUM

Today HP-CE resitedt the drive and the amber light went away however what we see now is the following entry in the amdsp outpu. The disk state shows a replaced, we never replace any drive any suggestions as what happened here

SCSI Channel:ID = 1:0
Enclosure = 0
Slot (0-based) = 0
Disk State = REPLACED
Disk Group and Type = 07A15C000E4001DA73 LUN
Capacity = 33.9 GB
Manufacturer and Model = SEAGATE ST136403LC
Serial Number = LT047837
Firmware Revision = HP02

In the syslog I see the following entry any suggestions
HP Advanced Disk Array serial number 000800A0B8001B70: Disk drive inserted: Disk 1:2: LT278227: Vendor: SEAGATE : Product ID: ST136403LC

Thank you

Juan

Re: FC60 array problems

Thomas J. Harrold — Sun, 09 Apr 2006 15:42:05 GMT

Juan,

The array utilities are probably confused, as the drive "disappeared", and then was re-inserted.

The important thing is the state of any LUNs you have created. Can you get some more information, and see if your mirrored/raid sets are intact?

Is this unit under a support contract with HP? If so, log a call. They'll walk you through some more advanced diagnostics.

-tjh

Re: FC60 array problems

Arend Lensen — Tue, 11 Apr 2006 06:00:49 GMT

If a disk is in replaced status then the rebuild should kick off. The rebuild will start on the first write to the lun.
Has there been io to the lun yet?

Arend

Re: FC60 array problems

Juan M Leon — Tue, 11 Apr 2006 11:29:52 GMT

Thomas: thank you for your answer, We have array 5.
Arend: there was a rebuild state for shor period of time and seems that resynced very quick.

The intersting point is that the 1:3 hanel ID was OPTIMAL after re-seated the drive. And another 2 drives from LUN 5 become unavailable. Called HP they suggested to force enable the drives.
After the drives were enable everything went ok. Has anybody seeing a issue where the FC60 lose connectivity to the drives and after re-enabling the drives everything is ok.
HP is suggesting to upgrade the firmware from HP10 (current) to HP12.
Anyone has any insides on the differeces.

Thank you

Re: FC60 array problems

Arend Lensen — Tue, 11 Apr 2006 11:47:07 GMT

Juan,

I think its a good idea to upgrade to HP12. Please have this arranged.

Arend

Re: FC60 array problems

Jose Conde — Mon, 09 Oct 2006 07:56:57 GMT

Juan,

What were the steps for force enabling the disks? I am currently in the exact same situation with drives in REPLACED status after reinserting them.

Thanks,
Jose

Re: FC60 array problems

Arend Lensen — Tue, 10 Oct 2006 03:16:07 GMT

Jose,

With the following command you can put the drive in replaced status en start the rebuild:

amutil -rr 2:3

2:3 is just an example, you need to specify the drive that needs rebuilding.

Regards,
Arend

Re: FC60 array problems

Jose Conde — Tue, 10 Oct 2006 08:45:13 GMT

Not quite the answer I was seeking.

What I was looking for was the commands
that would take a working disk that was
reinserted and thus in a REPLACED state,
offline, then back online, to an OPTIMAL
state.

What I found and what worked was:

1) To take a disk offline

amutil -D

2) To bring a disk back online

amutil -d

Once the disks (3) were back online, the dead
LUN immediately began a rebuild and
successfully recovered.

Re: FC60 array problems

Arend Lensen — Tue, 10 Oct 2006 09:06:43 GMT

Jose,

I should NEVER put a disk online unless you know exactly what to do. Disks that have been offline should ALWAYS be rebuilded.
I did not give you the commands to do that on purpose as these are undocumented and very risky.

Regards,
Arend

Re: FC60 array problems

Arend Lensen — Tue, 10 Oct 2006 09:11:11 GMT

When a lun is dead because of multiple disks offline, then you could use it but you have to be sure that the first disk that went offline is rebuilded.
The last disks that went offline can be setup to "online" but not the first one that went away.
I hope you understand what i mean.
When just 2 out of those 3 disks are set to "online' and the other (wrong one) is rebuilded then you might cause data corruption.

Regards,
Arend