EVA6100 Drive Bays Disapear After Inserting Disk

[Bob Firek]

I have an EVA6100 with four drive trays populated with 48 disks, 12 drives per tray. We want to fill up the remaining eight slots and ordered eight additional drives. When I inserted the first drive in tray 1 bay 13 and checked the Command View it showed me an error with all of the disk in Unmappable Hardware. I checked the disk groups and they seem fine. Has anyone seen this type of thing before?

[Bob Firek]

Strange. The error just seem to have cleared itself and now the disk enclosures are showing fine.

[Uwe Zessin]

Looks like the system got confused for a moment which _should not_ happen during a simple disk add. I recommend you take a short look into the event log or let somebody do it -- just to make sure nothing bad happened and got unnoticed.

[Bob Firek]

Uwe,

Good point. I started digging into the
Controller Events and I found a critical alert "An unrecognized Fibre Channel physical disk drive firmware revision has been detected by the physical disk drive firmware load process.Corrective action code: 4d"
Under more details I get:
Event header details:
Event sequence number: 12521
Flags: Time_set Time_synched Primary_Ctlr
Event rev: 2
Event size: 160
Software version: 6110
Base level: CR0ECA
Ctlr model: HSV200
Ctlr handle: 0000-7-8-500508B4-0006E12C-00000000-00000000
Report time: 3 Sep, 2009 10:41:11.015334
Location: 0x00181400
Event code: Type: 04 Cac: 4D Num: 35 Scid: 06
Event description: 0x06354D04 Severity: Critical -- failure or failure imminent. An unrecognized Fibre Channel physical disk drive firmware revision has been detected by the physical disk drive firmware load process.
Information packet rev: 06
Information packet type: 04
Information packet size: 0060
Software component: Fibre Channel Services
Corrective action: Load the latest physical disk drive firmware superfile for the physical disk drive type shown in the SCSI Product ID of this event's detailed information. If no such superfile exists, the physical disk drive is unsupported, and should be ungrouped and removed. Using a superfile that updates the controller approved firmware table may be sufficient to correct the problem.

Event structure details:
Information packet descriptor:
EIP04 - Fibre Channel Services Physical Disk Drive Error
An error was encountered while accessing a physical disk drive.
Structure: Event
Structure: flags
time_set: TRUE - Time has been set on this HSV200 controller
time_synched: TRUE - Time has been synchronized with all HSV200 controllers in the Storage System
seq_reset: FALSE - Event sequence number reset occurred
outofsequence: FALSE - Event reported out of sequence due to Final Event Block reconciliation or lost host event
requeued: FALSE - Event requeued following restart or resynchronization
labcode: FALSE - Event reported using LAB code
prictrlr: TRUE - Event reported by primary HSV200 controller (Note: Not valid until Storage System primary HSV200 controller is elected)
spsctrlr: FALSE - Single power supply HSV200 controller
End of structure
revision: 2 [0x02] - Structure revision number
count: 160 [0x00A0] - Event specific information size in bytes
sequence_number: 12521 [0x000030E9] - Sequence number assigned to the event
sw_version: 6110 - HSV200 controller software version number string
baselevel_id: CR0ECA - HSV200 controller baselevel build string
ctrlr_model_id: HSV200 - HSV200 controller model string
reporting_ctrlr: 0000-7-8-500508B4-0006E12C-00000000-00000000 - Storage System Management Interface Handle of HSV200 controller that reported the event
report_time: 3 Sep, 2009 10:41:11.015334 - Time event was reported
report_location: 1577984 [0x00181400] - Location of event report call
Structure: header
Union: u
Structure: ec
eiptype: 4 [0x04] - Event Information Packet Type Code
cac: 77 [0x4D] - Corrective Action Code
evnum: 53 [0x35] - Event Number
scid: 6 [0x06] - HSV200 Controller Software Component Identification
End of structure
End of union section
value: 104156420 [0x06354D04] - Event Code Value
End of union
End of union
revision: 6 [0x06] - Packet revision number
type: 4 [0x04] - Packet type
count: 96 [0x0060] - Number of bytes in packet
End of structure
device: 0C-00-08-20-58-02-41-CA-00-00-00-00-00-00-00-00 - UUID of physical disk drive associated with the event
cerp_id: DP-1A - HSV200 controller enclosure rear panel Fibre Channel port attached to the physical disk drive or mirror port
al_pa: 0 [0x00000000] - AL_PA of the physical disk drive or mirror port
dencl_num: 1 [0x0001] - Enclosure where the physical disk drive is located
port: 0 [0x0000] - HSV200 controller internal Fibre Channel port number attached to the physical disk drive or mirror port
rack_num: 0 [0x0000] - Rack where physical disk drive is located
bay: 13 [0x000D] - Enclosure bay where the physical disk drive is located
pid: BF300D6188 - Physical disk drive product identification string
rev: HP02 - Current firmware level of physical disk drive
Structure: enclosures[1]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 3 [0x03] - Enclosure number
End of structure
Structure: enclosures[0]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 2 [0x02] - Enclosure number
End of structure
Structure: enclosures[3]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 1 [0x01] - Enclosure number
End of structure
Structure: enclosures[2]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 4 [0x04] - Enclosure number
End of structure
Structure: enclosures[5]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 99 [0x63] - Enclosure number
End of structure
Structure: enclosures[4]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 99 [0x63] - Enclosure number
End of structure
Structure: enclosures[7]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 99 [0x63] - Enclosure number
End of structure
Structure: enclosures[6]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 99 [0x63] - Enclosure number
End of structure
Structure: enclosures[9]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 99 [0x63] - Enclosure number
End of structure
Structure: enclosures[8]
rack_num: 0 [0x00] - Rack were enclosure is located
dencl_num: 99 [0x63] - Enclosure number
End of structure
bypass_reason: 0 [0x00000000] - Reason the physical disk drive at this location has been bypassed
new_rev: - Latest known firmware level of physical disk drive
bypassb: 40960 [0xA000] - Mask showing bypass state for each slot in a shelf
bypassa: 40960 [0xA000] - Mask showing bypass state for each slot in a shelf
End of structure

Salient data:
Event: Severity: Critical -- failure or failure imminent. An unrecognized Fibre Channel physical disk drive firmware revision has been detected by the physical disk drive firmware load process.
Action: Load the latest physical disk drive firmware superfile for the physical disk drive type shown in the SCSI Product ID of this event's detailed information. If no such superfile exists, the physical disk drive is unsupported, and should be ungrouped and removed. Using a superfile that updates the controller approved firmware table may be sufficient to correct the problem.

It seems like it indicates that the firmware code on this drive isn't correct. But I checked and HP02 is the latest firmware for this drive model number BF300D6188. They are factory fresh drives. Made June 2009. The EVA6100 is at least 18 months old. Can that be a problem. The other drives in the array are BF300DA47B with a firmware version of HP01. I think HP02 may be the latest for those drives according to HDDBundled_Image_2009_05_12. Maybe I should update the older drives first. I was going to do that after inserting these new drives and get them all in one shot. Maybe that was the wrong thing to do. What do you think?

[Víctor Cespón]

That message by itself does not indicate a failure. It's just that the disk you have inserted is more recent than the firmware on the EVA.

The fact that the disks appeared on "unmappable hardware" indicates an interruption of the communication with the enclosure EMU, that can happen when inserting a disk. Communication was restored a little later.

If you have not done so yet, you should update the firmware on the controllers. The 6.110 version they came with is currently out of support.

[Sheldon Smith · Accepted solution]

Just looked on SPOCK - http://www.hp.com/storage/spock
6.110 *is* supported. Note it is the oldest supported version, with 6.220 as the recommended supported version. You *should* be making plans to upgrade the EVA firmware as soon as conveniently possible.

[Bob Firek]

vcespon and Sheldon,

Thanks for the reply and advice. I have been working on getting the controllers upgrades but progress has been slow. I still have two FC switches to upgrade first before I can move on to the controllers. The two fibre channel switchs are Cisco MDS9124. One switch is running 3.1(3a) code and the other is running 4.1(1b) code. I have to updated then to the latest code which I think is 4.3(3a). At least that was the latest code when I starting working through this about a month or so ago. Do you know if this is the latest code or is there a later one out there.

I do have a question: Would it be safe to populate the EVA with the other disks or should I remove the last drive I put in that caused this trouble. It is not grouped to and disk group so I wouldn't think that it would matter. But I've been supprised before.

Also any tips that you would care to pass along on thinks I should look out for in either upgrading the FC switches or the EVA controllers I would consider it a great kindness.