EVA 6400 Disk group failure.

[Agfa Helpdesk]

Hi all,

I want some thoughts on a major issue we have with one of our customers please?! :)

We have lost the only disk group on the EVA containing about 17 x 500GB vdisks to a SUN server serving a hospital imaging system (not good!)

From my initial examinations of the logs (and HP have reviewed the logs and dispatched an engineer also) is that Controller A has failed, but not passed mastership and thus the LUN's over to Controller B.

There have been some drives entering MISSING states as well but the hardware looks good.

My main concern is this error in the controller termination log:

"Controller A: Console requested restore without dump (not coupled)"

I'm stabbing in the dark but I'm presuming that the coupling it refers to is the connectivity between it and controller B?

In the disk group we have two options...a DELETE VDISKS and restore from Data replication...well we dont have DR so that leaves us with the other option 'CONTINUE AS NORMAL' which I presume trys to bring things back despite a potentially damaged state.

I assume this is a feature of CV9x as we have only recently upgraded most arrays from CV6 and this EVA 6400 is obviously a newish install requiring CV9.

The F/W is at the current recommended level.

I've attached a copy of the log files folder if anyone fancies a peek!

Any ideas appreciated folks!

Thanks!!

Ryan Harris

[SUBHAJIT KHANBARMAN_1]

Hi,

I have looked at the logs and found that there are loop errors captured for the HDD location Enc 3 Bay 4 and Enc 4 Bay 8.

Also there are 31 Medium errors for Enc 3 Bay 4 and 55 for Enc 4 Bay 8.

These two HDD needs to be replace.


Now, coming to the root cause, I had a similar case and it was found the root cause of these disk failures was caused by a faulty backend loop cable.

Action Plan,
1. Isolated the suspect cables by EVA backend loop diagnostic method.
2. Replace the suspect cables as soon as possible.


Why the controller failed these disks in this situation is still under investigation.

Can I ask you to send me the configuration log as well?

I hope you have already logged a case with HP and it needs high level investigation.

[Johan Guldmyr]

Hi,

Wouldn't it be possible that if a disk is having medium errors and loop errors, that the controller believe there is something quite seriously wrong with those two disks. So much that it might take them off the loops and fail them?

About the coupling, I believe it means that if there was restore for both controllers, it would indeed be coupled. Not that coupling necessarily refers to the connectivity. A couple. A pair. Something like that :p

Before you'd click on the "continue as normal" - you definitely want to take care of the back end loop issues. I don't have the tools to look at the logs but a reason it doesn't fail over is because of back end loop issues.

It might be hard to find the config - or do you get that far in CV? (server options -> capture system information -> controller configuration dump)

Then if you cannot get the config, as you may have two broken disks - do you have backup? Hopefully the two disks are in different RSS:s..

[SUBHAJIT KHANBARMAN_1]

I forgot to mention the following,

From SMA or CV EVA server -> Go to the folder C:\Program Files\Hewlett-Packard\SANworks\Element Manager for StorageWorks HSV\cache.

Logs are then located under the identifiable EVA WWN.

Either send me only the configuration file or try to zip the eva as per WWN and send me the same.