Redundancy Failure testing EVA 4400 controllers

[gregersenj]

I agree Torsten.

And it's not a nice thing to do.

Even though a deviceport failure on both ports of a controller, is highly unlikely.
Then I would expect it to suvieve it.

BR
/jag

[Torsten.]

Review for example this document

http://bizsupport1.austin.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c01629457〈=en&cc=us&taskId=110&prodSeriesId=3664763&prodTypeId=12169&printver=true

and you will see this kind of test is also not a good idea.

What version of code is running, by the way?

[gregersenj]

What do you want to test?

The redundancy of the Storagebox?
or
The Redundancy of the SAN?
or
The Redundancy of you VMWare?

The EVA is fully redundant. It will continue running no matter witch component fail.
There are some dependencies, wether it can run with a full disk shelf failure.
The EVA is designed to be Changed/reconfigured, maintaned and serviced online.
You can replace any part online. On the 4400 there are some exceptions, The Controller chassis and midplanes. And again a disk chassis depends on the number of disk shelfs.
There's also some limitations on the CAB bus, on the other EVA's.

If you really want to test if you can remove a controller, you can shutdown 1 of the controllers from CV, or the WOCP.
Remove all cables for that controller. Thats the closest way to a real controller failure, and a controller replacement. You can even try to remove the controller after shutdown.

If you want to test the SAN, you can shutdown one of the fabrics.

By the way. Command view is running on the master controller, so if you shutdown the master, it must failover CV.
On a 4400 controller 1 allways become master, when the controller enclosure is powered on.

As to why the EVA hung on your test, you could try to open a case at HP, and have them analyze the log. They properbly will tell you not to test this way. But they might be able to tell why it became hung.

>Torsten, I don't see the relevance for this advisory.

BR
/jag

[Torsten.]

The advisory should suggest to NOT pull a running controller ...

[gregersenj]

Yes don't pull a running controller :)

BR
/jag

[Torsten.]

Some time ago I heard HP sold meanwhile 100000 EVAs - so IMHO the basic functionality should be tested enough.

I would consider to play around with the access paths, e.g. disabling some paths between the servers and the EVA to test the frontend connections and to check if the servers/san is properly configured and are able to use all the paths.

[Uwe Zessin]

I've read that, too, but they did not sell 100,000 EVA-x400 !!

And some problems simply result from wrong design decisions -
no amount of testing alone can fix that.

[Torsten.]

Agreed.

Apropos design, the guys should take a look at the XP array to learn what "online firmware update" really means ...

;-)

[Uwe Zessin]

Let's be fair, here.

The XP is an enterprise-class fully-symmetric cache-centric array.

The EVA is a midrange array that works asymmetric-active/active (due to the vDisk ownerships). And controllers need to be "re-synchronized" from time to time ;-)

[linuxbass]

Controllers running XCS 09534000
CV is running at Version: 9.03.00.100811

We've decided to buy another array, probably an EVA 8400 as after this incident we are not comfortable with the redundancy or behaivior of CV after this test. We will of course make some other redundancy and simpler tests for instance break one side SAN optical FC links to both controllers to the SAN. Break both FC optical links to one controller etc. work with CV and switching between CV host and chasis CV.

Also we will be going through excercises like downing a controller, removal, replace to be prepared in the event of an issue. The test scenario was remote and I now understand it was not a true test of controller failure, however it was a test of sorts and the array stopped serving storage as well as quit it's administrative access. This, to me, is not acceptable and we will seek further redundancy in buying a 2nd disk array for our SAN.

Thank you for the help so far, I may post my findings here as to assist other owners of this product.