topic Re: EVA5000 controller/battery nightmare in HPE EVA Storage

EVA5000 controller/battery nightmare

yanaki2112 — Thu, 27 Nov 2008 15:57:20 GMT

Two weeks ago my EVA5000 started showing soft diagnostic controller errors. It works but HP recommens to replace that controller...and the nightmare begins.

Day one: HP Field Service replaces the failed contoller and it wont boot. We put back the old contoller and continues working.
Day two: HP field Service tries with another controller, that gives the same soft diagnostic erros. So we put back the old controller and continue working.
Day three: HP agains tries with a new contoller; but now uses the batteries of the existing contoller. Now the batteries fail with a RED led. They try to put back the old controller with those batteries, and now the batteries fail in the old controller as well. So now i cannot even start the old controller.
Day four: HP brings new batteries. After installing, they start charging, after a few hours, the batteries fail.
Day five: HP brings four new batteries. Install the first battery and it fails. Then install the second battery, that seems to work. Install another battery to replace the first that failed, and now both batteries fail the initial tests. Finally, they install just one, and that one has been charging for 10 hours now.

We have escalated this. According to HP, we need to wait for the battery (only one is installed right now) finishes charging; and that could take up to 24 hours.

Can anybody help us in this. I have lost fate.

Tks.

Re: EVA5000 controller/battery nightmare

Víctor Cespón — Thu, 27 Nov 2008 17:01:22 GMT

Error number 1 was to replace a controller because it shows "soft diagnostic failure".
That requires an analysis of the controller event log to see what caused it.
The solution is not always to replace the controller.

Mainly when the replacement controller shows the same message. There's a lot of things going on when a controller boots, needs a lot of log analysis and not 4 controllers and 8 batteries.

Re: EVA5000 controller/battery nightmare

yanaki2112 — Thu, 27 Nov 2008 17:09:18 GMT

Actually, HP was who decided after revewing the logs to replace the controllers. From there, has only be an issue of replacing failed parts as if this was a fridge. I just saw a video explaining the one battery takes 15 hours to charge; so ill wait 4 more hours and see if that works.
Also saw another thread with an article on how HP states there are problems with battery replacing.

So...ill wait today what happens.

Question: If ultimately, this battery works, whats the proper procedure for putting the second battery. Im afraid HP is going to do it wrong and fail the 1st battery too.

Re: EVA5000 controller/battery nightmare

IBaltay — Thu, 27 Nov 2008 17:23:55 GMT

Hi,
here are the recommendations:
a) never remove batteries from the controller while it is powered down, replace the cache battery only when the controller power is on
b) never install a battery that was previously failed by another controller.
c) both batteries have to removed before installing the replacement batteries. If one of the batteries is replaced whilst the other failed battery is still in the enclosure, the failure may be propagated to the newly installed battery. Furthermore, removing both batteries resets the battery system status information on the controller

Re: EVA5000 controller/battery nightmare

Rob Leadbeater — Thu, 27 Nov 2008 20:15:42 GMT

Hi,

It would be useful to know what analysis HP did before deciding to go down the replacement controller route. I assume they went through the controller logs. Can you post them here ?

It might be that you just had bad luck with the "new" batteries... Did they come out of sealed boxes with an expiry date on them ?

If not, then they may well be suspect.

Cheers,

Rob

Re: EVA5000 controller/battery nightmare

yanaki2112 — Thu, 27 Nov 2008 23:50:05 GMT

Its 6:46pm local time. More than 18 hours since the new battery was installed. Still shows as Low charge (charging according to HP).

The batteries (most of them came with no boxes, others with no expiration showing, so how knows.

My worst nightmare at this time is the battery not yet charged, and thus the controller not yet working, and my systems underperforming. HP now told me that it could take up to 96 hours to have the battery charged.

Im clueless.

Re: EVA5000 controller/battery nightmare

Rob Leadbeater — Fri, 28 Nov 2008 18:25:55 GMT

Hi,

> HP now told me that it could take up to
> 96 hours to have the battery charged.

I don't believe I've ever seen a good battery take more than 8 hours to charge. I would question whether the batteries are any good, especially in the absence of the original packaging.

I would push for someone at HP who know what they're talking about...

Cheers,

Rob

Re: EVA5000 controller/battery nightmare

yanaki2112 — Fri, 28 Nov 2008 19:43:58 GMT

Does anybody know anyone in HP thats an expert and could help me with this. So far i have nothing but a controller down. This is unacceptable paying such a huge contract, and receiving no answers.

Today, the battery has more thatn 40 hours being charging. According to HP, we need to wait until 96!!!!. That does not make any sense.

Re: EVA5000 controller/battery nightmare

Uwe Zessin — Fri, 28 Nov 2008 19:48:44 GMT

In the "good old days", a competent service center had a "manager on duty".

Re: EVA5000 controller/battery nightmare

yanaki2112 — Fri, 28 Nov 2008 20:21:05 GMT

Well, now HP is telling us that it makes no sense that after 40 hrs charging, the controller cannot even display the Luns.

So they are bringing another 2 batteries, as their plan.

After that, there is no plan B yet.

Re: EVA5000 controller/battery nightmare

IBaltay — Fri, 28 Nov 2008 21:01:19 GMT

The Competency centers are still very important part of the service support. So maybe its the right time to elevate the case to the EVA Competency center to give you the root cause analysis before any further interventions.

Re: EVA5000 controller/battery nightmare

Rob Leadbeater — Fri, 28 Nov 2008 21:41:29 GMT

Hi,

Have HP actually seen the controller logs ?

Cheers,

Rob

Re: EVA5000 controller/battery nightmare

Rob Leadbeater — Fri, 28 Nov 2008 21:46:32 GMT

Sorry, you already mentioned that earlier...

If you can, posting the logs here would be useful. People from HP, with the relevant tools, do frequent these forums, and might be able to make more sense of the logs than appears to have been made so far...

Cheers,

Rob

Re: EVA5000 controller/battery nightmare

yanaki2112 — Sun, 30 Nov 2008 02:02:39 GMT

I might just be closing this thread. After 5 controllers and 10 batteries, the final 2 batteries seemed to work. However, none of the controllers that HP shipped actually worked (the last one...the one that had 50 hours charging the battery finally failed with a memory or dimm error).

We had to install our original (soft diagnostic error) controller with a couple of new batteries...and finally it worked and everything back to normal.

HP was to upgrade our EVA5000 to a 8100 in December....but it will never happen; unless this installation is certified and actually performed in site by some 50+ year old EVA guru born anywhere but in latinamerica...and paid by HP with no charge to us.

The case is still open in HP; which is still a worry for us.

Tks everybody for all the comments. Actually some of the comments made us realize and actually "guide" HP in the right direction.

Conclusion: For latinamerica folks, beware of the parts you receive. If possible buy a couple of batteries yourself (they cost about $300 in the internet) and keep in your own inventory just in case.

Re: EVA5000 controller/battery nightmare

TTr — Sun, 30 Nov 2008 03:04:30 GMT

You keep mentioning Latin America!

Is it time to correct your "Country/Region in your profile?

Re: EVA5000 controller/battery nightmare

yanaki2112 — Sun, 30 Nov 2008 04:59:58 GMT

Tks for the observation. Just did the change.

Re: EVA5000 controller/battery nightmare

Rob Leadbeater — Sun, 30 Nov 2008 10:10:56 GMT

Hi,

I'm not sure of the support setup in Panama, but in many countries even though you pay your money to HP, it's not actually a HP engineer that comes out to site...

Back in the "good old days" (as Uwe called them), it used to be, and you'd get engineers who were properly trained on the kit they were coming to fix...

Make sure you keep pushing HP for the root cause analysis.

Cheers,

Rob

Re: EVA5000 controller/battery nightmare

Uwe Zessin — Sun, 30 Nov 2008 18:29:59 GMT

> keep in your own inventory just in case

No offense meant, but it does not make sense just to put them 'somewhere' unless you have a chance to 'maintain' them, because batteries discharge over time. A 'deep-discharge' even kills it.

That's why Rob asked if there was an expiry date printed on the box.

Re: EVA5000 controller/battery nightmare

hiroshi fung — Thu, 04 Dec 2008 03:15:09 GMT

>but it will never happen; unless this installation is certified and actually performed in site by some 50+ year old EVA guru born anywhere but in latinamerica.

Sorry EVA technology have not a lot time working and you will never find out a guru with that record experience, that not make sense.
In the computer environment things never happen as the manager wants "all a garden flower".

Is true that more care have to be taken.
After reading your history I will write in the same way.

Hiroshi

Re: EVA5000 controller/battery nightmare

Rob Leadbeater — Thu, 04 Dec 2008 11:31:57 GMT

It makes perfect sense to me...

He wasn't asking for a guru with 50+ years experience of EVAs, he was asking for a 50+ year old EVA guru. There's a subtle difference !

Cheers,

Rob