topic EVA4100 can‘t complete leveling operation in HPE EVA Storage

EVA4100 can‘t complete leveling operation

Jianmin Tan_1 — Tue, 05 Jul 2011 17:36:43 GMT

Dear all:

One of my customer have an EVA4100 storage system with two controllers and one disk enclosure.

And there are fourteen 300GB FC hard disk in that enclosure.

He create a disk group by use these hard disk and select the single mode for the disk failure protection.

On this basis,he used all sapce to create a Vdisk which raid type is RAID1.

Then there are no available space in this storage system.

Some days ago he told me there were two hard disk and two cooling system failure.

I replaced the two failure cooling system with the new hardware but the state was still failure.

I try to restart the controller,after that the state of cooling system become normal.

I suspect that there are some problem with the controller,so I replace the cooling system by useing old one and it works normal too.And the cache battery state of that controller is low charge for several days.Are there any question with the controller?

Secondary,I remove the two failure hard disk and insert the two good hard disk after 2 minutes later.Then group the two disk to the default disk group .

But the state of leveling was abnormal.The storage system can't complete leveling and the disk failure protect of the disk group was none.I found that there are two hard disk had some question because the occupied was 0.One of them was insert by me and another was original.

Can somebody told me what can I do now.

Thanks a lot for your kindly help.

jianmin Tan

Re: EVA4100 can‘t complete leveling operation

Johan Guldmyr — Wed, 06 Jul 2011 05:36:41 GMT

Hey, I would attach the controller event log and the configuration dump in this post. Hopefully somebody with the right tool has time to go through the logs and help you out.

If your battery is low - replace that too? What firmware are you on? In the latest I believe there were some more improvements to the battery mechanisms.

If I understand this right: you put in two new disks, one of these and a third one has occupancy 0 - is that correct?

If so then I guess that means you had three disks failed - even larger chance that two of them were in the same pair.

Re: EVA4100 can‘t complete leveling operation

Henry A.C. Jansen — Wed, 06 Jul 2011 18:25:17 GMT

Hi,

Perhaps a dump solution... but if is is just one big LUN of 2 TB (and there was space left in the LUN) you may consider to do a backup/restore action and re-initialize the array again.. otherwise you have to 'create' more space.. i.e. attaching an extra enclosure with a few disks...

Cheers..

Re: EVA4100 can‘t complete leveling operation

Jianmin Tan_1 — Thu, 07 Jul 2011 04:09:13 GMT

Dear JohanGuldmyr&Henry A.C. Jansen:

Thanks a lot for your kindly help.

My customer ungroup the two disk yesterday which occupied were 0.(He had backup the data before)

And group them into the default diskgroup again.

Then the storage system begin the first leveling operate.

At the first leveling operate,the available space become more and more,but the data failure protect was still none.

After the first leveling finished,it's begin the second leveling operate.The available space become less and less,but the data failure protect become single when the leveling operate completed.

Why the storage system needed twice leveling operate?

The attachment is the log,thanks a lot for your remind JohanGuldmyr.

Re: EVA4100 can‘t complete leveling operation

Johan Guldmyr — Thu, 07 Jul 2011 05:25:31 GMT

I think you forgot the log.

So now levelling is completed? No failed disks? That's great!

What if you set the disk protection level (not data failure protect?) to none and then back to single?

Re: EVA4100 can‘t complete leveling operation

Jianmin Tan_1 — Thu, 07 Jul 2011 06:15:17 GMT

Dear Johan Guldmyr,Thanks a lot for your remind again.My rememberance was so bad：（

The leveling had completed at today‘s morning.There are no failed disks now.

I had never change the disk driver failure protection before.It's change by system.

I think that the first leveling operate was used to made the data distribute on all disks because the disk failure protection was still none.

At the second time,the disk failure protection become single when the leveling operate complete.

So I think the second time was used to made the disk failure protection space distribute on all disks.

Am I right?

Re: EVA4100 can‘t complete leveling operation

Johan Guldmyr — Thu, 07 Jul 2011 06:32:43 GMT

Ah good your protection level also went back to single.

The protection space is "empty" - there is no data on it.
It's reserved space that can for example be used for reconstruction when a drive fails.

The EVA doesn't need to level/distribute that space. Levelling is when the EVA distribute so that all disks have an equal amount (in %) of data on them.

Maybe it only "levelled in" one disk at a time?

Re: EVA4100 can‘t complete leveling operation

Jianmin Tan_1 — Thu, 07 Jul 2011 07:14:25 GMT

Dear Johan,I know what you mean.

But I think when there had two disk failure at the first time,the storage system will begin leveling operate,and the disk failure protection space had to used to construct the data because there had no available space in disk group.

In this basis,the disk failure protection will become none.

When I insert two new disk and group them into the disk group.The storage system begin leveling operate again.

Why the system needed twice to complete the leveling operate?

In my opinion,the first time is used to distribute the data and the second time is used to construct the data failure protection,because the system can't complete these operate in once leveling.

Re: EVA4100 can‘t complete leveling operation

Johan Guldmyr — Thu, 07 Jul 2011 07:30:35 GMT

Maybe first time it levels in the two new disks.
Then the second time it levels between all the disks.

That's my guess now :)

Maybe somebody else has some insight?

Re: EVA4100 can‘t complete leveling operation

Jianmin Tan_1 — Thu, 07 Jul 2011 09:09:38 GMT

Can only think so like this..Hope some body can explain this situation.

Anyhow,thanks a lot for your help Johan.

Re: EVA4100 can‘t complete leveling operation

hvhari — Thu, 07 Jul 2011 13:59:23 GMT

In EVA data redundancy has priority. So when there is no redundancy available, first job it will do when disk space is available is to re-construct the data so that redundancy & protection level is regained. This process is call "re-construction"

Once this is done, "levelling" is undertaken which only distributes data equally across all disks in the disk group. this is a low priority job and has no relation to redundancy.

Re: EVA4100 can‘t complete leveling operation

Ruben Renders — Sun, 10 Jul 2011 13:04:18 GMT

The releveling process has a low priority. Host-IO will be answered first, all other operations (releveling, management operations, etc) will be postponed if the array is heavily used.

Is the operation completed? Does it still continue or does it hang?

Try to analyze the event log and let us know your findings.