HPE EVA Storage

HP MSA Crash When Adding New Disks

 
SOLVED
Go to solution
Cossy
Occasional Advisor

HP MSA Crash When Adding New Disks


Hi Gents,

I have a HP MSA 1000 disk array with 9 x 146GB SCSi drives configured as a single LUN using RAID 5. The drive in bay 10 is configured as a spare.

We recently decided to populate the remaining 4 spare bays(11-14) with a further 4 x 300GB disks in what was going to be another RAID 5 configuration to yield a further 900GB of disk space.

I assumed that this would be a zero risk activity as the unit should be hot swappable.

I added the disks one after another and the whole thing went to pot.

Drives 8, 9 & 10 went bad and the LUN that was supporting several production VMWare based servers stopped responding.

We rebooted the disk array and after entering the 'accept' cli command the LUN rebuit parity and eventually came up.

One of the two redundant array controllers went bad and is still down. Not sure what happened to it.

So, does anyone have any ideas as to what on earth happened?

Is it ok to 'hot' add disks to the spare bays in the array? Should this have worked?

Where do I go from here? The disk array is under maintenance so I can get the RAID conntroller replaced but how do I populate the remaining disks?

Should you always switch an MSA off when changing the configuration?

Appreciate any help/advice you can give.

Cossy
8 REPLIES 8
Johan Guldmyr
Honored Contributor
Solution

Re: HP MSA Crash When Adding New Disks

Ouch!

What firmware are you running on it? Any later out there with fixes to something like this?
Are there 14 in this? So that exactly half (the right one) of the disks whent bad? Maybe there is an issue with the backplane..

A show tech_support from the cli (in the front) would be interesting.
gregersenj
Honored Contributor

Re: HP MSA Crash When Adding New Disks

There's allways a risk.

It should work perfectly well. And what went wrong can be a theory only.
Possibly, your existing hasn't been properly seatet.
A common mistake is, when you install the drives, is to seat it closing the leaver.
That won't do. Once the leaver has been closed, push hard on the drive (I usely put my thumb on the LED cover, and push hard).
If the drives havn't been thoroughly seatet, and if the backplane has been movng slightly inwards, when you install new disks, that might have caursed the disk to loose connection.
Or one of the new drives might made a little noise on the bus.
Just theory from my side, based on my experience.

Beware, theres never a zero risk, nor a 100% uptime.

BR
/jag

Accept or Kudo

Cossy
Occasional Advisor

Re: HP MSA Crash When Adding New Disks


Here's the firmware details. Please let me know if you think its out of date. It seems consistent with the age of the kit.

CLI-1> show version
MSA1000 Firmware Revision:
7.10b1200 (SGM0734AL0)

Build Time:
2008-06-04 08:37:52
MSA1000 Hardware Revision:
a [AutoRev: 0x030000]
Fibre Module AutoRev:
0x010000

Box 1a PROLIANT 4L7I DB Rev: 1.86 (SGM0734AL0)
Box 1b PROLIANT 4L7I DT Rev: 1.86 (SGM0734AL0


Cossy
Cossy
Occasional Advisor

Re: HP MSA Crash When Adding New Disks

There are 14 bays. I have 10 populated and four spares.

I simply inserted four disks andthat was it.

Cossy
Joshua Small_2
Valued Contributor

Re: HP MSA Crash When Adding New Disks

Hi,

Do you have automatic disk expansion turned on?
Cossy
Occasional Advisor

Re: HP MSA Crash When Adding New Disks


Not sure where 'Automatic Disk Expansion' is configured.

I would add though that we have two other HP MSA's both running at the same firmware and one of them is running with 300GB disks and the other is running with all 146GB disks.

The only difference between those arrays and the one that I had issues with is that I inserted 300GB disks in to an array that had 10 146GB disks.

Not sure if mixing the disks threw it in to a spin.

Thanks for your feedback.

Cossy
Johan Guldmyr
Honored Contributor
Cossy
Occasional Advisor

Re: HP MSA Crash When Adding New Disks

 

This problem turned out to be related to a bad disk.

 

The supplier took the disk offsite, checked it and experienced similar issues there too.

 

They replaced the drive and it went in no problem.

 

Cossy