Safe Disk Firmware Upgrade on a P2000 G3?

[phlegmer]

Looking for advice on how to upgrade the disk firmware on a P2000.  What I thought would be a safe way to do this was to first get all the data off of the vdisk by migrating them to a different vdisk on the MSA.  Once it's all clear, go through the vdisk's drives and upgrade their firmware via the MSA's Update Firmware section using the .fla file download.  However I've noticed that even though I'm not touching any of the other vdisks on the MSA, I'm always seeing disconnects of the other vdisks during the upgrade.  Even if I'm doing the upgrade using controller A and the other vdisks are using controller B, still getting disconnects. 

Fortunately they are quick and things seem to recover, but I'd rather have a much safer method to get the drives flashed without the worry of disconnections.

Please advise and thanks!

[Torsten.]

the official recommendation is to stop all i/o to the array when upgrading disk firmware. not so nice ...

from a randomly selected disk firmware page:

http://h20566.www2.hpe.com/hpsc/swd/public/detail?sp4ts.oid=4208988&swItemId=MTX_d528065c2cb44087bc6c2bef43&swEnvOid=4064#tab6

...

* RECOMMENDED * Firmware Flash Component for Linux - HP MSA 1040/2040 and P2000 G3 2TB, 3TB and 4TB 6G SAS 7.2K rpm LFF Drive Models ( HUS724020ALS640, HUS724030ALS640 and HUS724040ALS640 )

NOTE: Disk drive upgrades on the HP MSA 1040/2040 and P2000 G3 is an offline process. All host and array I/O must be stopped prior to the upgrade. For more information, see the HP Modular Storage Array Reference Guides. All firmware flash progress messages are logged to /var/cpq/MSA2000_IP.log and flash summary is logged to /var/cpq/Component.log.

[phlegmer]

Yeah, I did read that as well.  Seems super odd that if there are critical HDD firmware to apply, one has to stop all production in order to do so.  I have applied FW updates before on a live system.  Get lots of "Ophaned Data" alerts from the MSA and vSphere hicups a bit.  But it is short lived.  Rather not have to do it like that again unless I must.

In my mind, it would seem logical as long as you get all the data off the vDisk, one should be able to do whatever they want to the disks without it affecting the rest of the MSA.

Guess not.  :(

Thanks

[Torsten.]

Updating disks in a MSA is a sad story ... you need to get each file one by one and put your array offline ... :-(

Other arrays (old EVA and new 3PAR) can do this much better.

However, I already did this update online too, but on a single disk at a time only. When selecting more disks, then the update sequence will be too fast and create a mess resulting in unavailable data.

[phlegmer]

Ahh, that makes sense.  When I did just 1 drive of the vDisk, I didn't really see much in vSphere happen.  Just the Orphaned Data e-mail from the controller.  Wasn't until I did the rest of that array did I noticed some issues in vSphere.  Fortunately vSphere seems to be pretty resiliant to drops like that.

I might have to resort to 1 drive at a time like you said and pick the quietest time possible. 

Thanks!

[Torsten.]

I do not expect a change at least for the P2000 (this is G3), but hopefully seagate (dothill formerly) as the OEM will do the homework and improve this process for newer generations like MSA2040  ... :-)

A procedure like used on EVA (use a single bundle with all firmware for manual online upgrade) or better like 3PAR (firmware will maintain the disk firmware automatically) would be ideal ...

By the way, the 3PAR set the disk in question in logging mode, that means more or less offline, reads and writes are going to the other disks in the disk set, update the firmware, put the disk online again and sync the data from the logging drives; even a replaced disk will be adjusted automatically - sounds perfect.

[Dennis Handly]

> 3PAR set the disk in question in logging mode, that means more or less offline ...

You still need to be careful not to upgrade too many PDs at the same time.