MSA8000:Replacing mirror dusk

[Learn_1]

Hi,
we have alpha server GS80 running OpenVMS7.3-1. and connected with MSA8000.System disk is on the msa8000 and is mirrored.one of the mirror disk has failed and i want to replace the failed mirrored disk with new one.can anyone suggest the procedure to replace the faulty disk so that both disks synch with each other.the controller is HSG80.

[Veli Körkkö]

So, you have MA8000 so to speak. The basic procedure to do is:



HSG> show failedset
note the failed disk

HSG> show DISKxxxxx
note the details, esp the PTL info

HSG> delete failedset DISKxxxxx
HSG> delete DISKxxxxx
HSG> locate all

now we should have all OTHER disks orange led blinking slowly except the bad disk


quiesce approriarite bus by pressing momentarily the channel but on HSG80 controller (does not matter if lower or upper)

once all (other?) disks on the channel in question had orange blinks somewhat fast, you can remove the disk

wait some 30 seconds

quiesce the channel again and once orange leds blinking fast again, insert new disk


wait some 30 seconds

HSG> add disk DISKxxxxx nn mm 0
HSG> initialize DISKxxxxx

now next step depends how the mirrorset is configured and whether you had any HSP or not

suppose you did not have any HSP (Hot SPare)
we have now one particualr mirrorset with one less than nominal members

if said mirrorset is set BEST_PERFORMANCE or BEST_FIT policy, simply

add spare DISKxxxxx

if not, one manual replacement to mirror set
via

set mirrsetname replace=DISKxxxxx

If you had on the other hand HSP, then probably disk has been rebuilt to mirrorset
and you MAY want to "fix" the config back to original, e.q

set mirrsetname nopolicy
set mirrosetname members=3
set mirrorsetname replace=DISKxxxxx
show mirrorsetname

wait until the disk has been fully rebuilt and then remove the disk that originally was HSP using e.q.

reduce DISKyyyyy
set mirrorsetname policy=best_performance
(or best_fi)
add spare DISKyyyyy


Of course one could also set FAILEDSET to autospare but I'd rather not.

Hope this helps,


_veli

[Learn_1]

Hi veli,
thanks for the detailed reply.this really helped to understand the procedure.
let me eplain few point and then i will write down the exact procedure that i will be following to replace a failed mirrorset member disk.
the mirrorset-name is M1.
this mirrorset consists of 2 disks i.e disk10000 & disk30000.disk10000 has failed and now system isfunctioning from disk30000.the failed disk is shown as failedset.no spare disk is configured in mirrorset and no plan to configure any spare disk in mirrorset.
one thing tha i want to have eplaination is about quiescing the controller.how can we do this and what is the function of this?
..........Procedure...........
1.show failedset
2.show disk10000 (noting the PTL)
3.delete failedset disk10000
4.locate all
5.add disk Disk10000
quiescing the controller and replacing a new disk in place of faulty disk i.e disk10000
6.init disk10000
7.set M1 nopolicy
8.set M1 replace=disk10000 (copy operation will start automatically)
9.show M1
10. set M1 policy=best_performance

correct me where i am wrong.
Your Points are due.
cheers

Anwar.

[Rob Buxton]

I do not agree with the procedure outlined.
We're not running VMS on the SAN, but we do have an RA8000 and EMA12000 al lwith HSG80 Controllers.

Show Faildset will show the disk.
It is useful to know the Disk ID and PTL from a show disk.

There is no way that I know of to quiesce an HSG80 Bus.
Just pull the failed disk and replace with a good disk. The HSG80 should do the rest.
The disk should then move from the failedset to the Mirror and rebuild.

You may want to consider obtaining another disk and configuring it as a Spareset. With this the HSG automatically replaces the failed disk and moves the failed disk to the failedset. When you replace that disk it moves it to the Spareset again. Reduces the amount of time your array is exposed.

[Mike Naime]

You need a pen, or other small sharp object in order to press the small BUS buttons on the HSG80 controller itself.

IN order to quies the bus, you press the button of the bus that you want to quies. You can press the button on either controller to do this. you then have a bout a minute before drive activity resumes on that SCSI bus.

[Uwe Zessin]

Anwar,

as Mike has already said, you don't quiesce the whole controller - just the SCSI bus where the failed disk is located. That procedure is described on page 2-8 of the 'HSG80 Version 8.7 Maintenance and Service Guide' EK-G80MS-SA-B01. You can download it from:

ftp://ftp.compaq.com/pub/products/storageworks/techdoc/acs/EK-G80MS-SA-B01.pdf

This manual and other useful ones are available from:

http://h18000.www1.hp.com/products/storageworks/acs/g80ovms.html

I have also heard mixed responses of how to replace a failed disk. Some say you have to quiesce the bus, some say that 'asynchronous hot swap' is supported.

There seems to be a small misunderstanding regarding spare disks: you cannot assign a spare disk to a mirrorset - spare disks are always global within the entire array.


I don't think that steps 4 and 5 make sense here.

4) Will just let all fault LEDs blink (that is a good test to see if they work and should be done to see if they work - I had a disk drive enclosure where one bay wasn't assembled with any LEDs at all).

5) Does not make sense, because 'DELETE FAILEDSET DISK10000' will remove the entry from the FAILEDSET - it will not delete DISK10000 from the configuration.

7,8+10) As you don't have any spare disks and assuming you have no other reduced storagesets you can do 'ADD SPARESET DISK10000' instead and the HSG80 will automatically add the disk to the mirrorset.

[Jan van den Ende]

Anwar,

would you PLEASE add a reply-entry to

Re: Blocking intrusion detection

so I can give you your well-earned points?


Jan

[Rob Buxton]

Oopps missed those buttons.
But, I've never quiesced the bus on our HSG80s.
I've also never deleted any devices/units in order to replace a failed disk.
Simply removing the failed disk and add the new disk has been all that I have needed to do.
The only caveat is that we do have a spareset configured. The Spareset gets used to repair the broken Raid/Mirror and the failed disk goes to a failedset.
Replacing the failed disk results in the FailedSet being cleared and the new disks ends up in the SpareSet with no interaction, disk/unit deletion required.

Does this not happen if there is no spareset?

[Martin P.J. Zinser]

Hi,

one more caveat, quiescing the bus will do exactly that ;-), i.e. if the system performs write I/O s to these disks they will appear to hang for the time the bus is kept quiet. Depending on your application load you might want to perform such an action only during off hours...

Greetings, Martin

[Uwe Zessin]

Rob,

if you check your FAILEDSET, I am sure you will see that it has AUTOSPARE enabled. Many storage managers don't like it, because it will cause the spare disks to 'move around' and to undo its effect is additional work. AUTOSPARE will only work if there is no meta data on the replacement disk, BTW.


> Does this not happen if there is no spareset?

There is always a SPARESET, but on some arrays it is empty ;-)
AUTOSPARE is a switch on the FAILEDSET, not the SPARESET.

[Mike Naime]

While Veli didi a rather in-depth explanation of his procedure, I also do not totaly agree with his methods of getting the same results.

I have 6 spareset drives on all of my HSG controllers. (MA and EMA styles) One in each SCSI Bus/Channel/Shelf.

You must ensure that your spareset drives are as large as the largest spindle in the storage controller that you want to have a drive autospared for. I.E. IF you have 18's, 36's, and 72's in an HSG, make sure that you have some 72's (If not all) in your spareset.

If you INITialize the drive prior to placing it into the spareset, it will autospare in for a smaller drive if it fails in the same channel.

Make sure that you are patched above 8.6F-0 There where some patches between -0 and -8 that addressed autospareing failure issues.


With all that said, here is what I do.

Drive fails ==> Pager alarms. (Wake me up)
Drive Autospared ==> Pager alarms. (Go back to sleep)
Net day, get on the HSG and:
DELETE FAILED DISKXXX00
DELETE DISKXXX00
Pull disk drive.
If I have an extra on my desk to insert.
Wait 30 seconds before inserting new drive.
(Insert new drive)
RUN CONFIG
INIT DISKXXX00
ADD SPARE DISKXXX00

(Log call to HP for replacement)
(Update storage docs with new configuration for that HSG)

We do not care to try and reset the storage sets back to the original configuration. Especially with the raidsets. I do not want to be the cause of a toasted LUN because I forced a raidset rebuild and another drive failed during the rebuild time.

[Uwe Zessin]

Well, Mike,

you are living in a luxury world ;-)

Not everybody can afford 6 spare disk drives. It is too bad that the HSG does not allow to RAID-5 over mirrorsets...