topic Re: Queued for Rebuilding - MSA 1500CS + MSA20 in Disk Enclosures

Queued for Rebuilding - MSA 1500CS + MSA20

Philipp Deneu — Sat, 29 May 2010 10:07:48 GMT

Hello,

we ve got a MSA 1500CS with MSA20 SATA Array.
One Month ago there was one disk broken and we changed it out. After that the MSA startet for rebuild the spare disk to the new disk.
All leds are lighting normal in front of the msa20 but on web user interface the msa shows following status alerts:
(Queued for Rebuilding) Logical Drive 1 (2086667 MB, RAID 5) is queued for rebuilding.
Mapped Logical Drive 1 (lun1t1, Logical Drive 1) is queued for rebuilding.

After restarting the msa the message comes again and in the display auf the msa1500cs there stands following:
106 Volume#0 State waiting to rebuild
112 Volume#0 Rebuild failure
151 Beacon off

What does this mean.
Is it possible that the msa rebuilded the disks rightly but there is still another rebuild process in queue?
Can i solve this issue anyway?

Thanks,
Kind Regards,
Philipp

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Clarete Riana — Mon, 31 May 2010 04:36:20 GMT

After the reset, what is the state of the MSA logical drive 1 as shown in thw web interface? How many drives do you have in the RAID 5(logical drive1)?

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Philipp Deneu — Mon, 31 May 2010 15:21:07 GMT

after reset, it still shows rebuilding.
Weve got one array with one logical drive (2tb). Is there any options to show the command or process queue from cli?

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Clarete Riana — Tue, 01 Jun 2010 13:08:24 GMT

Hi,

There is a SHOW UNIT command available in the CLI that displays the status of the unit. There is also a SET GLOBAL command that can be used to set the priority of rebuild operations in relation to input/output operations. You can view the current settings using the SHOW GLOBALS command.Below is an explanation on each of the options available for rebuild_priority.

*lowâ expansion or rebuild takes place only when the array controller is not busy handling normal I/O requests. This setting has minimal effect on normal I/O operations. However, there is an increased risk that data will be lost if another physical drive fails while the rebuild is in progress.

* mediumâ expansion or rebuild occurs for half of the time, and normal I/O requests are handled during the rest of the time.

* Highâ rebuild or expansion occurs at the expense of normal I/O operations. Although system performance is affected, this setting provides better data protection because the array is vulnerable to additional drive failures for a shorter time.

Refer to the HP storage works MSA1000-1500 CLI user guide( link below) for more information on these commands.

http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c01183955/c01183955.pdf

Regards,
Clarete

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Philipp Deneu — Tue, 01 Jun 2010 16:28:01 GMT

Hey Clarete,

here are the values form the commands.
CLI-1> SHOW UNIT

Unit 0:
Unit Identifier :
Device Identifier : 600508B3-00926840-A00016A5-A94C0001
Preferred Path : Controller 1 (this controller)
Cache Status : Enabled
Max Boot Partition: Disabled
Volume Status : VOLUME NEEDS RECOVER
Parity Init Status: Complete
7 Data Disk(s) used by lun 0:
Disk501: Box 5, Bay 01, (B:T:L 0:04:01) REPLACEMENT
Disk502: Box 5, Bay 02, (B:T:L 0:04:02)
Disk503: Box 5, Bay 03, (B:T:L 0:04:03)
Disk504: Box 5, Bay 04, (B:T:L 0:04:04)
Disk505: Box 5, Bay 05, (B:T:L 0:04:05)
Disk506: Box 5, Bay 06, (B:T:L 0:04:06)
Disk507: Box 5, Bay 07, (B:T:L 0:04:07)
Spare Disk(s) used by lun 0:
Disk508: Box 5, Bay 08, (B:T:L 0:04:08)
Logical Volume Raid Level: DISTRIBUTED PARITY FAULT TOLERANCE (RAID 5)
stripe_size=16kB
Logical Volume Capacity : 2,086,667MB

CLI-1> show globals

Global Parameters:
System Name: storagemsa01
Rebuild Priority: medium
Expand Priority: medium
Surface Delay: 3.0 seconds

Total Cache: 256MB
50% Read Cache: 128MB
50% Write Cache: 128MB
2 cache lines unable to flush!

Temperature:
EMU: 25 Celsius, 77 Fahrenheit
PS1: 26 Celsius, 78 Fahrenheit
PS2: 26 Celsius, 78 Fahrenheit

Disk 501 is the disk which we replaced.
For me it looks like that he cant rebuild it.
Whats your idea? Setting the rebuild priorty to high?

Kind regrads,
Philip

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Clarete Riana — Wed, 02 Jun 2010 04:57:27 GMT

Looks like the recovery has not started at all. If it had, the status of the UNIT will be VOLUME RECOVERING and not VOLUME NEEDS RECOVER. Before you change the rebuild priority, do you have any I/O at all targeted to this Unit ( In other words, Is this logical volume in use?) If it is not, then do not change the rebuild priority. It only makes a difference if I/Os are in progress.

The other suggestion is to check the firmware version on the controller(SHOW VERSION) and see if it is the latest.

It may also be a good idea to check if the replaced disk is detected by the controller or not. SHOW DISK command output would help.

Regards,
Clarete

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Philipp Deneu — Wed, 02 Jun 2010 05:03:24 GMT

Hello,
so you would say that i can change the priority level anytime)
There is IO on the target. Its mapped to a VMware vSphere Cluster. Should i change the rebuild priorty level during the time its mounted or should unmount it first?

Here my show disks, the disk was discoverd:
CLI-1> show disks
Disk List: (box,bay) (B:T:L) Size Speed Units
Enclosure 5: SATA (0:04:00) PROLIANT 8LCE 2.08
Disk501 (5,01) (0:04:01) 500.0GB 160 MB/s 0
Disk502 (5,02) (0:04:02) 500.0GB 160 MB/s 0
Disk503 (5,03) (0:04:03) 500.0GB 160 MB/s 0
Disk504 (5,04) (0:04:04) 500.0GB 160 MB/s 0
Disk505 (5,05) (0:04:05) 500.0GB 160 MB/s 0
Disk506 (5,06) (0:04:06) 500.0GB 160 MB/s 0
Disk507 (5,07) (0:04:07) 500.0GB 160 MB/s 0
Disk508 (5,08) (0:04:08) 500.0GB 160 MB/s 0 (spare)

Notes:
The speed is the currently negotiated speed to the disk. This may
be less than the maximum speed supported by the device due to bus
faults, loss of signal integrity, etc.

And here is show versions, but it shows me a 1510i and not my 1500cs like on the chassis:
CLI-1> show version
MSA1510i Firmware Revision: 2.02b100 (SGA061400Y) [release code]
Build Time: 2008-11-17 12:07:19
MSA1510i Hardware Revision: a [AutoRev: 0x020000]
MSA1510i Processor Revision: 2

iSCSI Daughterboard Primary Firmware Revision: v1.0.259 (active)
iSCSI Daughterboard Backup Firmware Revision: b1.0.0121

iSCSI Driver Revision: 259

SCSI I/O Module Bus 0 Revision: 2.02
SCSI I/O Module Bus 1 Revision: <>
SCSI I/O Module Bus 2 Revision: <>
SCSI I/O Module Bus 3 Revision: <>
Fan Control Module A Revision: 2.38
Fan Control Module B Revision: 2.38
Box 5 PROLIANT 8LCE Rev: 2.08 (E050MLJ15P)
Box 5 Fan Control Module A Rev: 2.36
Box 5 Fan Control Module B Rev: 2.36

you see any problems?

Kind regards,
Philipp

Ps.: we had the crashes of the msa, is it poosbile that this is one reason for it?

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Clarete Riana — Wed, 02 Jun 2010 09:35:03 GMT

MSA 1510i is an iSCSI interface based MSA controller whereas MSA1500Cs is FC based. Are you using the iSCSI interface on Windows?

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Philipp Deneu — Wed, 02 Jun 2010 10:06:10 GMT

Yes its only iSCSI no FibreChannel.

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Philipp Deneu — Wed, 02 Jun 2010 10:13:15 GMT

Sorry we are using iscsi with vmware...

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Clarete Riana — Wed, 02 Jun 2010 10:55:18 GMT

I would suggest you to report the problem to HP.

Regards,
Clarete

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Clarete Riana — Mon, 07 Jun 2010 04:51:21 GMT

Did you have any luck with the rebuild?

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Philipp Deneu — Mon, 07 Jun 2010 04:53:49 GMT

No, but the Support Engineer told me to replace the disk again.
So now im waiting for the new disk and i hope it will arrive today.
The rebuild process stops at 2%.

Kind regards.

Re: Queued for Rebuilding - MSA 1500CS + MSA20

Erwin Eichinger — Mon, 16 Aug 2010 12:52:44 GMT

Hi,

i have the exact same problem, did you manage to rebuild the logical drive?

regards