1748171 Members
4219 Online
108758 Solutions
New Discussion юеВ

Re: Array rebuild

 
Engineer2
Occasional Advisor

Array rebuild

One of our servers has a problem with the discs. One drive has totally failed, and two more are in a predictive fail state.  The server is configured RAID 5.

I got new discs shipped out by HP which arrived today. I swapped out the failed drive with one of the new drives, but I cannot see in the array config utility if it is actually rebuilding the array. It still registeres the disc as failed. However, when I plugged the new disc in, all the discs (including the new one) lit up blue.

About 15 minutes later, only disc 3 was blue. Another few minutes after that, all the discs went blue again. I was under the impression the array should rebuild automatically when the failed disc was replaced?  It shows in the configuration utility as being in "interim recovery mode" but other than that, I cannot see a way to confirm if the array is rebuilding.

The server in question is an HP Proliant DL380 Gen 8.

Can anyone offer any advice? Thanks

15 REPLIES 15
parnassus
Honored Contributor

Re: Array rebuild

What exact HPE Smart Array RAID Controller your Server is equipped with (just to have a complete picture to start from)?


I'm not an HPE Employee
Kudos and Accepted Solution banner
Engineer2
Occasional Advisor

Re: Array rebuild

Hi, its an HP Smart Array P420i array controller.

Thanks

parnassus
Honored Contributor

Re: Array rebuild

Supposing Array member disks are (included the failed one and the new one you ordered as replacement disk) Hot-Plug capable (and all with the correct size, especially the ordered one as replacement), was the swap failed disk->out | replacement-new disk->in happened online or offline (with Server powered off)?

Read (from HPE Smart Array Controllers User Guide) under Replacing drives paragraph:

For systems that support hot-pluggable drives, if you replace a failed drive that belongs to a fault-tolerant configuration while the system power is on, all drive activity in the array pauses for 1 or 2 seconds while the new drive is initializing. When the drive is ready, data recovery to the replacement drive begins automatically.
 
If you replace a drive belonging to a fault-tolerant configuration while the system power is off, a POST message appears when the system is next powered up. This message prompts you to press the F1 key to start automatic data recovery. If you do not enable automatic data recovery, the logical volume remains in a ready-to-recover condition and the same POST message appears whenever the system is restarted.

I'm not an HPE Employee
Kudos and Accepted Solution banner
Engineer2
Occasional Advisor

Re: Array rebuild

Hi, the server was power on and running when the swap-over occurred. It was not powered down. I took out the disc that has failed completely and simply replaced it with one of the new discs from HP.

parnassus
Honored Contributor

Re: Array rebuild

Isn't Automatic Data Recovery enabled? Automatic Data Recovery should be enabled by default so the HPE Smart Array RAID Controller will start the rebuild automatically (you just swapped the failed drive with the new one without changing the disk bay where the failed one initially was):

When you replace a drive in an array, the controller uses the fault-tolerance information on the remaining drives in the array to reconstruct the missing data (the data that was originally on the replaced failed drive) and then write the data to the replacement drive
 
so basically if the type of disk is coherent (SATA with SATA, SAS with SAS, SSD with SSD) with disks type used on the Array (SATA, SAS or SSD) without any mix and the disk capacity requirement is fulfilled (replacement disk size should be equal to or greater than the size of the failed drive) the rebuild should start automatically if ADR is left enabled.
 
The logical drive will remain in a "Ready to recover" condition and the same status will be displayed at the next system restart until the rebuild happens (automatically or invoked manually).
 
Could be a physical drive Firmware issue (need to be updated and the server rebooted?) if you see Disk Locate LED flashing blue (solid blue means the disk is recognized).
HP_ProLiant_Gen8_Hot-Plug_disk_LEDs.png

 

 

What the HPE Smart Storage Administrator appliacation reports?

I'm not an HPE Employee
Kudos and Accepted Solution banner
Engineer2
Occasional Advisor

Re: Array rebuild

Where can I check if ADR is enabled?

TTr
Honored Contributor

Re: Array rebuild

What is the status of the logical disk in the array? Are they rebuilding or still degraded?

You may have to set the unused drive as a "auto-replace" hot spare in order for it to be used to rebuild the array.

Engineer2
Occasional Advisor

Re: Array rebuild

The disc which I replaced is still showing as failed in the Storage Administration utility. I cannot see anywhere it indicates if it's rebuilding the array or not.

The other three discs; two are still in predictive fail mode, while the third disc is all ok.

Because the other discs haven't completely failed yet I am sure the array can still be rebuilt on to the new discs.
TTr
Honored Contributor

Re: Array rebuild

So the new disk does not show up as unused/unassigned? What does it show up as?