Proliant ML370 G3 RAID Re-building

[Ted Tate]

One of the drives on our main server failed recently so we need to replace it with a new drive. The drive arrived from HP and we have made a good backup of the system just in case. The system uses a Smart Array 6400 controller and hotswappable SCSI drives.

I'm mostly wondering if there is anything I have to do with the server after I insert the new drive? Also how long will it take to mirror roughly 100gigs of data (the drives are 146.8GB 10K RPM drives) onto the new drive? Lastly should I change the rebuild priority to maybe medium or high since I will be doing this after hours when theoretically there should be no I/O requests from other machines?

[Andrzej Kowalik · Accepted solution]

From SA6400 user guide:

"Automatic data recovery (rebuild)

When you replace a hard drive in an array, the controller uses the fault-tolerance
information on the remaining drives in the array to reconstruct the missing data
(the data that was originally on the replaced drive) and write it to the replacement drive. This process is called automatic data recovery, or rebuild. If fault tolerance is compromised, this data cannot be reconstructed and is likely to be permanently lost.
If another drive in the array fails while fault tolerance is unavailable during
rebuild, a fatal system error may occur, and all data on the array is then lost. In
exceptional cases, however, failure of another drive need not lead to a fatal
system error.
These exceptions include:
- Failure after activation of a spare drive
- Failure of a drive that is not mirrored to any other failed drives (in a RAID
1+0 configuration)
- Failure of a second drive in a RAID ADG configuration

Time required for a rebuild

The time required for a rebuild varies considerably, depending on several factors:
- The priority that the rebuild is given over normal I/O operations (you can
change the priority setting by using ACU)
- The amount of I/O activity during the rebuild operation
- The rotational speed of the hard drives
- The availability of drive cache
- The brand, model, and age of the drives
- The amount of unused capacity on the drives
- The number of drives in the array (for RAID 5 and RAID ADG).

Allow approximately 15 minutes per gigabyte for the rebuild process to be completed. This figure is conservative, and newer drive models usually require less time to rebuild.
System performance is affected during the rebuild, and the system is unprotected
against further drive failure until the rebuild has finished. Therefore, replace
drives during periods of low activity when possible.

CAUTION: If the Online LED of the replacement drive stops blinking and the amber Fault LED glows, or if other drive LEDs in the array go out, the replacement drive has failed and is producing unrecoverable disk errors. Remove and replace the failed replacement drive.
When automatic data recovery has finished, the Online LED of the replacement drive stops blinking and begins to glow steadily."

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00574141/c00574141.pdf

regards,
Andrzej Kowalik

[Ted Tate]

Thanks alot that was the user manual that I needed (I've been looking at the ACU manual which isn't very detailed). If anyone has experience with how re-building went for them I would like to here it. The drives are Ultra320 SCSI so I'm hoping the re-build should be able to finish over night. I'm going to set medium priority just incase there is some latent server traffic that will slow the re-build because it shouldn't be important in the middle of the night (probably just email left open).