P410i array won't rebuild.

[SkyeB]

I have an ML350 G6 with a P410I controller. It has the backup battery installed.

It has 8 ports and 2 arrays configured on 7 disks

RAID5 SAS ARRAY A disk 1:1, 1:2, 1:3, 1:4, 1:5 - disk 1:4 failed. 

RAID1 SATA ARRAY B disk 1:7, 1:8 - good

When disk I:4 failed, I replaced it, but the array did not rebuild. It says it is in Interim REcovery Mode. I am not having trouble accessing the logical disk from the OS (WIN2k8 R2), but I would like to have the fault tolerance back in place.

I have tried 2 new replacement drives, but neither work.  I can see them if I plug them into slot I:6, but not when I plug them into I:4. I:4 continuously reads failed in HPACU There is nothing I can seem to do.

I have tried removing the drive in the acucli, but keep receiving the message that the operation is not suported by the configuration.

I have updated to latest drivers, and firmware - to no avail

Please advise.

[Edwin Eppel]

Hi Skye, was hoping someone else might chime in. I'm also running an ML350 G6 with a raid 5 array and Win2k8R2. We also have a hot spare installed. When a drive showed errors last year, I pulled the bad drive and swapped it with the spare (so the drive with errors was then in the hot spare position and the spare was in the raid 5 array). The LED on the replaced drive should then blink indicating that the rebuild process is in progress. It can take a full day for the drive to rebuild depending upon the capacity. The rebuild process was automatic and didn't require any intervention on my part. After I purchased a new spare, I replaced the drive in the hot spare position, but the system would not recognize the new drive and HPACU did not show any change. I ended up having to cold boot the system. It wasn't until I shut the server down and then restarted it that HPACU recognized the new drive. It did not recognize it as a new spare though. I had to remove the drive from the list and then add it again as a hot spare to get the system back to its original state. I hope that is of some help.

When you replaced the drive, what did the drive LED show? Have you had the chance to take the server offline and reboot?

[Jimmy Vance]

---

@Edwin Eppel wrote:

Hi Skye, was hoping someone else might chime in. I'm also running an ML350 G6 with a raid 5 array and Win2k8R2. We also have a hot spare installed. When a drive showed errors last year, I pulled the bad drive and swapped it with the spare (so the drive with errors was then in the hot spare position and the spare was in the raid 5 array).

---

While I don't have an answer for the original poster. I do have a comment on your procedure.  When you have a drive configured as a hot spare, the controller should automatically use it to replace a failed drive. You shouldn't move the drive positions around.

[SkyeB]

Thanks for the reply.

I have powered down multiple times. I replaced the failed drive online, to no avail. I then tried replacing it offline.  No difference. The controller is preventing me from making any changes to the degraded array.

It appears that the array thinks the failed drive is still installed. The new drive won't spin up when placed in the failed drive's location. It spins up fine, and functions in other locations without issues.

[Jimmy Vance]

so it could be a backplane issue.  The array information is stored on each disk conencted to the controller.  One test you could do would be to power of, unplug each disk (you don't have to pull them completely out, just enough they are not making electrical contact) and insert a new disk in teh effected slot and see if it spins up and/or if the array utilites recognise the new disk.

[Edwin Eppel]

---

@Jimmy Vance wrote:

---

@Edwin Eppel wrote:

Hi Skye, was hoping someone else might chime in. I'm also running an ML350 G6 with a raid 5 array and Win2k8R2. We also have a hot spare installed. When a drive showed errors last year, I pulled the bad drive and swapped it with the spare (so the drive with errors was then in the hot spare position and the spare was in the raid 5 array).

---

While I don't have an answer for the original poster. I do have a comment on your procedure.  When you have a drive configured as a hot spare, the controller should automatically use it to replace a failed drive. You shouldn't move the drive positions around.

---

Jimmy, the reason I did this was that the drive did not fail completely. Logs reported an error and stated that the error count threshold on the drive had been exceeded, but the drive was still in use and showing green in HPACU.. The hot spare was still sitting idle. Rather than wait for the drive to fail completely, I opted to replace it with the spare and ordered a new spare. Not the conventional way of doing things but I was being proactive. ;)

As to the poster, I agree, if none of the drives are spinning up in bay 4 that would indicate a back plane issue.

[SkyeB]

So, I have inserted a couple of different disks in the failed drive's slot I:4 - and none of the drives spin up.  I will look into the possibility of a backplane issue.

[SkyeB · Accepted solution]

Backplane replaced. No change.

When I took the server apart, I noticed that the SAS cable used to connect the lower half of the backplane was different than the top cable. It split off to the tape drive.  I replaced the cable with one that came with the new backplane, and powered on the server.

The array immediately began rebuilding itself.

Thanks for the help.