Our DL360G3 is configured for RAID1+0 using 2 36Gb drives. When one drives failes during operation and I hotplug a new drive while running Windows 2003, the OS completely freezes, only the mouse pointer works but nothing else. Only turning the server off and on again helps. After that, the bios of the 5i raid controller (version 2.56) recognizes the situation and aks if it must rebuild the mirror. Likewise when removing a drive while windows 2003 is running results in an OS freezes. After a reset, the beos recognizes the situation again and asks for interim recovery and then boots the OS. Why doesn't this work while running windows 2003 ? This way the mirror configuration is of no use, since continues operation of the server is not guaranteed since you have to hard reset the server. I've installed the server from Smartstart 6.40.
Hello Franc, Even though the hard drives are hot pluggable I always recommend bringing the server down and then replace the failed drive. It's just out of the habit and a similar experience likes yours that I suggest replacing the HDD when the server is off. In RAID 5 I have not experienced the same situation though. I hope this answers your dilema.
Hello, ok I understand. Problem however is, that when the drive fails, the complete OS freezes and I can't shutdown the OS gracefully and have to turn powercycle the server. However, I simulated the failure by pulling the drive out. Is this situation different than when a real failure occurs? In other words, when the drive fails for real will the system be able to recover from that easier then when pulling a drive out ?
my first reaction was the same as yours. But yes it's possible. The raid configuration utility reports the array as a RAID 0+1. From the help file of the ACU:
----------------
RAID stands for Redundant Array of Inexpensive Disks. RAID 1+0 (drive mirroring) is a fault tolerance method that uses 50 percent of drive storage capacity to provide greater data reliability by storing a duplicate of all user data. Half the physical drives in the array are duplicated or "mirrored" by the other half.
RAID 1+0 first stripes your data across half of the disks, then mirrors this data to the other half.
Drive mirroring creates fault tolerance by storing two sets of duplicate data on a pair of disk drives. There must be an even number of drives for RAID 1+0. This is the most costly fault tolerance method.
If a drive fails, the mirror drive provides a backup copy of the files and normal system operations are not interrupted. The mirroring feature requires a minimum of two drives, and in a multiple drive configuration (four or more drives), mirroring can withstand multiple simultaneous drive failures as long as the failed drives are not mirrored to each other.
---------------
I never heard of it before, but it seems possible.
Hello Franc, Just a word of caution do not unplug the working HDD from the system while it's up and running. I can quote you tons of examples as to what happened when some other clients pulled the good HDD out of the system. I'm surprised you did not end up building the OS altogether. The HDDs are hot pluggable and NOT hot unpluggable. If the system froze when you removed the live HDD then it's normal that you experience the OS halting.
finally I got it. You make a cross with the data. When you removed the disk, was the disk actually marked as bad or still running? Controllers don't like it, when you abruptly remove a disk, which is under load. Once I caused a reset with this on a machine with hundreds of user on it. :-(
thanks for your clarification. Guess I was under the assumption that you can pull the drive while it's running. But is there a safe way to fail a drive and simulate the recovery process ? Is there some sort of utility to do it ? I need to test the entire process before I can take the server into production.
Hello Franc, You are not the only one thinking that you can pull the HDD from a functioning server when it has not failed. The word Hot Pluggable has a lot to say about it. But it really hits hard when we pull the drive out when it has not failed. You can short circuit the back plane board as well. I am letting you know as per my personal experience I had with lots of clients in these unfortunate situations. As for as I know we do not have any simulation or utility to just fail the HDD and then see the behaviour of the server. Sorry but this is the bitter truth.
ok, thanks for your prompt answers. Indeed most people think you can plug and unplug the drives while it's running only because of the word 'hot pluggable'. I've asked several people and they all say, sure it should work...
About the simulation, well I have to take HP's word then that the controller functions as expected (but that shouldn't be too difficult) and we'll see what happens when the moment is there (if we ever get to that moment).
I've had the same problem here with DL360G3's running Windows 2000. I've powered the server down, removed the hdd in slot 1 then powered the server up. So you would think the controller thinks the hdd has failed. It runs fine. I then plug a brand new drive into slot 1 and the server freezes up. All disk activity stops on hdd 0 and hdd 1 shows the red led. I have to power down. When the server comes back up the array rebuilds and everything is good. I've tested this on 3 different G3's and get the same result. I'm willing to bet if I do have a hdd failure I'm going to have to take the server down to replace it. Not cool!
now you mention it. Same experience. If the server is powered off I remove the drive. Turn on the server, bios reports interim recovery mode since one drive is missing. Pressing F1 boots Windows 2003 fine. Now, I plug-in the replacement dirve, then the OS freezes comletely. Resetting the server, bios reports Data Recovery Mode because second drive is detected, Windows 2003 boots fine and the mirror is being rebuilt. So hot plugging a drive while the controller is in interim recovery mode doesn't work also. In fact, hot-plugging doesn't work at all when windows 2003 is up and running.
Hey Franc. I even called HP support and told them what I was experiencing. They pretty much told me if a drive did fail I would be able to plug a new drive in without rebooting. They also said when I turn the box off, pull the drive, and power back on that's not a valid test because the drive didn't actually fail. I guess the controller is smart enough to know that I physically pulled the drive out! I also tested the same thing on a DL360G1 and DL360G2. Both worked fine as far as being able to hot plug the drive and rebuild w/out rebooting and w/out the O/S locking up. The G2 has the same 5i controller as the G3 so it must be a problem with the G3. I have a feeling this message thread will fill up as soon as more people experience hdd failures. Doug.
Franc. One more question. When you replaced your HDD was it because of a HDD failure or were you testing DR?
Also I've tested powering down a DL360G1 w/ Ultra320 drives, removing disk1, powering up, F2, and plugging the drive back in. No lock up and the array rebuilds. So it can't be a problem with the Ultra320 drives. Doug
I pulled the drive when the server was powered down. Not an actual failure though. But hey, it surely is possible that a drive can fail when the server is powered off (bad power or whatever). If that's the case how will the controller tell the difference then how the drive failed ?
Magic! They put little camaras to tell if you actually pulled the drive. I think it's an engineering problem and I hope HP is looking into it. If it truely does work in the case of a hdd failure then I'll stand corrected. Unfortunatly there is no way to know for sure until a hdd does fail. I wouldn't put my name on it!
