MSA1500 -- Mirrored Volumes

[Maarten Van Parys]

Hello all,

A customer has following setup:
2 MSA1500, 4 servers (W2k Server, W2k3 Server)
The servers have 2 HBA's. 1 HBA of each server is connected with the first MSA1500, the other with the second. On the 2 MSA1500, identical Logical Drives are created and presented to the hosts. This way, the OS of each server sees two identical raw disks. These are mirrored using Windows' Mirrored Volumes.
Problem:
On 2 of the 4 servers, mirrors get broken after a while. Reactivating the failed mirror, restores the mirror for some time, but the problem returns.
What could be the problem?
I really doubt that any hardware is defective. Is this configuration (Mirrored Volume on SAN-device) possible? Is it performance? Is it something else?

Thanks for answering,
Maarten.

[SAKET_5]

Maarten,

Your setup intrigues me! To my mind, you have taken a rather long shot at defining redundancy in your SAN setup.

Why did you think that just having redundant controllers in a MSA1500 is not providing sufficient failure protection? Sure, if you did some analysis and came up to the conclusion that it doesn't provide sufficient insurance around that - then why did you buy an entry level array in the first place? Why not an EVA of some kind?

Maarten, dont get me wrong - I am not having a go at you!...In my experience with a number of MSA1000/1500s - the following things have been reported to cause all sorts of grief:
1. Controller lock up
2. Frequent Controller Failover
3. FW upgrades on Controller

Could you confirm that no failovers are occuring between the individual MSA controllers? I realise that only 2 of your servers are being impacted. Any similarities between these 2 servers and any dissimilarities between these 2 servers that are impacted from the other 2 that arent impacted?

Hate to sound like a support guy here - but are you running the latest firmware, driver, MPIO versions of the components in question?

[Martin Lange]

Hi Maarten,

I made similar experiences with EVA and VA devices with W2K systems. I never had these problems with W2K3 systems. So I guess your troublemakers are the W2K systems too.

[Ivan Ferreira]

Have you already tried this?:

http://support.microsoft.com/kb/329075/en-us

[Maarten Van Parys]

Saket,

I understand your questions about this setup. Im a support guy and not the mastermind behind the design, so I cannot give you much more information on the creation of this concept. All I now is that its supposed to be a cheap Disaster Recovery solution.
In Event Viewer, I dont get any errors on a failing MSAcontroller (or other events generated by HP-agents). I only see following events in the system log:

Event Type: Information
Event Source: dmio
Event Category: None
Event ID: 30
Date: 20/03/2006
Time: 18:09:31
User: N/A
Computer: BEVMAIL01
Description:
dmio: Harddisk1 write error at block 39316799: status 0xc0000185

Event Type: Warning
Event Source: dmio
Event Category: None
Event ID: 4
Date: 20/03/2006
Time: 18:09:31
User: N/A
Computer: BEVMAIL01
Description:
dmio: write error on Plex Volume2-01 of volume Volume2 offset 39316736 length 128

All 4 servers are DL380G3, same hardware configuration. 1 problemserver runs on W2k Server, the other on W2k3 Server. The 2 servers with no problem have W2k3 Server installed. The only difference I can think of (I may forget something) is the load on the servers.
All firmware, drivers, HP software, etc. are the latest version.

Ivan,

For http://support.microsoft.com/kb/329075/en-us, the status code for the event shoud be 0xC000009A, I get 0xc0000185.

[SAKET_5]

Maarten,

Ok, I could think of the following approaches:
1. How about you run the Microsoft Product Reporting Tool (http://www.microsoft.com/downloads/details.aspx?familyid=cebf3c7c-7ca5-408f-88b7-f9c79b7306c0&displaylang=en)
to gather the ins and outs of your SAN configuration on the impacted Windows hosts. This may provide us some pointers.

2.I have a nasty suspicion that may be a hard disk is generating a significant amount of irrecoverable SCSI errors and is possibly about to fail. This would explain what you see at Windows end - freak outs at the disk management layer! Dont forget it is after all Windows that we are dealing with!

3. Could you gather statistics about your disk error information through either the management agents or running the diagnostics utility (ADU perhaps?).

4. On a pro-active note, you could actually map out the mirrored volume which is impacted to the physical hard disks that it relates to and try replacing those hard drives one at a time.

Let me know what your thoughts are.

[Maarten Van Parys]

Hi Saket,

I'm giving up the idea of Mirrored Volumes. I'm thinking about using the 2 controllers in a redundant setup and MIO DSM as multi-path software. If I then connect the 2 storage enclosures, I'll have a (working) redundant system, twice the amount of storage (what to do with it??) at no extra cost (except maybe for extra SCSI-cables).
Is this possible? Or am I forgetting something?